Gen AI Training in Hyderabad Kukatpally

RAG

Here’s the corrected and working version of your snippet to read a PDF, extract text, and split it into chunks using LangChain’s RecursiveCharacterTextSplitter:

######################################

from pypdf import PdfReader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_core.documents import Document

# Load the PDF file
pdf = PdfReader('./pdf-docs/oci-ai-foundations.pdf')

# Extract text from all pages
text = ""
for page in pdf.pages:
text += page.extract_text() # <-- add parentheses to call the method

# Split the text into smaller chunks
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=1000, # you can adjust this value
chunk_overlap=100, # overlap between chunks
separators=["\n\n", "\n", " ", ""]
)

# Step 4: Split the text using the splitter
chunks = text_splitter.split_text(text)

# Create document objects

# Step 5: Convert each text chunk into a LangChain Document object.
# Each Document stores both the text (page_content) and optional metadata (like page number, source, etc.)

docs = []
for i, chunk in enumerate(chunks):
docs.append(Document(
page_content=chunk,
metadata={"source": "oci-ai-foundations.pdf", "chunk_index": i}
))

######################################

from langchain_core.documents import Document
from langchain_core.embeddings import Embeddings
from langchain_community.vectorstores import OracleVectorStore

def store_to_oracle_vector(
chunks: list[str],
source_file: str,
embedding_model: Embeddings,
connection_string: str,
collection_name: str = "oci_ai_docs"
):
"""
Convert text chunks into Documents, embed them, and store them in Oracle 23 AI Vector Store.

Parameters
----------
chunks : list[str]
The list of text chunks extracted from your PDF or other sources.
source_file : str
Name or path of the source document (used in metadata).
embedding_model : Embeddings
Any LangChain-compatible embedding model (e.g., OpenAIEmbeddings, HuggingFaceEmbeddings).
connection_string : str
Oracle Database connection string in the format:
oracle+oracledb://user:password@hostname:port/service_name
collection_name : str, optional
Logical name of the vector collection in Oracle (default "oci_ai_docs").

Returns
-------
OracleVectorStore
The initialized and populated Oracle Vector Store object.
"""

# Step 1: Create Document objects with metadata

# Create an empty list to hold all Document objects
docs = []

# Loop through each text chunk and its index
for i, chunk in enumerate(chunks):
# Create a Document with content and metadata
doc = Document(
page_content=chunk, # the text content of the chunk
metadata={
"source": source_file, # original file name or path
"chunk_index": i, # order of the chunk in the document
"doc_type": "pdf" # type of file (can adjust as needed)
}
)
# Add the Document to the list
docs.append(doc)

# Now 'docs' holds all Document objects

# Step 2: Initialize Oracle Vector Store and store documents
vector_store = OracleVectorStore.from_documents(
documents=docs,
embedding=embedding_model,
collection_name=collection_name,
connection_string=connection_string
)

print(f"✅ Stored {len(docs)} chunks from '{source_file}' into Oracle Vector Store collection '{collection_name}'.")
return vector_store

USAGE

from langchain_openai import OpenAIEmbeddings

# Example: create embeddings and store chunks
embedding_model = OpenAIEmbeddings(model="text-embedding-3-small")

connection = "oracle+oracledb://admin:MyPass@myhost:1521/myservice"

store_to_oracle_vector(
chunks=chunks,
source_file="oci-ai-foundations.pdf",
embedding_model=embedding_model,
connection_string=connection,
collection_name="ai_docs_demo"
)

# Optional: print the first chunk to verify
print(docs[0].page_content)

🧩 Explanation

• PdfReader reads your PDF file.

• extract_text() pulls out all the text from each page.

• RecursiveCharacterTextSplitter divides the big text into smaller chunks that are easier for AI models to process later.

• Document objects store each chunk, which you can later feed into embedding or question-answering models.

LANGCHAIN Basics

 full LangChain basics demo, but this time using Oracle’s OCIChatGenAI instead of OpenAI’s ChatOpenAI.

This shows you how to build:
✅ A simple text invocation
✅ A PromptTemplate with variables
✅ A ChatPromptTemplate for multi-message prompts
✅ A conversation memory chain (ConversationChain)
✅ All powered by OCI Generative AI (Chat)

🧠 Full Example — LangChain + OCIChatGenAI Basics

⚙️ Notes

• Replace compartment_id and service_endpoint with your actual OCI Generative AI setup.

• The model name can be one of:

  • "cohere.command-r-plus"

  • "meta.llama-3-70b-instruct"

  • "mistral.mixtral-8x7b-instruct" (depending on region and availability).

• This works best when your OCI SDK and credentials are configured under ~/.oci/config.

Would you like me to show the modern LCEL version (using the pipe syntax prompt | llm) next? It’s much more compact and ideal for RAG pipelines.

gen1.py