How benitomartin/agentic-rag-langchain-pinecone works
Project Type and Purpose
• This is a research‐oriented Retrieval‐Augmented Generation (RAG) pipeline implemented in Python via a Jupyter Notebook.
• It uses LangChain to orchestrate document ingestion, chunking, embedding, vector storage (Pinecone), and a RetrievalQA agent loop that can consult both Pinecone and the Tavily external search API.
File Structure Analysis
• Top‐level files only:
• RAG_Langchain_Agents.ipynb – the single notebook hosting all pipeline code.
• tedx.csv – the raw transcript dataset.
• requirements.txt – Python dependencies (langchain, pinecone-client, openai, etc.).
• No separate “src” or “frontend/backend” directories; everything lives in one notebook.
Key Architecture Insights from README
• Data Ingestion via CSVLoader → Tokenization (illustrated conceptually) → Text Splitting (RecursiveCharacterTextSplitter).
• Embeddings generated by OpenAI’s embedding model.
• Indexing & Vector Store: data written into Pinecone, with multi‐tenant namespaces.
• Retrieval QA Chain:
– ChatOpenAI LLM configured with zero temperature.
– Conversational memory (ConversationBufferWindowMemory).
– RetrievalQA chain as one tool.
– TavilySearchResults as second tool.
– Tools registered into a ReAct‐style agent and run via AgentExecutor.