# benitomartin/agentic-rag-langchain-pinecone architecture

> Architecture diagram of the GitHub repository benitomartin/agentic-rag-langchain-pinecone, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/benitomartin/agentic-rag-langchain-pinecone
- Repository: https://github.com/benitomartin/agentic-rag-langchain-pinecone

## Overview

##### Project Type and Purpose
- This is a research‐oriented Retrieval‐Augmented Generation (RAG) pipeline implemented in Python via a Jupyter Notebook.
- It uses LangChain to orchestrate document ingestion, chunking, embedding, vector storage (Pinecone), and a RetrievalQA agent loop that can consult both Pinecone and the Tavily external search API.

##### File Structure Analysis
- Top‐level files only:
- RAG_Langchain_Agents.ipynb – the single notebook hosting all pipeline code.
- tedx.csv – the raw transcript dataset.
- requirements.txt – Python dependencies (langchain, pinecone-client, openai, etc.).
- No separate “src” or “frontend/backend” directories; everything lives in one notebook.

##### Key Architecture Insights from README
- **Data Ingestion** via CSVLoader → **Tokenization** (illustrated conceptually) → **Text Splitting** (RecursiveCharacterTextSplitter).
- **Embeddings** generated by OpenAI’s embedding model.
- **Indexing** & **Vector Store**: data written into Pinecone, with multi‐tenant namespaces.
- **Retrieval QA Chain**:
– ChatOpenAI LLM configured with zero temperature.
– Conversational memory (ConversationBufferWindowMemory).
– RetrievalQA chain as one tool.
– TavilySearchResults as second tool.
– Tools registered into a ReAct‐style agent and run via AgentExecutor.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Notebook Resources
    subgraph "Notebook Resources"
        UI["User Interface\n(Jupyter Notebook)"]:::notebook
        Req["requirements.txt"]:::notebook
        Doc["README.md"]:::notebook
    end

    %% Data Layer
    DataLayer["tedx.csv"]:::data
    DataLayer -->|raw data| CSVLoader["CSVLoader"]:::ingestion

    %% Data Ingestion & Processing
    CSVLoader -->|text chunks| Tokenizer["Tokenizer\n(concept)"]:::ingestion
    Tokenizer -->|chunks| TextSplitter["RecursiveCharacterTextSplitter"]:::ingestion
    TextSplitter -->|snippets| EmbeddingsAPI["OpenAI Embeddings API"]:::external

    %% Storage & Indexing
    EmbeddingsAPI -->|vectors| PineconeClient["Pinecone Client"]:::storage
    PineconeClient -->|store vectors| PineconeStore["Pinecone Vector DB\n(namespaces)"]:::storage

    %% Agent & Reasoning Layer
    subgraph "Agent & Reasoning" 
        AgentExec["AgentExecutor"]:::agent
        Memory["ConversationBufferWindowMemory"]:::agent
        ChatLLM["ChatOpenAI\n(gpt-3.5-turbo)"]:::agent
        RetrievalQA["RetrievalQA Chain"]:::agent
        TavilyTool["TavilySearchResults Tool"]:::agent
    end

    %% External APIs
    subgraph "External Services"
        TavilyAPI["Tavily API"]:::external
    end

    %% User Query Flow
    UI -->|user query| AgentExec
    AgentExec -->|uses memory| Memory
    AgentExec -->|calls tool| RetrievalQA
    AgentExec -->|calls tool| TavilyTool

    RetrievalQA -->|query| PineconeStore
    TavilyTool -->|search| TavilyAPI

    RetrievalQA -->|results| ChatLLM
    TavilyTool -->|results| ChatLLM

    Memory -->|context| ChatLLM
    ChatLLM -->|response| UI

    %% File Click Events
    click UI "https://github.com/benitomartin/agentic-rag-langchain-pinecone/blob/main/RAG_Langchain_Agents.ipynb"
    click DataLayer "https://github.com/benitomartin/agentic-rag-langchain-pinecone/blob/main/tedx.csv"
    click Req "https://github.com/benitomartin/agentic-rag-langchain-pinecone/blob/main/requirements.txt"
    click Doc "https://github.com/benitomartin/agentic-rag-langchain-pinecone/blob/main/README.md"

    %% Styles
    classDef notebook fill:#f9f,stroke:#333,stroke-width:1px
    classDef data fill:#bbf,stroke:#333,stroke-width:1px
    classDef ingestion fill:#cde8ff,stroke:#333,stroke-width:1px
    classDef storage fill:#c8e6c9,stroke:#333,stroke-width:1px
    classDef agent fill:#ffe0b2,stroke:#333,stroke-width:1px
    classDef external fill:#eeeeee,stroke-dasharray: 5 5,stroke:#666,stroke-width:1px
```
