# prodip1023/rag-application-1 architecture

> Architecture diagram of the GitHub repository prodip1023/rag-application-1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/prodip1023/rag-application-1
- Repository: https://github.com/prodip1023/rag-application-1

## Overview

##### Project Type and Purpose
- This is an open‐source Retrieval‐Augmented Generation (RAG) tool.
- Purpose: ingest a text corpus, index it with FAISS, and at runtime retrieve relevant chunks to augment prompts sent to an LLM (e.g., OpenAI) for generation.

##### Main Architectural Components
a. Data Ingestion & Indexing (Offline)
- Source: data/mydata.txt
- Chunking module: utils/chunking.py
- Embedding module: utils/embedding.py
- FAISS store: faiss_store/index.faiss + chunk_mapping.pkl
b. Retrieval & Prompt Construction (Online)
- Retrieval module: utils/retrieval.py
- Prompt builder: utils/prompt.py
c. Completion (LLM)
- Completion module: utils/completion.py calls external OpenAI API
d. Orchestration & API Layer
- app.py: coordinates user queries, retrieval, prompt assembly, completion, and response

##### Relationships & Data Flows
- Offline Flow:
1) Read raw text → 2) Chunker → 3) Embed → 4) FAISS index build → Persist index & mapping
- Online Flow:
1) User request → app.py → 2) Embed query (optional) → 3) FAISS retrieval → 4) Prompt build → 5) Send to OpenAI → 6) Return generation to user

##### Architectural Patterns & Design Principles
- Modular pipeline: each utility (chunking, embedding, retrieval, prompt, completion) is a separate component
- Separation of concerns: offline indexing vs. online serving
- Single‐process/script based orchestration (no microservices)
- External service dependency: OpenAI API

##### Technologies & Frameworks to Call Out
- Python 3.10 (specified in environment)
- FAISS (vector store)
- OpenAI Python SDK (in utils/completion.py)
- Conda / pip for environment & dependencies

##### Essential Elements
- Clear component boundaries (Offline vs. Online)
- All five utils modules as separate blocks
- The FAISS store and mapping file as persistent storage
- The external OpenAI API
- The orchestration script app.py

- Separates indexing & serving phases
- Clearly labels modules and data flows
- Emphasizes modularity and extension points inherent in this RAG application

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Offline Indexing Components
    subgraph "Offline Indexing"
        Data_Raw["data/mydata.txt"]:::offline
        Chunking["Chunking Module"]:::offline
        Embedding["Embedding Module"]:::offline
        FAISS[(FAISS Vector Store)]:::store
        Mapping[(Chunk Mapping)]:::store
    end

    %% Online Serving Components
    subgraph "Orchestration & API Layer"
        App["app.py"]:::online
    end
    subgraph "Online Serving"
        Retrieval["Retrieval Module"]:::online
        Prompt["Prompt Builder"]:::online
        Completion["Completion Module"]:::online
        OpenAI["OpenAI API"]:::external
        User((User)):::external
    end

    %% Offline Data Flow
    Data_Raw -->|"raw text"| Chunking -->|"chunks"| Embedding -->|"vectors"| FAISS
    FAISS -->|"persist mapping"| Mapping

    %% Online Query Flow
    User -->|"query"| App
    App -->|"embed query"| Retrieval
    Retrieval -->|"relevant chunks"| Prompt
    Prompt -->|"prompt + context"| Completion
    Completion -->|"LLM call"| OpenAI
    OpenAI -->|"response"| Completion
    Completion -->|"completion response"| App
    App -->|"return result"| User

    %% Click Events
    click Data_Raw "https://github.com/prodip1023/rag-application-1/blob/main/data/mydata.txt"
    click Chunking "https://github.com/prodip1023/rag-application-1/blob/main/utils/chunking.py"
    click Embedding "https://github.com/prodip1023/rag-application-1/blob/main/utils/embedding.py"
    click FAISS "https://github.com/prodip1023/rag-application-1/blob/main/faiss_store/index.faiss"
    click Mapping "https://github.com/prodip1023/rag-application-1/blob/main/faiss_store/chunk_mapping.pkl"
    click Retrieval "https://github.com/prodip1023/rag-application-1/blob/main/utils/retrieval.py"
    click Prompt "https://github.com/prodip1023/rag-application-1/blob/main/utils/prompt.py"
    click Completion "https://github.com/prodip1023/rag-application-1/blob/main/utils/completion.py"
    click App "https://github.com/prodip1023/rag-application-1/blob/main/app.py"

    %% Styles
    classDef offline fill:#FFD7D7,stroke:#D9534F,stroke-width:1px
    classDef online fill:#D7FFD7,stroke:#5CB85C,stroke-width:1px
    classDef store fill:#FFF2CC,stroke:#F0AD4E,stroke-width:1px
    classDef external fill:#D7EEFF,stroke:#5BC0DE,stroke-width:1px
```
