# jatiink/rag-qa-generator architecture

> Architecture diagram of the GitHub repository jatiink/rag-qa-generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jatiink/rag-qa-generator
- Repository: https://github.com/jatiink/rag-qa-generator

## Overview

##### Project Type and Purpose
- This is a single-page Streamlit web application (“full-stack” in the sense of UI + backend pipeline) whose purpose is to:
- Ingest a user-uploaded PDF (or TXT)
- Split and embed the text into a FAISS vector store
- Use RetrievalQA with an LLM (llama2) to generate MCQs, True/False, and one-word-answer questions
- Results in the browser and persist them in a TXT file

##### Main Components
a. Presentation Layer (Frontend)
– Streamlit Web UI (app.py)
– File uploader widget
– Question-type selector
– Display area for generated questions
b. Application/Processing Layer (Backend pipeline inside app.py)
1) Document Loader: PyPDFLoader or TXT loader
2) Text Splitter: RecursiveCharacterTextSplitter
3) Embeddings: HuggingFaceEmbeddings (local or cloud)
4) Vector Store: FAISS index (in-memory or on-disk)
5) RetrievalQA: orchestrates query → vector search → prompt building → LLM inference
6) LLM Inference: llama2 engine via Ollama runtime
c. Persistence
– Input storage: uploaded PDF/TXT (e.g., Big Mac Index.pdf)
– Output store: Generated_Questions.txt
d. External Dependencies
– Ollama daemon (serving llama2 model)
– Python libraries (requirements.txt)

##### Relationships & Data Flow
1) Browser → Streamlit App: user uploads a PDF/TXT and selects question type
2) Streamlit App → Document Loader: loads raw text
3) Document Loader → Text Splitter: breaks text into chunks
4) Text Splitter → Embedding Engine: transforms chunks into vectors
5) Embeddings → FAISS Vector Store: builds/searchable index
6) User Query (RetrievalQA) → FAISS: retrieves top-k relevant chunks
7) Retrieved Chunks → Prompt → llama2 via Ollama → generates questions
8) Streamlit App → Browser: displays questions
9) Streamlit App → File System: writes Generated_Questions.txt

##### Architectural Patterns & Design Principles
- Modular pipeline: clearly separated ETL (load, split, embed, store) and QA (retrieve, prompt, infer) phases
- Single-container monolith: all components run inside one Python process, but logically separated
- Reuse of open-source building blocks (LangChain patterns)
- Stateless UI with stateful storage (FAISS + TXT file)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Browser"
        Browser["Web Browser"]:::ui
    end

    subgraph "File System"
        InputFile["Big Mac Index.pdf / .txt"]:::storage
        OutputFile["Generated_Questions.txt"]:::storage
        Requirements["requirements.txt"]:::storage
        Readme["README.md"]:::storage
    end

    subgraph "Ollama Runtime"
        LLMService["llama2 via Ollama"]:::external
    end

    subgraph "Streamlit App (app.py)"
        direction TB
        UI["Streamlit UI"]:::ui
        Uploader["File Uploader"]:::ui
        Selector["Question Type Selector"]:::ui
        Display["Display Area"]:::ui
        Loader["Document Loader"]:::processing
        Splitter["Text Splitter"]:::processing
        Embed["Embedding Engine"]:::processing
        VectorStore["FAISS Index"]:::processing
        QA["RetrievalQA Orchestration"]:::processing
        LLMClient["LLM Client"]:::processing
    end

    Browser -->|uploads file| Uploader
    Uploader -->|PDF/TXT| Loader
    Loader -->|raw text| Splitter
    Splitter -->|text chunks| Embed
    Embed -->|embedding vectors| VectorStore
    Selector -->|question type| QA
    QA -->|query vectors| VectorStore
    VectorStore -->|relevant chunks| QA
    QA -->|prompt| LLMClient
    LLMClient -->|calls llama2| LLMService
    LLMService -->|generated questions| LLMClient
    LLMClient -->|questions| QA
    QA -->|generated questions| Display
    QA -->|write questions| OutputFile
    Display -->|renders questions| Browser

    click UI "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click Uploader "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click Selector "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click Display "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click Loader "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click Splitter "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click Embed "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click VectorStore "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click QA "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click LLMClient "https://github.com/jatiink/rag-qa-generator/blob/read_me/app.py"
    click InputFile "https://github.com/jatiink/rag-qa-generator/blob/read_me/Big Mac Index.pdf"
    click OutputFile "https://github.com/jatiink/rag-qa-generator/blob/read_me/Generated_Questions.txt"
    click Requirements "https://github.com/jatiink/rag-qa-generator/blob/read_me/requirements.txt"
    click Readme "https://github.com/jatiink/rag-qa-generator/blob/read_me/README.md"

    classDef ui fill:#9BE7FF,stroke:#333,stroke-width:1px;
    classDef processing fill:#B6E6BD,stroke:#333,stroke-width:1px;
    classDef storage fill:#FFC593,stroke:#333,stroke-width:1px;
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px;
```
