# tmtien31/history_question_generator architecture

> Architecture diagram of the GitHub repository tmtien31/history_question_generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tmtien31/history_question_generator
- Repository: https://github.com/tmtien31/history_question_generator

## Overview

##### Project type and purpose
- This is a full-stack web application (“History Question Generator”) built with FastAPI (backend) and Jinja2/HTML (frontend).
- Its purpose is to ingest Vietnamese history PDF documents, run a Retrieval-Augmented Generation (RAG) pipeline (LangChain + FAISS + Google Gemini 2.0), and produce multiple-choice question/answer CSVs for educational use.

##### File structure
- Top-level files:
- app.py: FastAPI server with upload and analysis endpoints
- requirements.txt, Dockerfile, .env → configuration & deployment
- README.md, LICENSE → documentation
- Directories:
- src/ (helper.py, prompt.py): core pipeline logic, prompt templates
- templates/index.html: Jinja2 frontend
- static/docs/: uploaded PDFs
- static/output/: generated CSVs
- data/: sample PDFs for testing
- research/: experimentation notebook
- Build & deploy artifacts: Dockerfile, .dockerignore, .gitignore

##### Extract architectural insights from README
- Architecture:
- Document upload → static/docs
- Text extraction (PyPDFLoader/PyPDF2)
- Chunking (TokenTextSplitter)
- Embeddings (HuggingFace) → FAISS vector store
- Generation (LangChain refine chain + Google Gemini 2.0)
- RetrievalQA chain for answers
- Output saved to CSV in static/output
- External dependencies: Google Gemini API, sentence-transformers, FAISS, LangChain

##### Main components
a. Client (Web Browser)
b. FastAPI Server (app.py + Uvicorn)
c. File storage
- PDF Store: static/docs
- CSV Store: static/output
d. Processing Pipeline
1. PDF Loader (PyPDFLoader/PyPDF2)
2. Text Chunking (TokenTextSplitter)
3. Embedding Generator (HuggingFace models)
4. Vector Database (FAISS)
5. RAG chains (LangChain refine & RetrievalQA)
e. External AI Service
- Google Gemini 2.0 API
f. Data Models & Helpers
- src/helper.py, src/prompt.py

##### Relationships and data flows
- Browser → HTTP POST /upload → FastAPI
- FastAPI writes PDF → static/docs
- FastAPI triggers processing:
- PDF Loader reads file → raw text
- TokenTextSplitter → text chunks
- HuggingFace embedding call → embedding vectors
- FAISS ingest & similarity search
- LangChain refine chain calls Google Gemini API → questions
- RetrievalQA chain uses FAISS & Gemini → answers
- FastAPI writes CSV → static/output
- Browser polls GET or page reload to download CSV

##### Architectural patterns & design principles
- Layered architecture: Presentation, API, Processing pipeline, Storage, External services
- Single-process monolith (all pipeline code in one service)
- RAG micro-pattern: separation of embedding store (FAISS) and LLM calls
- Asynchronous file I/O (aiofiles) and non-blocking endpoints

##### Technologies / frameworks to annotate
- FastAPI + Uvicorn (web framework & ASGI server)
- Jinja2 (templating)
- aiofiles (async file handling)
- PyPDFLoader, PyPDF2 (PDF extraction)
- TokenTextSplitter (LangChain)
- sentence-transformers (embeddings)
- FAISS (vector store)
- LangChain (refine & RetrievalQA chains)
- Google Gemini 2.0 Flash Lite API

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation Layer
    subgraph "Presentation Layer"
        Browser["Web Browser UI"]:::ui
    end

    %% API & Processing
    subgraph "API & Processing"
        FastAPI["FastAPI Server\n(app.py)"]:::api
        subgraph "Processing Pipeline"
            Loader["PDF Loader\n(PyPDFLoader/PyPDF2)"]:::compute
            Chunker["TokenTextSplitter"]:::compute
            Embedding["Embedding Generator\n(sentence-transformers)"]:::compute
            VectorDB["FAISS Vector Store"]:::storage
            RefineChain["LangChain Refine Chain"]:::compute
            RetrievalQA["LangChain RetrievalQA Chain"]:::compute
        end
    end

    %% Storage
    subgraph "File Storage"
        PDFStore["PDF Store\n(static/docs/)"]:::storage
        CSVStore["CSV Store\n(static/output/)"]:::storage
    end

    %% External Service
    Google["Google Gemini 2.0 API"]:::external

    %% Dev & Config
    subgraph "Dev & Config"
        Helper["Helpers\n(src/helper.py)"]:::docs
        Prompt["Prompt Templates\n(src/prompt.py)"]:::docs
        SampleData["Sample Data\n(data/WW2.pdf)"]:::docs
        Notebook["Experiment Notebook\n(research/experiment.ipynb)"]:::docs
        Req["Requirements\n(requirements.txt)"]:::docs
        Docker["Dockerfile"]:::docs
        Readme["README.md"]:::docs
        License["LICENSE"]:::docs
    end

    %% Connections
    Browser -->|"POST /upload"| FastAPI
    FastAPI -->|"save PDF"| PDFStore
    FastAPI -->|"trigger processing"| Loader
    Loader --> Chunker
    Chunker --> Embedding
    Embedding --> VectorDB
    VectorDB --> RefineChain
    RefineChain -->|"call LLM"| Google
    Google --> RefineChain
    VectorDB --> RetrievalQA
    RetrievalQA -->|"call LLM"| Google
    Google --> RetrievalQA
    RefineChain --> FastAPI
    RetrievalQA --> FastAPI
    FastAPI -->|"write CSV"| CSVStore
    Browser -->|"download CSV"| CSVStore

    %% Click Events
    click FastAPI "https://github.com/tmtien31/history_question_generator/blob/main/app.py"
    click Browser "https://github.com/tmtien31/history_question_generator/blob/main/templates/index.html"
    click PDFStore "https://github.com/tmtien31/history_question_generator/tree/main/static/docs/"
    click CSVStore "https://github.com/tmtien31/history_question_generator/tree/main/static/output/"
    click Helper "https://github.com/tmtien31/history_question_generator/blob/main/src/helper.py"
    click Prompt "https://github.com/tmtien31/history_question_generator/blob/main/src/prompt.py"
    click SampleData "https://github.com/tmtien31/history_question_generator/blob/main/data/WW2.pdf"
    click Notebook "https://github.com/tmtien31/history_question_generator/blob/main/research/experiment.ipynb"
    click Req "https://github.com/tmtien31/history_question_generator/blob/main/requirements.txt"
    click Docker "https://github.com/tmtien31/history_question_generator/tree/main/Dockerfile"
    click Readme "https://github.com/tmtien31/history_question_generator/blob/main/README.md"
    click License "https://github.com/tmtien31/history_question_generator/tree/main/LICENSE"

    %% Styles
    classDef ui fill:#cce5ff,stroke:#004085;
    classDef api fill:#d4edda,stroke:#155724;
    classDef compute fill:#fff3cd,stroke:#856404;
    classDef storage fill:#ffe5b4,stroke:#b56d00;
    classDef external fill:#f8d7da,stroke:#721c24;
    classDef docs fill:#e2e3e5,stroke:#6c757d;
```
