# ngothanhnam0910/vietnamese-law-question-answering-system architecture

> Architecture diagram of the GitHub repository ngothanhnam0910/vietnamese-law-question-answering-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ngothanhnam0910/vietnamese-law-question-answering-system
- Repository: https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system

## Overview

##### Project type and purpose
- This is a full-stack Retrieval-Augmented Generation (RAG) Q&A chatbot system specialized on Vietnamese legal documents.
- It combines a Python-based frontend chat interface, a FastAPI backend with Celery tasks, vector and lexical search (Qdrant + Elasticsearch), OpenAI models for LLM inference, and separate pipelines for finetuning embedding/rerank models and the answer-generation LLM.

##### File structure
- Top-level directories:
- backend – FastAPI service, Celery configuration, retrieval logic, caching, database models/schemas.
- chatbot-ui – Python CLI or simple UI client that calls the backend API.
- retrieval – data creation & finetuning scripts for embedding and rerank models.
- finetune_llm – data generation and finetuning scripts for the answer-generation LLM.
- images – static assets for documentation.
- Architectural patterns:
- Separation of concerns: UI ↔ API ↔ background workers ↔ storage ↔ external AI services.
- Modular search_document package encapsulating multiple search strategies (Elastic, BGE, E5, ensemble, rerank).
- Celery for asynchronous tasks (e.g., batch retrieval, long-running finetune jobs).
- External state services: Qdrant (vector DB), Elasticsearch (inverted index), and a relational DB (SQLAlchemy models).

##### Extract insights from the README
- There is a two-phase retrieval pipeline: chunking → embedding via two models → Qdrant; and parallel Elasticsearch lookup → merge → rerank → top-k documents.
- RAG flow: intent detection → query rewriting → retrieval → reranking → answer generation → fallback “tool” search.
- Two finetuning subsystems: one for the rerank model in retrieval/; one for the base LLM in finetune_llm/.

a. Main components
1. Chatbot UI (chat_interface.py)
2. FastAPI Backend (app.py, agent.py, brain.py)
3. Celery Workers (celery_app.py + tasks.py)
4. Vector Storage (Qdrant)
5. Elasticsearch Cluster
6. Relational Database (Postgres/MySQL)
7. OpenAI API (gpt-4o-mini, Chat Completions)
8. Tavily Search Tool (external web search)
9. Rerank model service (BGE-v2-m3 finetuned)
10. Finetuning Pipelines (scripts in retrieval/ and finetune_llm/)

b. Relationships and data flows
- User → Chatbot UI → HTTP → FastAPI API
- API → “Intent Detection” via OpenAI → branch: Chitchat vs. Legal
- If Legal: API → Query Rewriting (OpenAI) → send rewritten query →
- Parallel: embed via BGE & E5 → Qdrant → top-n vectors
- Elasticsearch text search → top-n docs
- Merge results → send to Rerank model (Celery or sync) → top-k docs
- API → Assemble prompt with top-k docs + user query → OpenAI → answer
- If answer == “no”: trigger Tavily Search Tool → fetch web snippets → re-invoke LLM
- All API calls for retrieval/LLM can be dispatched to Celery for concurrency and rate-limiting.
- Persist chat history & metadata in relational DB; cache recent queries in Redis or in-memory.

c. Architectural patterns & principles
- Micro-service style modularity inside one containerized backend.
- CQRS-like separation: synchronous queries vs. asynchronous tasks (Celery).
- Pipeline orchestration: intent → rewrite → retrieve → rerank → generate.
- Ensemble and fallback for robustness (Elastic + vector + tool search).

d. Key technologies & frameworks
- Python 3.x, FastAPI, Celery, Pydantic (schemas.py), SQLAlchemy (models.py)
- Qdrant (vector DB), Elasticsearch
- OpenAI Python SDK, huggingface-trl for LLM finetuning
- BGE and Multilingual-e5 embedding models
- Docker / shell scripts for entrypoints (entrypoint.sh)

##### Detailed coverage
- Every file/folder with unique responsibility is reflected: search_document submodules, finetune_llm scripts, retrieval training scripts.
- Celery’s role in decoupling long-running tasks (e.g., rerank training, LLM finetune).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "Frontend (Chatbot UI)"
        UI["Chatbot UI\n(chat_interface.py)"]:::app
        EP_UI["UI Entrypoint\n(entrypoint.sh)"]:::app
        CFG["UI Config\n(config.toml)"]:::app
    end

    %% Backend API & Workers
    subgraph "Backend API & Workers"
        API["FastAPI API Server\n(app.py)"]:::app
        AG["Agent Logic\n(agent.py)"]:::app
        BR["Brain Logic\n(brain.py)"]:::app
        CELERY["Celery Worker Tasks\n(tasks.py)"]:::app
        EP_BE["Backend Entrypoint\n(entrypoint.sh)"]:::app
        CACHE["Cache Layer\n(cache.py)"]:::app
        DB_CONN["DB Connection\n(database.py)"]:::app
        MODELS["ORM Models\n(models.py)"]:::app
        SCHEMAS["Pydantic Schemas\n(schemas.py)"]:::app
        UTILS["Utilities\n(utils.py)"]:::app
        ELS["Elasticsearch Text Search\n(search_elastic.py)"]:::app
        BGE["BGE Embedding Search\n(search_with_bge.py)"]:::app
        E5["E5 Embedding Search\n(search_with_e5.py)"]:::app
        COMB["Combine Search Strategies\n(combine_search.py)"]:::app
        RR["Rerank Module\n(rerank.py)"]:::app
        TAV["Tavily Fallback Tool\n(tavily_search.py)"]:::app
    end

    %% Datastores
    subgraph "Data Stores"
        QDR["Qdrant Vector DB"]:::data
        ES["Elasticsearch Cluster"]:::data
        RDB["Relational DB"]:::data
        REDIS["Redis Cache"]:::data
    end

    %% External AI Services
    subgraph "External AI Services"
        OA["OpenAI API"]:::ext
        TA["Tavily Search Tool"]:::ext
    end

    %% Training Pipelines
    subgraph "Training Pipelines"
        subgraph "Retrieval Finetune"
            RD1["Create Rerank Data\n(create_data_rerank.py)"]:::pipe
            HNB["Hard Negative BGE\n(hard_negative_bge_round1.py)"]:::pipe
            HNE["Hard Negative E5\n(hard_negative_e5.py)"]:::pipe
            FT_RR["Rerank Finetune\n(finetune.sh)"]:::pipe
        end
        subgraph "LLM Finetune"
            GD["Data Generation\n(gen_data.py)"]:::pipe
            DL["Download Base Model\n(download_model.py)"]:::pipe
            FT_LLM["Fine-tune LLM\n(finetune.py)"]:::pipe
            MV["Merge with Base\n(merge_with_base.py)"]:::pipe
            EV["Evaluate Model\n(evaluate_finetuned_model.py)"]:::pipe
            TM["Test Inference\n(test_model.py)"]:::pipe
            RUNINF["Run Inference\n(run_infer_tgi.sh)"]:::pipe
        end
    end

    %% Application Flow
    UI -->|"HTTP Request"| API
    API -->|"Invoke Agent"| AG
    AG --> BR
    BR -->|"Intent Detection"| OA
    BR -->|"Query Rewrite"| OA
    BR --> COMB
    COMB --> BGE
    COMB --> E5
    BGE --> QDR
    E5 --> QDR
    COMB --> ELS
    ELS --> COMB
    COMB --> RR
    RR --> BR
    BR -->|"Assemble Prompt"| OA
    OA --> BR
    BR --> API
    API --> UI

    BR -->|"No Answer Fallback"| TA
    TA --> BR

    %% Asynchronous Tasks
    BR --> CELERY
    CELERY --> COMB
    CELERY --> RR
    CELERY --> OA

    %% Caching & Persistence
    API --> CACHE
    API --> DB_CONN
    DB_CONN --> RDB
    CACHE --> API
    API --> MODELS
    API --> SCHEMAS
    API --> UTILS

    %% Click Events
    click UI "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/chatbot-ui/chat_interface.py"
    click EP_UI "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/chatbot-ui/entrypoint.sh"
    click CFG "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/chatbot-ui/config.toml"
    click API "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/app.py"
    click AG "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/agent.py"
    click BR "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/brain.py"
    click CELERY "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/tasks.py"
    click EP_BE "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/entrypoint.sh"
    click CACHE "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/cache.py"
    click DB_CONN "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/database.py"
    click MODELS "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/models.py"
    click SCHEMAS "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/schemas.py"
    click UTILS "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/utils.py"
    click COMB "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/search_document/combine_search.py"
    click ELS "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/search_document/search_elastic.py"
    click BGE "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/search_document/search_with_bge.py"
    click E5 "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/search_document/search_with_e5.py"
    click RR "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/search_document/rerank.py"
    click TAV "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/backend/src/tavily_search.py"
    click RD1 "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/retrieval/create_data_rerank.py"
    click HNB "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/retrieval/hard_negative_bge_round1.py"
    click HNE "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/retrieval/hard_negative_e5.py"
    click FT_RR "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/retrieval/finetune.sh"
    click GD "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/gen_data.py"
    click DL "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/download_model.py"
    click FT_LLM "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/finetune.py"
    click MV "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/merge_with_base.py"
    click EV "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/evaluate_finetuned_model.py"
    click TM "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/test_model.py"
    click RUNINF "https://github.com/ngothanhnam0910/vietnamese-law-question-answering-system/blob/main/finetune_llm/run_infer_tgi.sh"

    %% Styles
    classDef app fill:#89CFF0,stroke:#000,stroke-width:1px;
    classDef data fill:#98FB98,stroke:#000,stroke-width:1px;
    classDef ext fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef pipe fill:#D3D3D3,stroke:#000,stroke-width:1px;
```
