# phamxuankhang/vietnamese_law_assistant_chatbot architecture

> Architecture diagram of the GitHub repository phamxuankhang/vietnamese_law_assistant_chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/phamxuankhang/vietnamese_law_assistant_chatbot
- Repository: https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot

## Overview

##### Main Components
- Data Source & Raw Data Repository:
- The “BoPhapDienDienTu” directory contains the original legal texts (JSON and HTML files).
- Data Crawling and Insertion Module:
- Located in chatbot_v1/src/crawl_and_insert_to_db, this module is responsible for crawling the legal data, parsing content, and storing it in a MySQL database. It includes database connection handling (db.py), schema models (models.py), and utility functions.
- Embedding Generation Module:
- Found in chatbot_v1/src/embedding, this code preprocesses text and creates vector embeddings (using language models or Sentence Transformers). The embeddings are stored (possibly both in files under data/ and within Weaviate).
- LLM Response Module:
- In chatbot_v1/src/LLM_response, this subsystem integrates with external LLMs (like Google Palm) to generate responses. It includes modules for generating responses (LLM_generator.py/llm.py), routing requests (semantic_router.py), and retrieving context-specific data (retriever.py).
- Database Systems:
- A traditional MySQL database stores structured legal information as indicated by the detailed schema in the README.
- A vector database (Weaviate) is deployed via Docker (weaviate-docker/) to store and query text embeddings.
- REST API & Web Backend:
- The backend is built with Python & FastAPI. It exposes endpoints such as search (/api/search), detailed law article view (/api/articles/{article_id}), topics (/api/topics), and sections (/api/sections).
- Frontend (User Interface):
- The “user_interface” directory holds a React-based web UI with pages (e.g., Vietnamese_Law_Assistant_Chatbot_v1.py and v2.py) that interact with the backend API via HTTP requests.
- Additional Modules (chatbot_v2):
- There is a secondary version (chatbot_v2) with NER and relation creation modules. These address enhanced text processing, such as Named Entity Recognition for legal texts and creating relationships between legal entities.

##### Relationships and Data Flow
- The Raw Data (BoPhapDienDienTu) is first processed by the Crawl Module, which extracts and inserts structured data into the MySQL Database.
- The Embedding Module takes cleaned/processed text from the database (or from intermediate processed files) and produces vector representations that are inserted into the Weaviate vector database.
- The LLM Response Module then leverages the vector database to retrieve contextually similar legal documents and uses the LLM (Google Palm) to generate natural language responses.
- The FastAPI backend acts as an orchestrator, exposing these capabilities through REST API endpoints.
- Finally, the Frontend (built in React) sends requests to the FastAPI backend and displays response data (search results, law article details, etc.) to users.

##### Architectural Patterns and Design Principles
- Modularization: Each functionality (crawling, embedding, LLM response) is built as an independent module to ensure separation of concerns.
- Service Orientation: Although packaged together, the components (crawling, embedding, LLM response) can be seen as microservices that interact via defined interfaces.
- API-Driven Design: The FastAPI endpoints define clear contracts between the frontend and backend, and between services.
- Containerization: The Weaviate vector database is run in a Docker container, and other services can similarly be containerized for deployment.

##### Technologies, Frameworks, and Libraries
- Backend: Python (FastAPI, SQLAlchemy, PyMySQL)
- Data Processing: BeautifulSoup, embedding libraries (e.g., Sentence Transformers), LangChain
- Database: MySQL for structured legal data and Weaviate for vector operations.
- Frontend: React, Tailwind CSS, Axios, React Query
- DevOps: Docker, Docker Compose

##### Specific Elements
- Clear labels for each component (e.g., “Data Crawler”, “Embedding Service”, “LLM Handler”, “MySQL Database”, “Weaviate Vector DB”, “API Gateway (FastAPI)”, “React Frontend”).
- From the Data Crawler to MySQL.
- From the Embedding Service to Weaviate.
- From the FastAPI backend to both MySQL and Weaviate.
- Notations for key technologies (e.g., “Python 3.8”, “FastAPI”, “SQLAlchemy”, “Docker”, “Weaviate”, “React”) to provide context.
- For components like chatbot_v2, you might include sub-modules for “NER” and “Relation Creation” and show how they feed into the overall data workflow.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data & Ingestion Layer"
        RD["BoPhapDienDienTu (Raw Legal Texts)"]:::raw
        DC["Data Crawler"]:::process
        DB["MySQL Database"]:::database
        RD -->|"processed_by"| DC
        DC -->|"stores_data_in"| DB
    end

    subgraph "Processing & AI Layer"
        EG["Embedding Generation"]:::process
        WV["Weaviate Vector DB"]:::database
        LLM["LLM Response Module"]:::process
        NER["NER Module"]:::process
        RR["Relation Creation Module"]:::process
        DB -->|"provides_text"| EG
        EG -->|"stores_embeddings_in"| WV
        DB -->|"provides_context"| LLM
        WV -->|"vector_search"| LLM
        NER -->|"enhances_text"| LLM
        RR -->|"creates_relations"| LLM
    end

    subgraph "API & Service Layer"
        API["API Gateway (FastAPI)"]:::api
        LLM -->|"generates_response"| API
    end

    subgraph "Presentation Layer"
        FE["React Frontend"]:::frontend
        API -->|"api_calls"| FE
    end

    %% Click Events
    click DC "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/chatbot_v1/src/crawl_and_insert_to_db"
    click EG "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/chatbot_v1/src/embedding"
    click LLM "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/chatbot_v1/src/LLM_response"
    click WV "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/chatbot_v1/weaviate-docker"
    click API "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/blob/master/fastAPI.py"
    click FE "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/user_interface"
    click NER "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/chatbot_v2/Code/NER"
    click RR "https://github.com/phamxuankhang/vietnamese_law_assistant_chatbot/tree/master/chatbot_v2/Code/create_relation"

    %% Styles
    classDef raw fill:#FFEEAA,stroke:#333,stroke-width:2px;
    classDef process fill:#AAFFAA,stroke:#333,stroke-width:2px;
    classDef database fill:#DDEEFF,stroke:#333,stroke-width:2px;
    classDef api fill:#CCCCFF,stroke:#333,stroke-width:2px;
    classDef frontend fill:#FFCCCC,stroke:#333,stroke-width:2px;
```
