# hungnlp/vibiogpt architecture

> Architecture diagram of the GitHub repository hungnlp/vibiogpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hungnlp/vibiogpt
- Repository: https://github.com/hungnlp/vibiogpt

## Overview

##### Project Type and Purpose
- The project focuses on fine-tuning a large language model for biomedical Q&A in Vietnamese.
- It provides both training and inference capabilities, with training scripts (training.py, train.sh, train_debug.sh) and an inference script (inference.py) tailored for adapting the base model to a specific medical Q&A dataset.
- The README clearly explains usage examples, dependency management, and training pipelines.

##### Analyzing the File Structure
- Data Layer:
- Data files (under the “data” directory, e.g., qa_medical_pairs.json, qa_pairs_edoctor.json, qa_vinmec.json) serve as the training dataset.
- Tools like dataset.py and data_collators.py are responsible for preparing and batching the data.
- Prompting Utilities:
- The “prompt” folder (with base.py, qa_prompt.py, role.py) defines the chat prompt structures and roles which are vital for interaction during inference.
- Training and Inference Pipelines:
- The training pipeline is orchestrated by training.py and the associated shell scripts (train.sh, train_debug.sh) which leverage settings from arguments.py, constants.py, and utils.py.
- The inference.py script provides command-line-driven inference to generate responses using the fine-tuned model.
- Tokenization:
- tokenizer.py configures and manages how text is tokenized, ensuring compatibility with both training and inference processes.
- Configuration & Environment:
- The .env.example and dependency files (requirements.txt) suggest environment configuration and dependency management.

##### How to Represent the System
a. Main Components:
- Data Ingestion & Preprocessing: (data files, dataset.py, data_collators.py)
- Prompt Infrastructure: (prompt/ folder, base and qa prompt logic)
- Training Pipeline: (training.py, train.sh/scripts, arguments.py, constants.py)
- Inference Pipeline: (inference.py, tokenizer.py, utils.py)
- External Dependencies/Models: (Pre-trained Vistral-Chat-7B model, the fine-tuned adapter, and external libraries like Transformers, PEFT, bitsandbytes, Flash Attention)
- Environment & Configuration: (.env, requirements.txt)

b. Relationships & Interactions:
- Data flows from the dataset (JSON files in the data folder) into the preprocessing modules (dataset.py and data_collators.py).
- The training pipeline fetches the pre-trained model, applies the QLora-based fine-tuning (using training.py and shell scripts), and incorporates prompt structures (via the prompt folder).
- The inference module leverages both the tokenizer and prompt templates to generate answers.
- Environment variables (from .env) and dependency configurations (requirements.txt) are used across the pipelines.
- Data flow: from raw dataset → preprocessing → training/inference.
- Control flow: from training scripts to model fine-tuning and then to inference.
- Dependency links: e.g., the model fetch from Hugging Face using tokens from the environment.

c. Important Architectural Patterns & Design Principles:
- Separation of concerns is clearly maintained between training and inference.
- Modularity through dedicated files for data management, prompting, tokenization, and utility functions.
- Extensibility in design, as evidenced by the clear demarcation of the prompt template logic (making it easier to adjust the conversational context).

d. Relevant Technologies and Frameworks:
- Transformer-based Models: Transformers, PEFT library.
- QLora Fine-Tuning Technique.
- Quantization libraries like bitsandbytes and Flash Attention.
- Python ecosystem with dependencies managed in requirements.txt.
- External model hosting from Hugging Face (both pre-trained and fine-tuned adapter).

- Use clear labels for each component (e.g., “Data Loader”, “Data Preprocessor”, “Training Pipeline”, “Prompt Templates”, “Inference Engine”).
▪ Rectangles for scripts/modules.
▪ Rounded rectangles for external services (Hugging Face model repository) and environment configuration.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Environment & Configuration
    subgraph "Environment & Configuration"
        env["Environment (.env.example & requirements.txt)"]:::env
    end

    %% Data Layer
    subgraph "Data Layer"
        qaData["QA Data (JSON files from data/)"]:::data
        datasetLoader["Dataset Loader (dataset.py)"]:::data
        dataCollators["Data Collators (data_collators.py)"]:::data
    end

    qaData -->|"loads"| datasetLoader
    datasetLoader -->|"processes with"| dataCollators

    %% Prompt Infrastructure
    subgraph "Prompt Infrastructure"
        promptTemplates["Prompt Templates (prompt/)"]:::prompt
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        trainingScript["Training Script (training.py)"]:::training
        shellTrain["Shell Script (train.sh)"]:::training
        shellDebug["Shell Script (train_debug.sh)"]:::training
        trainArg["Training Config (arguments.py)"]:::training
        trainConst["Training Config (constants.py)"]:::training
        trainUtils["Training Config (utils.py)"]:::training
        preTrainedModel["Pre-trained Model (Vistral-Chat-7B)"]:::external
    end

    dataCollators -->|"feeds data to"| trainingScript
    env --> trainingScript
    promptTemplates --> trainingScript
    preTrainedModel --> trainingScript
    shellTrain --> trainingScript
    shellDebug --> trainingScript
    trainArg --> trainingScript
    trainConst --> trainingScript
    trainUtils --> trainingScript

    trainingScript -->|"fine-tunes"| fineTuned["Fine-tuned Adapter Model"]:::training

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        inferenceScript["Inference Script (inference.py)"]:::inference
        tokenizer["Tokenizer (tokenizer.py)"]:::inference
    end

    fineTuned --> inferenceScript
    tokenizer --> inferenceScript
    promptTemplates --> inferenceScript
    env --> inferenceScript

    %% Click Events
    click qaData "https://github.com/hungnlp/vibiogpt/tree/master/data/"
    click datasetLoader "https://github.com/hungnlp/vibiogpt/blob/master/dataset.py"
    click dataCollators "https://github.com/hungnlp/vibiogpt/blob/master/data_collators.py"
    click promptTemplates "https://github.com/hungnlp/vibiogpt/tree/master/prompt/"
    click trainingScript "https://github.com/hungnlp/vibiogpt/blob/master/training.py"
    click shellTrain "https://github.com/hungnlp/vibiogpt/blob/master/train.sh"
    click shellDebug "https://github.com/hungnlp/vibiogpt/blob/master/train_debug.sh"
    click trainArg "https://github.com/hungnlp/vibiogpt/blob/master/arguments.py"
    click trainConst "https://github.com/hungnlp/vibiogpt/blob/master/constants.py"
    click trainUtils "https://github.com/hungnlp/vibiogpt/blob/master/utils.py"
    click inferenceScript "https://github.com/hungnlp/vibiogpt/blob/master/inference.py"
    click tokenizer "https://github.com/hungnlp/vibiogpt/blob/master/tokenizer.py"
    click env "https://github.com/hungnlp/vibiogpt/blob/master/.env.example"
    click env "https://github.com/hungnlp/vibiogpt/blob/master/requirements.txt"

    %% Styles
    classDef env fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef prompt fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef training fill:#FFB6C1,stroke:#333,stroke-width:2px;
    classDef external fill:#FFA500,stroke:#333,stroke-width:2px;
    classDef inference fill:#DDA0DD,stroke:#333,stroke-width:2px;
```
