# pengxiao-song/lawgpt architecture

> Architecture diagram of the GitHub repository pengxiao-song/lawgpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pengxiao-song/lawgpt
- Repository: https://github.com/pengxiao-song/lawgpt

## Overview

No explanation provided

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Legend
        style Legend fill:#f0f0f0,stroke:#333,stroke-width:2px
        PROC[Processing Component]:::blue
        DATA[(Data Storage)]:::gray
        MODEL{Model Component}:::green
        UI[User Interface]:::orange
        DEC{Decision Point}:::yellow
    end

    subgraph DataLayer["Data Processing Layer"]
        RawData[(Raw Legal Data)]:::gray
        LegalRes[("Legal Resources")]:::gray
        click LegalRes "https://github.com/pengxiao-song/LaWGPT/tree/main/resources"
        DataClean["Data Cleaning"]:::blue
        click DataClean "https://github.com/pengxiao-song/LaWGPT/blob/main/tools/clear_law.py"
        VocabProc["Vocabulary Processing"]:::blue
        click VocabProc "https://github.com/pengxiao-song/LaWGPT/blob/main/tools/merge_vocabulary.py"
        Templates[("Legal Templates")]:::gray
        click Templates "https://github.com/pengxiao-song/LaWGPT/blob/main/templates/law_template.json"
    end

    subgraph TrainingLayer["Training Pipeline"]
        PreTrain["Pre-training"]:::blue
        click PreTrain "https://github.com/pengxiao-song/LaWGPT/blob/main/train_clm.py"
        FineTune["Fine-tuning"]:::blue
        click FineTune "https://github.com/pengxiao-song/LaWGPT/blob/main/finetune.py"
        TrainScripts["Training Scripts"]:::blue
        click TrainScripts "https://github.com/pengxiao-song/LaWGPT/tree/main/scripts"
    end

    subgraph ModelLayer["Model Layer"]
        BaseModel{Chinese-LLaMA}:::green
        click BaseModel "https://github.com/pengxiao-song/LaWGPT/tree/main/models/base_models"
        LoraWeights{LoRA Weights}:::green
        click LoraWeights "https://github.com/pengxiao-song/LaWGPT/tree/main/models/lora_weights"
        ModelMerge["Model Merging"]:::blue
        click ModelMerge "https://github.com/pengxiao-song/LaWGPT/blob/main/merge.py"
    end

    subgraph InferenceLayer["Inference Layer"]
        WebUI["Web UI Interface"]:::orange
        click WebUI "https://github.com/pengxiao-song/LaWGPT/blob/main/webui.py"
        CLI["Command-line Interface"]:::orange
        click CLI "https://github.com/pengxiao-song/LaWGPT/blob/main/infer.py"
        InferScripts["Inference Scripts"]:::blue
        click InferScripts "https://github.com/pengxiao-song/LaWGPT/tree/main/scripts"
    end

    subgraph Utils["Supporting Components"]
        Evaluation["Evaluation Tools"]:::blue
        click Evaluation "https://github.com/pengxiao-song/LaWGPT/blob/main/utils/evaluate.py"
        Callbacks["Callback Handlers"]:::blue
        click Callbacks "https://github.com/pengxiao-song/LaWGPT/blob/main/utils/callbacks.py"
        Prompter["Prompt Processing"]:::blue
        click Prompter "https://github.com/pengxiao-song/LaWGPT/blob/main/utils/prompter.py"
    end

    %% Relationships
    RawData --> DataClean
    LegalRes --> DataClean
    DataClean --> VocabProc
    VocabProc --> Templates
    Templates --> PreTrain
    PreTrain --> FineTune
    TrainScripts --> PreTrain
    TrainScripts --> FineTune
    BaseModel --> ModelMerge
    LoraWeights --> ModelMerge
    ModelMerge --> WebUI
    ModelMerge --> CLI
    InferScripts --> WebUI
    InferScripts --> CLI
    Evaluation --> ModelMerge
    Callbacks --> TrainScripts
    Prompter --> InferScripts

    %% Styles
    classDef blue fill:#3498db,stroke:#2980b9,color:white
    classDef gray fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef green fill:#2ecc71,stroke:#27ae60,color:white
    classDef orange fill:#e67e22,stroke:#d35400,color:white
    classDef yellow fill:#f1c40f,stroke:#f39c12,color:white

    %% Notes
    note["Hardware: 8x Tesla V100-SXM2-32GB<br/>Pre-training: 24h/epoch<br/>Fine-tuning: 12h/epoch<br/>Data: 350k legal QA pairs"]
    style note fill:#f9f9f9,stroke:#666,stroke-width:1px
```
