# duyhai020703/danlp architecture

> Architecture diagram of the GitHub repository duyhai020703/danlp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/duyhai020703/danlp
- Repository: https://github.com/duyhai020703/danlp

## Overview

DANLP is a notebook-led workflow for adapting BLOOM-560M to Vietnamese conversational QA and serving the resulting model. Its documented path runs from JSON corpus preprocessing and prompt/token preparation through LoRA fine-tuning and adapter export, then through model integration and controlled decoding to a Flask chat API, web UI, and optional public tunnel. The repository includes the notebook and datasets, but no source excerpts were provided; implementation links below reflect README-documented architecture, not inspected notebook code.

Main entry points: [`NLP_DA .ipynb`](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb) (Causal LM training), [`Data_DA/train.json`](https://github.com/duyhai020703/danlp/blob/main/Data_DA/train.json) (Training corpus), [`Data_DA/test.json`](https://github.com/duyhai020703/danlp/blob/main/Data_DA/test.json) (Evaluation corpus)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_train_data["Training corpus<br/>[train.json]"]
  node_eval_data["Evaluation corpus<br/>[test.json]"]
  node_preprocess["Clean and normalize<br/>[NLP_DA .ipynb]"]
  node_prompt["Prompt formatting<br/>[NLP_DA .ipynb]"]
  node_tokenize["BLOOM tokenization<br/>[NLP_DA .ipynb]"]
end

subgraph group_training["Model training"]
  node_lora["LoRA adapter setup<br/>[NLP_DA .ipynb]"]
  node_trainer["Causal LM training<br/>[NLP_DA .ipynb]"]
  node_monitoring["Training metrics<br/>[NLP_DA .ipynb]"]
  node_adapter["Exported adapter<br/>[NLP_DA .ipynb]"]
end

subgraph group_serving["Inference and serving"]
  node_integration["PEFT integration<br/>[NLP_DA .ipynb]"]
  node_decoding["Controlled decoding<br/>[NLP_DA .ipynb]"]
  node_api["Flask chat API<br/>[NLP_DA .ipynb]"]
  node_web_ui["Web interface<br/>[NLP_DA .ipynb]"]
  node_tunnel["Ngrok tunnel<br/>[NLP_DA .ipynb]"]
end

node_base_model["BLOOM-560M"]
node_user(("Chat user"))

node_train_data -->|"provides records"| node_preprocess
node_preprocess -->|"cleans text"| node_prompt
node_prompt -->|"formats for"| node_tokenize
node_eval_data -->|"provides evaluation data"| node_tokenize
node_base_model -->|"receives adapter"| node_lora
node_lora -->|"configures"| node_trainer
node_tokenize -->|"supplies tokens"| node_trainer
node_trainer -->|"logs training"| node_monitoring
node_trainer -->|"exports weights"| node_adapter
node_adapter -->|"loads into"| node_integration
node_base_model -->|"provides base weights"| node_integration
node_integration -->|"supplies model"| node_decoding
node_decoding -->|"serves responses"| node_api
node_api -->|"serves interface"| node_web_ui
node_api -.->|"exposes endpoint"| node_tunnel
node_user -->|"submits chat"| node_web_ui

click node_train_data "https://github.com/duyhai020703/danlp/blob/main/Data_DA/train.json"
click node_eval_data "https://github.com/duyhai020703/danlp/blob/main/Data_DA/test.json"
click node_preprocess "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_prompt "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_tokenize "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_lora "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_trainer "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_monitoring "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_adapter "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_integration "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_decoding "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_api "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_web_ui "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"
click node_tunnel "https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_data,node_eval_data,node_preprocess,node_prompt,node_tokenize,node_user toneBlue
class node_lora,node_trainer,node_monitoring,node_adapter toneAmber
class node_integration,node_decoding,node_api,node_web_ui,node_tunnel toneMint
class node_base_model toneIndigo
```

## Components

### Data preparation

- [Training corpus](https://github.com/duyhai020703/danlp/blob/main/Data_DA/train.json): `Data_DA/train.json`
- [Evaluation corpus](https://github.com/duyhai020703/danlp/blob/main/Data_DA/test.json): `Data_DA/test.json`
- [Clean and normalize](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Prompt formatting](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [BLOOM tokenization](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`

### Model training

- [LoRA adapter setup](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Causal LM training](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Training metrics](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Exported adapter](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`

### Inference and serving

- [PEFT integration](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Controlled decoding](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Flask chat API](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Web interface](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`
- [Ngrok tunnel](https://github.com/duyhai020703/danlp/blob/main/NLP_DA%20.ipynb): `NLP_DA .ipynb`

### Other components

- **BLOOM-560M**
- **Chat user**

## Connections

- Training corpus → Clean and normalize: provides records (evidence: `README.md`)
- Clean and normalize → Prompt formatting: cleans text (evidence: `README.md`)
- Prompt formatting → BLOOM tokenization: formats for (evidence: `README.md`)
- Evaluation corpus → BLOOM tokenization: provides evaluation data (evidence: `README.md`)
- BLOOM-560M → LoRA adapter setup: receives adapter (evidence: `README.md`)
- LoRA adapter setup → Causal LM training: configures (evidence: `README.md`)
- BLOOM tokenization → Causal LM training: supplies tokens (evidence: `README.md`)
- Causal LM training → Training metrics: logs training (evidence: `README.md`)
- Causal LM training → Exported adapter: exports weights (evidence: `README.md`)
- Exported adapter → PEFT integration: loads into (evidence: `README.md`)
- BLOOM-560M → PEFT integration: provides base weights (evidence: `README.md`)
- PEFT integration → Controlled decoding: supplies model (evidence: `README.md`)
- Controlled decoding → Flask chat API: serves responses (evidence: `README.md`)
- Flask chat API → Web interface: serves interface (evidence: `README.md`)
- Flask chat API → Ngrok tunnel: exposes endpoint (evidence: `README.md`)
- Chat user → Web interface: submits chat (evidence: `README.md`)
