# tencent-hunyuan/hunyuan-a13b architecture

> Architecture diagram of the GitHub repository tencent-hunyuan/hunyuan-a13b, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tencent-hunyuan/hunyuan-a13b
- Repository: https://github.com/tencent-hunyuan/hunyuan-a13b

## Overview

##### Project type and purpose
- This is an open-source large-language-model (LLM) repository (Hunyuan-A13B) providing:
- Model definition & configuration (Mixture-of-Experts architecture)
- Training pipelines (data prep, fine-tuning, merging LoRA weights)
- Inference pipelines (API server scripts for various backends: TensorRT-LLM, vLLM, SGLang)
- Demo clients, agent tools, benchmarks, and quantized model support

##### File structure
- Top level:
- `models/` – model code (configuration, tokenization, modeling)
- `train/` – training configs, scripts, example data, merge utilities, DS-Zero JSON files
- `inference/` – scripts to launch API servers (FP8, Int4, Int8, BF16) and client helpers
- `examples/` – sample data, training & inference demos, eval scripts
- `agent/` – agent-style demo tools (Excel parsing, MCP demo, custom parsers)
- `benchmark/` – benchmark scripts and results
- `report/` – the technical PDF report
- root files – READMEs, licensing, requirements, submodule references

##### README for additional insights
- Core LLM (80B total, 13B active parameters) with MoE
- Dual “thinking” modes (fast/slow reasoning)
- Ultra-long context (256K tokens)
- Quantization via AngleSlim: FP8 and GPTQ-Int4
- Supported backends: TensorRT-LLM, vLLM, SGLang
- Deployment via Docker images, OpenAI-compatible API

##### Final Recommendations
- Be as granular as needed: e.g., separate “Tokenization” from “Model Forward” if desired
- Both horizontal layers (Data→Training→Deployment) and vertical stacks (Model code reused across pipelines)
- With versioned config files (e.g., `ds_zero3_offload.json`) to highlight custom setups

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DL1["examples/data/"]:::data
        DL2["examples/data/example_data.jsonl"]:::data
        DL3["train/example_data.jsonl"]:::data
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        TP1["train/train.py"]:::training
        TP2["train/train.sh"]:::training
        TP3["train/merge_lora_weight.py"]:::training
        TP4["train/merge_lora_weight.sh"]:::training
        TP5["train/ds_zero2_no_offload.json"]:::training
        TP6["train/ds_zero3_no_offload.json"]:::training
        TP7["train/ds_zero3_offload.json"]:::training
        TP8["train/ds_zero3_offload_no_auto.json"]:::training
        TP9["train/llama_factory_support/"]:::training
    end

    %% Model Code
    subgraph "Model Code"
        direction TB
        MC1["models/configuration_hunyuan.py"]:::model
        MC2["models/modeling_hunyuan.py"]:::model
        MC3["models/tokenization_hy.py"]:::model
        MC4["models/hy.tiktoken"]:::model
        MC5["models/hunyuan.py"]:::model
        MC6["models/config.json"]:::model
        MC7["models/generation_config.json"]:::model
        MC8["models/tokenizer_config.json"]:::model
    end

    %% Trained Weights
    TW[(Trained Weights)]:::storage

    %% Quantization Module
    subgraph "Quantization Module"
        direction TB
        Q1["inference/run_server_fp8.sh"]:::quant
        Q2["inference/run_server_int4.sh"]:::quant
        Q3["inference/run_server_int8.sh"]:::quant
    end

    QM[(Quantized Models)]:::storage

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        direction TB
        IP1["inference/run_server.sh"]:::inference
        IP2["inference/login_ssh.py"]:::inference
        IP3["inference/openapi.sh"]:::inference
    end

    %% Client/Agent Layer
    subgraph "Client/Agent Layer"
        direction TB
        CA1["examples/eval_demo_vllm.py"]:::client
        CA2["examples/train_demo.sh"]:::client
        CA3["examples/build_env.sh"]:::client
        CA4["agent/gen_train_data.py"]:::client
        CA5["agent/hunyuan_tool_parser.py"]:::client
        CA6["agent/excel_demo/demo.py"]:::client
        CA7["agent/excel_demo/requirements.txt"]:::client
        CA8["agent/mcp_demo/demo.py"]:::client
        CA9["agent/mcp_demo/requirements.txt"]:::client
    end

    %% Benchmark and Report
    subgraph "Benchmark & Report"
        direction TB
        BM["benchmark/"]:::benchmark
        BM2["benchmark/README.md"]:::benchmark
        RP["report/Hunyuan_A13B_Technical_Report.pdf"]:::report
    end

    %% External Services
    subgraph "External Services"
        direction TB
        ES1["Hugging Face / ModelScope"]:::external
        ES2["Docker Hub"]:::external
        ES3["Cloud API Endpoints"]:::external
    end

    %% Flows
    DL1 --> DL2 --> DL3 --> TP1
    TP1 --> TP2 --> TP3 --> TP4
    TP2 --> TP5
    TP2 --> TP6
    TP2 --> TP7
    TP2 --> TP8
    TP2 --> TP9
    TP4 --> MC1
    MC1 --> MC2 --> MC3 --> MC4 --> MC5 --> MC6 --> MC7 --> MC8
    MC8 --> TW
    TW --> Q1
    TW --> IP1
    Q1 --> QM
    QM --> IP1
    IP1 --> IP2 --> IP3
    IP3 <--> CA1
    IP3 <--> CA4
    IP3 <--> CA6
    IP3 --> ES1
    IP3 --> ES2
    IP3 --> ES3
    IP1 --> BM
    MC2 --> BM2
    IP1 --> RP

    %% Click Events
    click DL1 "https://github.com/tencent-hunyuan/hunyuan-a13b/tree/main/examples/data/"
    click DL2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/examples/data/example_data.jsonl"
    click DL3 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/example_data.jsonl"
    click TP1 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/train.py"
    click TP2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/train.sh"
    click TP3 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/merge_lora_weight.py"
    click TP4 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/merge_lora_weight.sh"
    click TP5 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/ds_zero2_no_offload.json"
    click TP6 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/ds_zero3_no_offload.json"
    click TP7 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/ds_zero3_offload.json"
    click TP8 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/train/ds_zero3_offload_no_auto.json"
    click TP9 "https://github.com/tencent-hunyuan/hunyuan-a13b/tree/main/train/llama_factory_support/"
    click MC1 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/configuration_hunyuan.py"
    click MC2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/modeling_hunyuan.py"
    click MC3 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/tokenization_hy.py"
    click MC4 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/hy.tiktoken"
    click MC5 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/hunyuan.py"
    click MC6 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/config.json"
    click MC7 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/generation_config.json"
    click MC8 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/models/tokenizer_config.json"
    click Q1 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/inference/run_server_fp8.sh"
    click Q2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/inference/run_server_int4.sh"
    click Q3 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/inference/run_server_int8.sh"
    click IP1 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/inference/run_server.sh"
    click IP2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/inference/login_ssh.py"
    click IP3 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/inference/openapi.sh"
    click CA1 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/examples/eval_demo_vllm.py"
    click CA2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/examples/train_demo.sh"
    click CA3 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/examples/build_env.sh"
    click CA4 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/agent/gen_train_data.py"
    click CA5 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/agent/hunyuan_tool_parser.py"
    click CA6 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/agent/excel_demo/demo.py"
    click CA7 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/agent/excel_demo/requirements.txt"
    click CA8 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/agent/mcp_demo/demo.py"
    click CA9 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/agent/mcp_demo/requirements.txt"
    click BM "https://github.com/tencent-hunyuan/hunyuan-a13b/tree/main/benchmark/"
    click BM2 "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/benchmark/README.md"
    click RP "https://github.com/tencent-hunyuan/hunyuan-a13b/blob/main/report/Hunyuan_A13B_Technical_Report.pdf"

    %% Styles
    classDef data fill:#E0F7FA,stroke:#006064
    classDef training fill:#D0E6FF,stroke:#003366
    classDef model fill:#D6FFD6,stroke:#006600
    classDef quant fill:#DFFFD6,stroke:#669900
    classDef storage fill:#FFF1A8,stroke:#CCCC00
    classDef inference fill:#FFE0B2,stroke:#E65100
    classDef client fill:#FFF9C4,stroke:#F57F17
    classDef external fill:#CFD8DC,stroke:#37474F
    classDef benchmark fill:#F0E0FF,stroke:#6A1B9A
    classDef report fill:#F8F0F0,stroke:#B71C1C
```
