# 01-ai/yi architecture

> Architecture diagram of the GitHub repository 01-ai/yi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/01-ai/yi
- Repository: https://github.com/01-ai/yi

## Overview

##### Main Components

a. Core Model Components:
- Base Models (Yi-6B, Yi-9B, Yi-34B)
- Chat Models (Yi-6B-Chat, Yi-34B-Chat)
- Vision-Language Models (Yi-VL)
- Model Variants (200K context window versions)

b. Training & Fine-tuning Infrastructure:
- Training Pipeline
- Fine-tuning Module
- Quantization Systems (AWQ, GPTQ)

c. Deployment & Inference:
- Serving Layer
- API Integration
- Web Demo Interface
- Docker Container Support

##### Relationships and Data Flow

- Data flow from base models to chat models
- Fine-tuning process flow
- Quantization pathways
- Deployment options
- API integration points

##### Architectural Patterns

- Modular architecture allowing different components to be used independently
- Pipeline structure for training and inference
- Integration points with external tools and frameworks
- Scalability aspects for different deployment scenarios

##### Specific Implementation Details

Specialized components for:
- Model Architecture (based on Llama)
- Tokenization
- Multi-modal Support (VL models)
- Inference Optimization

**[Training Data & Resources] → [Core Model Layer]**
- ↓
**[Fine-tuning Module] ←→ [Base Models] → [Chat Models]**
- ↓                     ↓             ↓
**[Quantization Layer] → [Deployment Layer] ← [VL Models]**
- ↓                     ↓
**[Serving Infrastructure] → [API Layer]**
- ↓
- [Applications & Interfaces]

##### Additional Elements

Include:
- Version information
- Hardware requirements
- Integration points with external tools
- Scaling capabilities

##### Technical Details to Highlight

- Context window sizes (4K, 200K)
- Supported quantization methods
- Hardware requirements for different components
- Integration with Hugging Face, ModelScope, etc.
- Docker container support
- Fine-tuning capabilities

##### Implementation Notes

- Modular design allowing component reuse
- Clear separation of concerns
- Scalability options
- Integration flexibility
- Support for different deployment scenarios

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Core["Core Model Components"]
        BaseModels["Base Models\n(Yi-6B, Yi-9B, Yi-34B)"]
        ChatModels["Chat Models\n(Yi-6B-Chat, Yi-34B-Chat)"]
        VisionModels["Vision-Language Models"]:::vl
    end

    subgraph Training["Training & Fine-tuning"]
        FineTune["Fine-tuning Module"]:::finetune
        TrainScripts["Training Scripts"]:::trainscripts
        Utils["Utilities"]:::utils
        ExampleData["Example Dataset"]:::dataset
    end

    subgraph Quant["Quantization Layer"]
        AWQ["AWQ Implementation"]:::awq
        GPTQ["GPTQ Implementation"]:::gptq
    end

    subgraph Deploy["Deployment & Inference"]
        Transformers["Transformers"]:::transformers
        vLLM["vLLM"]:::vllm
        LMDeploy["LMDeploy"]:::lmdeploy
    end

    subgraph Local["Local Deployment"]
        LlamaCpp["Llama.cpp"]:::llamacpp
        Ollama["Ollama"]:::ollama
    end

    subgraph Interface["User Interfaces"]
        WebDemo["Web Demo Interface"]:::webdemo
        TextGen["Text Generation"]:::textgen
    end

    subgraph Integration["Integration Components"]
        RAG["RAG Implementation"]:::rag
        FuncCall["Function Calling"]:::funccall
    end

    BaseModels --> ChatModels
    BaseModels --> VisionModels
    FineTune --> BaseModels
    TrainScripts --> FineTune
    Utils --> FineTune
    ExampleData --> FineTune
    
    BaseModels --> AWQ
    BaseModels --> GPTQ
    
    AWQ --> Deploy
    GPTQ --> Deploy
    
    Deploy --> Local
    Deploy --> Interface
    
    VisionModels --> Interface
    ChatModels --> Interface
    
    Deploy --> Integration
    
    %% Click Events
    click VisionModels "https://github.com/01-ai/Yi/tree/main//VL"
    click FineTune "https://github.com/01-ai/Yi/tree/main//finetune"
    click TrainScripts "https://github.com/01-ai/Yi/tree/main//finetune/scripts"
    click Utils "https://github.com/01-ai/Yi/tree/main//finetune/utils"
    click ExampleData "https://github.com/01-ai/Yi/tree/main//finetune/yi_example_dataset"
    click AWQ "https://github.com/01-ai/Yi/tree/main//quantization/awq"
    click GPTQ "https://github.com/01-ai/Yi/tree/main//quantization/gptq"
    click Transformers "https://github.com/01-ai/Yi/blob/main//Cookbook/en/opensource/Inference/Inference_using_transformers.ipynb"
    click vLLM "https://github.com/01-ai/Yi/blob/main//Cookbook/en/opensource/Inference/vLLM_Inference_tutorial.ipynb"
    click LMDeploy "https://github.com/01-ai/Yi/blob/main//Cookbook/en/opensource/Inference/Inference_using_lmdeploy.ipynb"
    click WebDemo "https://github.com/01-ai/Yi/blob/main//demo/web_demo.py"
    click TextGen "https://github.com/01-ai/Yi/blob/main//demo/text_generation.py"
    click RAG "https://github.com/01-ai/Yi/tree/main//Cookbook/en/opensource/rag"
    click FuncCall "https://github.com/01-ai/Yi/tree/main//Cookbook/en/opensource/function_calling"
    click LlamaCpp "https://github.com/01-ai/Yi/blob/main//Cookbook/en/opensource/local/local-llama.cpp.md"
    click Ollama "https://github.com/01-ai/Yi/blob/main//Cookbook/en/opensource/local/local-ollama.md"

    %% Styling
    classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px
    classDef vl fill:#e1f5fe,stroke:#0288d1
    classDef finetune fill:#e8f5e9,stroke:#388e3c
    classDef trainscripts fill:#e8f5e9,stroke:#388e3c
    classDef utils fill:#e8f5e9,stroke:#388e3c
    classDef dataset fill:#e8f5e9,stroke:#388e3c
    classDef awq fill:#fff3e0,stroke:#f57c00
    classDef gptq fill:#fff3e0,stroke:#f57c00
    classDef transformers fill:#f3e5f5,stroke:#7b1fa2
    classDef vllm fill:#f3e5f5,stroke:#7b1fa2
    classDef lmdeploy fill:#f3e5f5,stroke:#7b1fa2
    classDef llamacpp fill:#e0f2f1,stroke:#00796b
    classDef ollama fill:#e0f2f1,stroke:#00796b
    classDef webdemo fill:#e8eaf6,stroke:#3f51b5
    classDef textgen fill:#e8eaf6,stroke:#3f51b5
    classDef rag fill:#fce4ec,stroke:#c2185b
    classDef funccall fill:#fce4ec,stroke:#c2185b
```
