# openbmb/minicpm architecture

> Architecture diagram of the GitHub repository openbmb/minicpm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openbmb/minicpm
- Repository: https://github.com/openbmb/minicpm

## Overview

##### Main Components Identification

The system should be divided into these major components:
- Model Core
- Training Infrastructure
- Inference Services
- Fine-tuning Components
- Quantization Pipeline
- Demo Applications

##### Detailed Component Breakdown

a) Model Core:
- Variants (MiniCPM-2B, MiniCPM-3-4B, MiniCPM-S-1B, etc.)
- Model weights and configuration flow

b) Training Infrastructure:
- Data processing pipeline
- Training configuration management
- Distributed training capabilities
- To fine-tuning components

c) Inference Services:
- Multiple inference backends (HuggingFace, vLLM, SGLang, llama.cpp)
- API layer for model serving
- Chat interface components
- Function calling system

d) Fine-tuning Components:
- Data preparation pipeline
- Configuration management
- Training script execution flow
- Model checkpoint handling

e) Quantization Pipeline:
- Different quantization methods (AWQ, GPTQ, BNB)
- Evaluation framework
- Model compression flow

f) Demo Applications:
- Web interface
- Code interpreter
- Function calling demonstrations
- Integration examples

##### Additional Elements

Include:
- Data flow indicators
- Version control integration
- Configuration management
- Resource requirements
- Scaling capabilities
- Error handling paths

##### Specific Implementation Details

Show:
- Model format conversions
- Integration points with external services
- Deployment options (CPU/GPU)
- Memory management
- Caching mechanisms

##### Documentation References

Add:
- Links to configuration files
- API documentation references
- Model license information
- Version compatibility notes

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Core["Model Core"]
        direction TB
        Base["MiniCPM Base Architecture"]:::core
        M2B["MiniCPM-2B"]:::model
        M34B["MiniCPM-3-4B"]:::model
        MS1B["MiniCPM-S-1B"]:::model
        Base --> M2B
        Base --> M34B
        Base --> MS1B
    end

    subgraph Training["Training Infrastructure"]
        direction TB
        DataProc["Data Processing"]:::training
        TrainConfig["Training Configuration"]:::training
        DistTrain["Distributed Training"]:::training
        DataProc --> TrainConfig
        TrainConfig --> DistTrain
    end
    click DataProc "https://github.com/OpenBMB/MiniCPM/blob/main/finetune/data_processing.ipynb"
    click TrainConfig "https://github.com/OpenBMB/MiniCPM/blob/main/finetune/configs/ds_config_zero2.json"

    subgraph Inference["Inference Services"]
        direction TB
        HF["HuggingFace Backend"]:::inference
        VLLM["vLLM Backend"]:::inference
        MLX["MLX Backend"]:::inference
        LC["LangChain Integration"]:::inference
    end
    click HF "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm/hf_based_demo.py"
    click VLLM "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm/vllm_based_demo.py"
    click MLX "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm/mlx_based_demo.py"
    click LC "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm/langchain_demo.py"

    subgraph FineTune["Fine-tuning Components"]
        direction TB
        DataPrep["Data Preparation"]:::finetune
        FTConfig["Fine-tuning Config"]:::finetune
        LoRA["LoRA Fine-tuning"]:::finetune
        SFT["SFT Fine-tuning"]:::finetune
    end
    click FineTune "https://github.com/OpenBMB/MiniCPM/tree/main/finetune/"
    click FTConfig "https://github.com/OpenBMB/MiniCPM/tree/main/finetune/configs/"
    click LoRA "https://github.com/OpenBMB/MiniCPM/blob/main/finetune/lora_finetune.sh"
    click SFT "https://github.com/OpenBMB/MiniCPM/blob/main/finetune/sft_finetune.sh"

    subgraph Quant["Quantization Pipeline"]
        direction TB
        AWQ["AWQ Quantization"]:::quant
        GPTQ["GPTQ Quantization"]:::quant
        BNB["BNB Quantization"]:::quant
        Eval["Quantization Evaluation"]:::quant
    end
    click AWQ "https://github.com/OpenBMB/MiniCPM/blob/main/quantize/awq_quantize.py"
    click GPTQ "https://github.com/OpenBMB/MiniCPM/blob/main/quantize/gptq_quantize.py"
    click BNB "https://github.com/OpenBMB/MiniCPM/blob/main/quantize/bnb_quantize.py"
    click Eval "https://github.com/OpenBMB/MiniCPM/blob/main/quantize/quantize_eval.py"

    subgraph Demo["Demo Applications"]
        direction TB
        CodeInt["Code Interpreter"]:::demo
        FuncCall["Function Calling"]:::demo
        ToolParser["Tool Parser"]:::demo
        ChatTemp["Chat Template"]:::demo
    end
    click CodeInt "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm3/code_interpreter/code_interpreter.py"
    click FuncCall "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm3/function_call/function_calling.py"
    click ToolParser "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm3/function_call/minicpm_tool_parser.py"
    click ChatTemp "https://github.com/OpenBMB/MiniCPM/blob/main/demo/minicpm3/function_call/minicpm_chat_template_with_tool.jinja"

    %% Connections
    Core --> Inference
    Core --> FineTune
    Core --> Quant
    Training --> FineTune
    Inference --> Demo
    FineTune -.-> Core
    Quant -.-> Core

    %% Styles
    classDef core fill:#2b6cb0,stroke:#1a4971,color:#fff
    classDef model fill:#3182ce,stroke:#2c5282,color:#fff
    classDef training fill:#38a169,stroke:#2f855a,color:#fff
    classDef inference fill:#dd6b20,stroke:#c05621,color:#fff
    classDef finetune fill:#38a169,stroke:#2f855a,color:#fff
    classDef quant fill:#805ad5,stroke:#6b46c1,color:#fff
    classDef demo fill:#d53f8c,stroke:#b83280,color:#fff

    %% Legend
    subgraph Legend
        direction TB
        L1["Core Components"]:::core
        L2["Training & Fine-tuning"]:::training
        L3["Inference Services"]:::inference
        L4["Demo Applications"]:::demo
        L5["Quantization"]:::quant
    end
```
