# dptech-corp/uni-core architecture

> Architecture diagram of the GitHub repository dptech-corp/uni-core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dptech-corp/uni-core
- Repository: https://github.com/dptech-corp/uni-core

## Overview

##### Project type and purpose
- Uni-Core is a high-performance, distributed PyTorch framework optimized for Transformer-based models.
- It provides Python APIs, command-line tools, fused CUDA kernels, distributed multi-GPU/multi-node training, mixed-precision support, optimizers, schedulers, data loaders, logging, checkpointing, and Docker support.

##### File structure
- Top-level directories:
- csrc: native C++/CUDA extension sources (kernels for Adam, layernorm, softmax_dropout, etc.)
- unicore: core Python package (subpackages: data, distributed, logging, losses, models, modules, optim, tasks, trainer, utils)
- unicore_cli: command-line entrypoints (train.py)
- examples/bert: reference example showing how to instantiate datasets, models, tasks, and start training
- docker: Dockerfiles for RDMA/GPU environments
- tests: unit tests for Python modules
- .github/workflows: CI for building wheels across CUDA versions, publishing, and Docker
- setup.py, requirements.txt: packaging and dependencies
- Patterns:
- Python package with clear separation of concerns (data / model / module / task / training orchestrator)
- Native extensions for performance-critical kernels, loaded at runtime
- CLI wrapper around Python API for user entrypoint
- CI pipelines for multi-CUDA builds and releases

##### README for additional insights
- Confirms project features: distributed training, mixed precision, fused CUDA, checkpoint management, logging, buffered data loader, gradient accumulation, optimizers, LR schedulers
- Mentions dependency on PyTorch and matching CUDA version
- References Docker image and example usage

a. Main components
1. User Interfaces
- CLI (unicore_cli/train.py)
- Example scripts (examples/bert/model.py, task.py)
2. Uni-Core Python Library (unicore/…)
- Data Layer: data loaders & datasets (unicore/data)
- Model Layer: modules & model abstractions (unicore/modules, unicore/models)
- Task Layer: high-level training definitions (unicore/tasks)
- Trainer: orchestration of epoch/step loop, gradient accumulation (unicore/trainer.py)
- Optimization: optimizers & schedulers (unicore/optim)
- Logging: metrics, progress bars (unicore/logging)
- Checkpointing & utils (unicore/checkpoint_utils, unicore/utils)
- Distributed Support: wrappers for DDP, multi-node utils (unicore/distributed)
3. Native Extension
- csrc: CUDA kernels compiled into shared objects loaded by Python
- Fused ops: Adam, layernorm, RMSNorm, softmax+dropout, rounding, multi-tensor
4. External Dependencies
- PyTorch (autograd, DDP, mixed precision API)
- CUDA runtime & GPU hardware
- MPI/RDMA (via docker/rdma) for multi-node communication
5. Infrastructure
- Docker environment for consistent GPU/OS setup
- GitHub Actions for building/publishing wheels

b. Relationships and interactions
- User invokes CLI or example script → Python API
- Python API uses Data Layer to feed mini-batches
- Model Layer builds computational graph with PyTorch modules
- Trainer orchestrates forward/backward passes, calls fused CUDA kernels via native extension
- Optimizer & LR scheduler update parameters (FP32/FP16/bf16)
- DDP wrappers distribute training loops across GPUs/nodes, use CUDA kernels and RDMA for communication
- Logging & checkpointing record metrics and save model state

c. Architectural patterns and principles
- Modular layer separation (Data, Model, Task, Trainer, Optim, Logging)
- Extension of PyTorch (wrapper pattern) rather than reinventing tensor APIs
- Native plugin for hot-path kernels (performance)
- CLI + Python API dual entrypoints for flexibility
- CI/CD ensures multi-CUDA compatibility and reliable releases

d. Key technologies, frameworks, libraries
- Python 3.x, PyTorch
- CUDA (cu102–cu118)
- C++/CUDA fusion kernels
- Docker + nvidia-docker
- GitHub Actions for cross-CUDA builds

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Points
    subgraph "Entry Points"
        CLI["CLI Interface"]:::user
        EX_MODEL["BERT Model Script"]:::user
        EX_TASK["BERT Task Script"]:::user
        EX_SH1["train_bert_test.sh"]:::user
        EX_SH2["train_bert_test_multi_node.sh"]:::user
    end

    %% Core Library
    subgraph "Uni-Core Python Library"
        subgraph "Data Layer"
            DataLoader["Data Loader"]:::python
        end
        subgraph "Model Layer"
            Modules["Modules (layer_norm, attention, etc.)"]:::python
            Models["Models (unicore_model, distributed_unicore_model)"]:::python
        end
        subgraph "Task Layer"
            Task["UniCore Task"]:::python
        end
        subgraph "Trainer"
            Trainer["Trainer"]:::python
        end
        subgraph "Optimization"
            Optimizers["Optimizers & Schedulers"]:::python
        end
        subgraph "Logging"
            Logging["Logging & Metrics"]:::python
        end
        subgraph "Checkpoint & Utils"
            Checkpoint["Checkpoint Utils"]:::python
            Utils["Utilities"]:::python
            Options["Options & Registry"]:::python
        end
        subgraph "Distributed Support"
            Dist["Distributed Wrappers & Utils"]:::python
        end
    end

    %% Native Extensions
    subgraph "Native CUDA Extensions"
        Kernels["Fused CUDA Kernels"]:::native
    end

    %% External Dependencies
    subgraph "External Dependencies"
        PyTorch["PyTorch"]:::external
        CUDA["CUDA Runtime & GPU"]:::external
        MPI["MPI/RDMA"]:::external
    end

    %% Infrastructure
    subgraph "Infrastructure"
        Docker["Docker Environment"]:::infra
        CI["GitHub Actions CI"]:::infra
        Packaging["Packaging & Docs"]:::infra
        Versioning["Versioning"]:::infra
        Tests["Unit Tests"]:::infra
    end

    %% Relationships
    CLI -->|"invokes"| Trainer
    EX_MODEL -->|"imports"| Task
    EX_TASK -->|"defines"| Task
    EX_SH1 -->|"runs"| CLI
    EX_SH2 -->|"runs"| CLI

    Trainer -->|"uses data"| DataLoader
    Trainer -->|"builds graph"| Modules
    Modules -->|"constructs"| Models
    Task -->|"configures"| Trainer
    Trainer -->|"calls kernels"| Kernels
    Trainer -->|"calls optim"| Optimizers
    Optimizers -->|"update params"| Models
    Trainer -->|"logs"| Logging
    Trainer -->|"saves"| Checkpoint
    Trainer -->|"util funcs"| Utils
    Trainer -->|"load options"| Options
    Trainer -->|"distributed"| Dist
    Dist -->|"wraps"| PyTorch
    Kernels -->|"executed on"| CUDA
    DataLoader -->|"buffers"| MPI
    Docker -->|"provides"| CUDA
    CI -->|"builds"| Packaging
    Packaging -->|"includes"| Versioning
    Tests -->|"validate"| Modules

    %% Click Events
    click CLI "https://github.com/dptech-corp/uni-core/blob/main/unicore_cli/train.py"
    click EX_MODEL "https://github.com/dptech-corp/uni-core/blob/main/examples/bert/model.py"
    click EX_TASK "https://github.com/dptech-corp/uni-core/blob/main/examples/bert/task.py"
    click EX_SH1 "https://github.com/dptech-corp/uni-core/blob/main/examples/bert/train_bert_test.sh"
    click EX_SH2 "https://github.com/dptech-corp/uni-core/blob/main/examples/bert/train_bert_test_multi_node.sh"
    click DataLoader "https://github.com/dptech-corp/uni-core/tree/main/unicore/data/"
    click Modules "https://github.com/dptech-corp/uni-core/tree/main/unicore/modules/"
    click Models "https://github.com/dptech-corp/uni-core/tree/main/unicore/models/"
    click Task "https://github.com/dptech-corp/uni-core/blob/main/unicore/tasks/unicore_task.py"
    click Trainer "https://github.com/dptech-corp/uni-core/blob/main/unicore/trainer.py"
    click Optimizers "https://github.com/dptech-corp/uni-core/tree/main/unicore/optim/"
    click Logging "https://github.com/dptech-corp/uni-core/tree/main/unicore/logging/"
    click Checkpoint "https://github.com/dptech-corp/uni-core/blob/main/unicore/checkpoint_utils.py"
    click Utils "https://github.com/dptech-corp/uni-core/blob/main/unicore/utils.py"
    click Options "https://github.com/dptech-corp/uni-core/blob/main/unicore/options.py"
    click Options "https://github.com/dptech-corp/uni-core/blob/main/unicore/registry.py"
    click Dist "https://github.com/dptech-corp/uni-core/tree/main/unicore/distributed/"
    click Kernels "https://github.com/dptech-corp/uni-core/tree/main/csrc/"
    click Docker "https://github.com/dptech-corp/uni-core/tree/main/docker/rdma/Dockerfile"
    click CI "https://github.com/dptech-corp/uni-core/tree/main/.github/workflows/"
    click Packaging "https://github.com/dptech-corp/uni-core/blob/main/setup.py"
    click Packaging "https://github.com/dptech-corp/uni-core/blob/main/requirements.txt"
    click Packaging "https://github.com/dptech-corp/uni-core/blob/main/MANIFEST.in"
    click Packaging "https://github.com/dptech-corp/uni-core/tree/main/LICENSE"
    click Packaging "https://github.com/dptech-corp/uni-core/blob/main/README.md"
    click Versioning "https://github.com/dptech-corp/uni-core/blob/main/unicore/version.py"
    click Versioning "https://github.com/dptech-corp/uni-core/blob/main/unicore/version.txt"
    click Tests "https://github.com/dptech-corp/uni-core/blob/main/tests/test_softmax.py"

    %% Styles
    classDef python fill:#cde6f7,stroke:#2c7bb6,color:#000;
    classDef native fill:#fdd49e,stroke:#d94801,color:#000;
    classDef external fill:#e0e0e0,stroke:#888,color:#000;
    classDef user fill:#d4efdf,stroke:#27ae60,color:#000;
    classDef infra fill:#efedf5,stroke:#6a51a3,color:#000;
```
