# amd-agi/primus architecture

> Architecture diagram of the GitHub repository amd-agi/primus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amd-agi/primus
- Repository: https://github.com/amd-agi/primus

## Overview

##### Project Type & Purpose
- Primus is not a web app but a full-scale, plugin-based distributed training framework for large foundation models (pretraining, posttraining, RL).
- It exposes a CLI/frontend, core orchestration and utils layers, pluggable “modules” and “backends,” and a suite of tools (benchmark, preflight, visualization).

##### Main Architectural Components
A. Frontend / Entrypoints
- CLI layer (primus/cli + pretrain.py + runner scripts)
- Example launcher scripts (examples/, runner/)
B. Core Orchestration
- Config parser & validator (primus/core/launcher/parser.py)
- Environment initialization & logging (primus/core/utils)
- YAML/constant utilities
C. Module Layer
- Abstract Module API (primus/modules/base_module.py)
- Trainer implementations per backend (Megatron, TorchTitan, JAX)
D. Backend Layer
- Backend adapters (primus/backends/*)
- Core distributed primitives: tensor + pipeline parallel, checkpointing, collectives, MOE, FP8/fusions
- Low-level kernels invoke Primus-Turbo operators or ROCm libraries
E. Platform Layer
- Platform abstraction (Local vs. Remote) for filesystem, Slurm/K8s integration
F. Tools & Benchmarks
- Preflight sanity checks (tools/preflight)
- Kernel‐ and model‐level benchmarks (benchmark/)
- Visualization of pipeline schedules (tools/visualization)
G. External Dependencies & Infrastructure
- ROCm-enabled Docker images
- HPC interconnect (NCCL/RCCL)
- Third-party submodules (Megatron-LM, TorchTitan)
- Kubernetes/Slurm schedulers
- AMD GPU hardware

##### Relationships & Data Flows
- User CLI ➔ Core Launcher: reads and validates YAML
- Core Launcher ➔ Module Loader: instantiates correct Trainer module
- Trainer Module ➔ Backend APIs: sets up distributed parallelism (tensor, pipeline, FSDP)
- Backend Core ➔ Operator Kernel Library: calls optimized GEMM, attention, collectives (Primus-Turbo/AITER/hipBLASLt/Triton)
- Backend Core ➔ Device Runtime (ROCm) ➔ GPU(s)
- Platform Layer manages job execution environment (local process vs. Slurm job vs. K8s pod)
- Tools & Benchmarks invoke kernels or full training flows and collect metrics

##### Key Architectural Patterns
- Layered / clean separation (CLI → Core → Modules → Backends → Hardware)
- Plugin/adapter pattern for “backends” and “modules”
- Configuration-driven wiring (YAML defines which backend, model, precision, parallelism)
- Distributed pipeline + tensor parallel scheduling with zero-bubble optimizations
- Extensibility points: new backends, new modules, new tools

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend / Entrypoints
    subgraph "Frontend / Entrypoints"
        CLI1["CLI: main.py"]:::frontend
        CLI2["CLI: train_cli.py"]:::frontend
        PRE["Pretrain script"]:::frontend
        EXAMPLES["Examples"]:::frontend
        RUNNER_SCRIPTS["Runner scripts"]:::frontend
    end

    %% Core Orchestration
    subgraph "Core Orchestration"
        PARSER["Config parser & validator"]:::core
        CFGLOAD["Config loader"]:::core
        INIT["Launcher initialize"]:::core
        ENV["Env setup"]:::core
        LOG["Logging"]:::core
        YAML_UTILS["YAML utilities"]:::core
        CONST_VARS["Constants"]:::core
    end

    %% Module Layer
    subgraph "Module Layer"
        BASE_MODULE["Abstract Module API"]:::module
        MODULE_UTILS["Module utilities"]:::module
        BASE_TRAINER["Trainer interface"]:::module
        JAX_TRAINER["JAX Trainer impl"]:::module
        MEGA_TRAINER["Megatron Trainer impl"]:::module
        TITAN_PRE["TorchTitan pre_trainer"]:::module
        TITAN_UTILS["TorchTitan parse_utils"]:::module
    end

    %% Backend Layer
    subgraph "Backend Layer"
        MEGA_BACKEND["Megatron backend"]:::backend
        JAX_BACKEND["JAX backend"]:::backend
        TITAN_BACKEND["TorchTitan backend"]:::backend
        TRANS_ENGINE["TransformerEngine"]:::backend
        KERNEL_LIB["Primus-Turbo kernels"]:::backend
        DEVICE_RT["ROCm runtime"]:::backend
    end

    %% Platform Layer
    subgraph "Platform Layer"
        BASE_PLATFORM["Abstract platform"]:::platform
        LOCAL_PLATFORM["Local platform"]:::platform
        REMOTE_PLATFORM["Remote platform"]:::platform
    end

    %% Tools & Benchmarks
    subgraph "Tools & Benchmarks"
        PREFLIGHT["Preflight checks"]:::tool
        BENCHMARK["Benchmarks"]:::tool
        VIS["Pipeline visualization"]:::tool
    end

    %% External Dependencies & Infrastructure
    subgraph "External Systems"
        SUB_MEGATRON["Megatron-LM submodule"]:::external
        SUB_TITAN["TorchTitan submodule"]:::external
        CI["CI workflows"]:::external
        DOCKER["Docker scripts"]:::external
        RUNNER_ENV["Runner env scripts"]:::external
        ROCM["ROCm Docker image"]:::external
        NCCL["NCCL/RCCL"]:::external
        K8S["Kubernetes/Slurm"]:::external
        GPU["AMD GPU hardware"]:::external
    end

    %% Connections
    CLI1 -->|parseConfig| PARSER
    CLI2 -->|parseConfig| PARSER
    PRE -->|parseConfig| PARSER
    EXAMPLES -->|invoke| PARSER
    RUNNER_SCRIPTS -->|invoke| PARSER

    PARSER -->|loadConfig| CFGLOAD
    CFGLOAD -->|initialize| INIT
    INIT --> ENV
    INIT --> LOG
    PARSER --> YAML_UTILS
    PARSER --> CONST_VARS

    INIT -->|launchTrainer| BASE_MODULE
    BASE_MODULE --> MODULE_UTILS
    BASE_MODULE --> BASE_TRAINER
    BASE_TRAINER --> JAX_TRAINER
    BASE_TRAINER --> MEGA_TRAINER
    BASE_TRAINER --> TITAN_PRE
    TITAN_PRE --> TITAN_UTILS

    JAX_TRAINER --> JAX_BACKEND
    MEGA_TRAINER --> MEGA_BACKEND
    TITAN_PRE --> TITAN_BACKEND

    MEGA_BACKEND --> KERNEL_LIB
    JAX_BACKEND --> KERNEL_LIB
    TITAN_BACKEND --> KERNEL_LIB
    TRANS_ENGINE --> KERNEL_LIB

    KERNEL_LIB --> DEVICE_RT
    DEVICE_RT --> GPU

    INIT --> BASE_PLATFORM
    BASE_PLATFORM --> LOCAL_PLATFORM
    BASE_PLATFORM --> REMOTE_PLATFORM
    REMOTE_PLATFORM --> K8S

    PREFLIGHT --> PARSER
    PREFLIGHT --> KERNEL_LIB
    BENCHMARK --> KERNEL_LIB
    VIS --> KERNEL_LIB

    SUB_MEGATRON -.-> MEGA_BACKEND
    SUB_TITAN -.-> TITAN_BACKEND
    CI -.-> INIT
    DOCKER -.-> DEVICE_RT
    RUNNER_ENV --> RUNNER_SCRIPTS
    ROCM -.-> DEVICE_RT
    NCCL -.-> KERNEL_LIB
    GPU -.-> DEVICE_RT

    %% Click Events
    click CLI1 "https://github.com/amd-agi/primus/blob/main/primus/cli/main.py"
    click CLI2 "https://github.com/amd-agi/primus/blob/main/primus/cli/train_cli.py"
    click PRE "https://github.com/amd-agi/primus/blob/main/primus/pretrain.py"
    click EXAMPLES "https://github.com/amd-agi/primus/tree/main/examples/"
    click RUNNER_SCRIPTS "https://github.com/amd-agi/primus/tree/main/runner/"
    click PARSER "https://github.com/amd-agi/primus/blob/main/primus/core/launcher/parser.py"
    click INIT "https://github.com/amd-agi/primus/blob/main/primus/core/launcher/initialize.py"
    click CFGLOAD "https://github.com/amd-agi/primus/blob/main/primus/core/launcher/config.py"
    click ENV "https://github.com/amd-agi/primus/blob/main/primus/core/utils/env.py"
    click LOG "https://github.com/amd-agi/primus/blob/main/primus/core/utils/logger.py"
    click YAML_UTILS "https://github.com/amd-agi/primus/blob/main/primus/core/utils/yaml_utils.py"
    click CONST_VARS "https://github.com/amd-agi/primus/blob/main/primus/core/utils/constant_vars.py"
    click BASE_MODULE "https://github.com/amd-agi/primus/blob/main/primus/modules/base_module.py"
    click MODULE_UTILS "https://github.com/amd-agi/primus/blob/main/primus/modules/module_utils.py"
    click BASE_TRAINER "https://github.com/amd-agi/primus/blob/main/primus/modules/trainer/base_trainer.py"
    click JAX_TRAINER "https://github.com/amd-agi/primus/tree/main/primus/modules/trainer/jax/"
    click MEGA_TRAINER "https://github.com/amd-agi/primus/blob/main/primus/modules/trainer/megatron/trainer.py"
    click TITAN_PRE "https://github.com/amd-agi/primus/blob/main/primus/modules/trainer/torchtitan/pre_trainer.py"
    click TITAN_UTILS "https://github.com/amd-agi/primus/blob/main/primus/modules/trainer/torchtitan/parse_utils.py"
    click MEGA_BACKEND "https://github.com/amd-agi/primus/tree/main/primus/backends/megatron/core/"
    click JAX_BACKEND "https://github.com/amd-agi/primus/tree/main/primus/backends/jax/"
    click TITAN_BACKEND "https://github.com/amd-agi/primus/tree/main/primus/backends/torchtitan/"
    click TRANS_ENGINE "https://github.com/amd-agi/primus/tree/main/primus/backends/transformer_engine/"
    click BASE_PLATFORM "https://github.com/amd-agi/primus/blob/main/primus/platforms/base_platform.py"
    click LOCAL_PLATFORM "https://github.com/amd-agi/primus/blob/main/primus/platforms/local_platform.py"
    click REMOTE_PLATFORM "https://github.com/amd-agi/primus/blob/main/primus/platforms/remote_platform.py"
    click PREFLIGHT "https://github.com/amd-agi/primus/tree/main/tools/preflight/"
    click BENCHMARK "https://github.com/amd-agi/primus/tree/main/benchmark/"
    click VIS "https://github.com/amd-agi/primus/tree/main/tools/visualization/pp_vis/"
    click SUB_MEGATRON "https://github.com/amd-agi/primus/tree/main/third_party/Megatron-LM"
    click SUB_TITAN "https://github.com/amd-agi/primus/tree/main/third_party/torchtitan"
    click CI "https://github.com/amd-agi/primus/blob/main/.github/workflows/ci.yaml"
    click DOCKER "https://github.com/amd-agi/primus/tree/main/tools/docker/"
    click RUNNER_ENV "https://github.com/amd-agi/primus/tree/main/runner/"

    %% Styles
    classDef frontend fill:#f9f,stroke:#333,stroke-width:1px
    classDef core fill:#bbf,stroke:#333,stroke-width:1px
    classDef module fill:#bfb,stroke:#333,stroke-width:1px
    classDef backend fill:#ffb,stroke:#333,stroke-width:1px
    classDef platform fill:#fbf,stroke:#333,stroke-width:1px
    classDef tool fill:#fbb,stroke:#333,stroke-width:1px
    classDef external fill:#eee,stroke-dasharray: 5 5,stroke:#888,stroke-width:1px
```
