How amd-agi/primus works
Project Type & Purpose
• Primus is not a web app but a full-scale, plugin-based distributed training framework for large foundation models (pretraining, posttraining, RL).
• It exposes a CLI/frontend, core orchestration and utils layers, pluggable “modules” and “backends,” and a suite of tools (benchmark, preflight, visualization).
Main Architectural Components
A. Frontend / Entrypoints
• CLI layer (primus/cli + pretrain.py + runner scripts)
• Example launcher scripts (examples/, runner/)
B. Core Orchestration
• Config parser & validator (primus/core/launcher/parser.py)
• Environment initialization & logging (primus/core/utils)
• YAML/constant utilities
C. Module Layer
• Abstract Module API (primus/modules/base_module.py)
• Trainer implementations per backend (Megatron, TorchTitan, JAX)
D. Backend Layer
• Backend adapters (primus/backends/*)
• Core distributed primitives: tensor + pipeline parallel, checkpointing, collectives, MOE, FP8/fusions
• Low-level kernels invoke Primus-Turbo operators or ROCm libraries
E. Platform Layer
• Platform abstraction (Local vs. Remote) for filesystem, Slurm/K8s integration
F. Tools & Benchmarks
• Preflight sanity checks (tools/preflight)
• Kernel‐ and model‐level benchmarks (benchmark/)
• Visualization of pipeline schedules (tools/visualization)
G. External Dependencies & Infrastructure
• ROCm-enabled Docker images
• HPC interconnect (NCCL/RCCL)
• Third-party submodules (Megatron-LM, TorchTitan)
• Kubernetes/Slurm schedulers
• AMD GPU hardware
Relationships & Data Flows
• User CLI ➔ Core Launcher: reads and validates YAML
• Core Launcher ➔ Module Loader: instantiates correct Trainer module
• Trainer Module ➔ Backend APIs: sets up distributed parallelism (tensor, pipeline, FSDP)
• Backend Core ➔ Operator Kernel Library: calls optimized GEMM, attention, collectives (Primus-Turbo/AITER/hipBLASLt/Triton)
• Backend Core ➔ Device Runtime (ROCm) ➔ GPU(s)
• Platform Layer manages job execution environment (local process vs. Slurm job vs. K8s pod)
• Tools & Benchmarks invoke kernels or full training flows and collect metrics
Key Architectural Patterns
• Layered / clean separation (CLI → Core → Modules → Backends → Hardware)
• Plugin/adapter pattern for “backends” and “modules”
• Configuration-driven wiring (YAML defines which backend, model, precision, parallelism)
• Distributed pipeline + tensor parallel scheduling with zero-bubble optimizations
• Extensibility points: new backends, new modules, new tools