# radixark/miles architecture

> Architecture diagram of the GitHub repository radixark/miles, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/radixark/miles
- Repository: https://github.com/radixark/miles

## Overview

1. **Top-level shape**
   - Miles is a Python-first RL/post-training framework with two main runtime planes: **training** and **rollout/inference**.
   - The core packages are `miles/` and `miles_plugins/`; the rest of the repo is orchestration (`scripts/`, `examples/`), packaging (`docker/`, `tools/`), and validation (`tests/`, CI).

2. **Primary entrypoints**
   - `train.py` and `train_async.py` are the unified launch points for RL/SFT-style runs.
   - They feed into shared argument parsing and utility layers under `miles/utils/`, then dispatch to the selected backend and rollout mode based on model, precision, and distributed topology.

3. **Training backend boundary**
   - `miles/backends/training_utils/` contains the training-side logic: data shaping, loss functions, advantage/reward handling, replay data, metrics, and distributed helpers.
   - `miles/backends/megatron_utils/` and `miles/backends/experimental/fsdp_utils/` are the two main training implementations, with Megatron being the more feature-rich path for large-scale and low-precision runs.

4. **Rollout / inference boundary**
   - `miles/rollout/` owns the generation side: single-turn, multi-turn, tool-call, SFT-style rollout, sleep/idle rollout, and inference-rollout compatibility layers.
   - This package is where prompt construction, sampling, logprob handling, filtering, and trajectory/session management happen before data is handed back to training.

5. **SGLang integration**
   - `miles/backends/sglang_utils/` and `miles/rollout/sglang_rollout.py` bridge the framework to SGLang for high-throughput serving and generation.
   - Rollout data flows through SGLang-specific config/engine layers, then returns tokens, logprobs, and routing metadata needed by the training side.

6. **Distributed orchestration**
   - `miles/ray/` is the distributed control plane for rollout and training workers.
   - It manages actor groups, placement, rollout servers, router management, address allocation, and server cells/groups, making Ray the coordination layer for multi-node execution.

7. **Routing, replay, and mismatch handling**
   - `miles/router/` and parts of `miles/rollout/` support routing-aware execution, including the “R3” style replay path described in the docs.
   - This is a key architectural boundary: inference records routing decisions, and training can replay or align them to reduce train-inference mismatch, especially for MoE models.

8. **Model- and kernel-specific extensions**
   - `miles_plugins/` contains model backends and custom kernels: DeepSeek, GLM, Qwen, Qwen-VL, MoE/indexer kernels, sparse MLA ops, and bridge logic.
   - `miles/backends/megatron_utils/megatron_to_hf/` and related update-weight code handle checkpoint conversion and parameter mapping between Megatron-style distributed weights and Hugging Face-style model layouts.

9. **True on-policy and task-specific schemas**
   - `miles/true_on_policy/` defines contracts, config, schemas, and model profiles for the “true on-policy” path.
   - This package is a narrower abstraction layer used when the training loop needs stricter rollout/training consistency and explicit data contracts.

10. **Utilities that support the whole stack**
   - `miles/utils/` is a broad shared layer for async/process control, distributed helpers, chat templates/tokenization, logging, metrics, health monitoring, replay, and debugging.
   - Several submodules are architecture-critical: chat-template utilities affect prompt/token parity, distributed utilities affect multi-node execution, and tracking utilities connect runs to TensorBoard, W&B, MLflow, or Prometheus.

11. **Examples and recipe layer**
   - `examples/` is not core runtime code; it is a catalog of runnable recipes for RL, SFT, LoRA, VLM, multi-agent, speculative decoding, search/tool use, and benchmark-specific setups.
   - These examples show the intended composition pattern: data prep → rollout → reward/filtering → training → checkpoint/output conversion.

12. **Model deployment and conversion tools**
   - `tools/` and many scripts under `scripts/` provide checkpoint conversion, precision conversion, and run-specific launch wrappers.
   - These tools are important because the framework spans multiple checkpoint formats and precisions, so the architecture includes explicit conversion boundaries between HF, Megatron, distributed checkpoints, INT4/FP8/MXFP8/NVFP4 variants, and runtime-friendly artifacts.

13. **Packaging, containerization, and CI**
   - `docker/` contains the build images and backend patches for GPU/vendor-specific support, including NVIDIA, ROCm, and NPU-related patch sets.
   - `.github/workflows/`, `tests/`, and `tests/ci/` enforce that the distributed orchestration, kernels, model bridges, and rollout/training parity remain stable across supported environments.

14. **End-to-end data flow to diagram**
   - A typical run starts in `train.py` or a recipe script, loads model/runtime config, launches rollout workers and training workers, generates trajectories through SGLang/Ray, applies reward/filter/loss logic, updates the model via Megatron/FSDP, then writes checkpoints and optionally converts them for inference or downstream serving.
   - The most important diagram boundaries are: **user scripts**, **rollout/inference plane**, **distributed orchestration plane**, **training plane**, and **model/plugin/kernel extensions**.

Main entry points: [`miles/rollout`](https://github.com/radixark/miles/tree/main/miles/rollout) (Rollout), [`miles/backends/training_utils`](https://github.com/radixark/miles/tree/main/miles/backends/training_utils) (Training), [`miles/ray`](https://github.com/radixark/miles/tree/main/miles/ray) (Ray), [`train.py`](https://github.com/radixark/miles/blob/main/train.py) (train.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entrypoints"]
  node_train_py["train.py<br/>launcher<br/>[train.py]"]
  node_train_async_py["train async<br/>launcher<br/>[train_async.py]"]
  node_scripts["Scripts"]
end

subgraph group_core["Core runtime"]
  node_utils["Utils<br/>shared utilities"]
  node_rollout["Rollout<br/>generation plane"]
  node_training_utils["Training"]
  node_megatron_utils["Megatron<br/>training backend"]
  node_fsdp_utils["FSDP<br/>training backend"]
  node_sglang_utils["SGLang<br/>serving bridge"]
  node_ray["Ray<br/>distributed control plane"]
  node_router["Router<br/>routing layer"]
  node_true_on_policy["On-policy<br/>schema/contracts"]
end

subgraph group_ext["Extensions"]
  node_miles_plugins["Plugins<br/>model extensions"]
end

subgraph group_ops["Ops & delivery"]
  node_tools["Tools"]
  node_examples["Examples<br/>recipes"]
  node_docker["Docker<br/>build images"]
  node_tests["Tests<br/>validation"]
  node_docs["Docs<br/>documentation"]
end

node_train_py -->|"parse/configure"| node_utils
node_train_async_py -->|"parse/configure"| node_utils
node_train_py -->|"dispatch"| node_rollout
node_train_async_py -->|"dispatch"| node_rollout
node_train_py -->|"train loop"| node_training_utils
node_train_async_py -->|"train loop"| node_training_utils
node_training_utils -->|"backend"| node_megatron_utils
node_training_utils -->|"backend"| node_fsdp_utils
node_rollout -->|"serve/generate"| node_sglang_utils
node_rollout -->|"coordinate"| node_ray
node_ray -->|"data plane"| node_rollout
node_ray -->|"workers"| node_training_utils
node_rollout -->|"route-aware"| node_router
node_router -->|"orchestrate"| node_ray
node_true_on_policy -->|"contracts"| node_training_utils
node_miles_plugins -->|"model kernels"| node_megatron_utils
node_miles_plugins -->|"model bridges"| node_sglang_utils
node_tools -->|"convert"| node_megatron_utils
node_examples -->|"invoke"| node_train_py
node_examples -->|"compose"| node_rollout
node_docker -->|"patches"| node_miles_plugins
node_tests -->|"validate"| node_ray
node_tests -->|"validate"| node_rollout
node_tests -->|"validate"| node_training_utils
node_docs -->|"describe"| node_examples
node_docs -->|"describe"| node_router

click node_train_py "https://github.com/radixark/miles/blob/main/train.py"
click node_train_async_py "https://github.com/radixark/miles/blob/main/train_async.py"
click node_scripts "https://github.com/radixark/miles/tree/main/scripts"
click node_utils "https://github.com/radixark/miles/tree/main/miles/utils"
click node_rollout "https://github.com/radixark/miles/tree/main/miles/rollout"
click node_training_utils "https://github.com/radixark/miles/tree/main/miles/backends/training_utils"
click node_megatron_utils "https://github.com/radixark/miles/tree/main/miles/backends/megatron_utils"
click node_fsdp_utils "https://github.com/radixark/miles/tree/main/miles/backends/experimental/fsdp_utils"
click node_sglang_utils "https://github.com/radixark/miles/tree/main/miles/backends/sglang_utils"
click node_ray "https://github.com/radixark/miles/tree/main/miles/ray"
click node_router "https://github.com/radixark/miles/tree/main/miles/router"
click node_true_on_policy "https://github.com/radixark/miles/tree/main/miles/true_on_policy"
click node_miles_plugins "https://github.com/radixark/miles/tree/main/miles_plugins"
click node_tools "https://github.com/radixark/miles/tree/main/tools"
click node_examples "https://github.com/radixark/miles/tree/main/examples"
click node_docker "https://github.com/radixark/miles/tree/main/docker"
click node_tests "https://github.com/radixark/miles/tree/main/tests"
click node_docs "https://github.com/radixark/miles/tree/main/docs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_py,node_train_async_py,node_scripts toneBlue
class node_utils,node_rollout,node_training_utils,node_megatron_utils,node_fsdp_utils,node_sglang_utils,node_ray,node_router,node_true_on_policy toneAmber
class node_miles_plugins toneMint
class node_tools,node_examples,node_docker,node_tests,node_docs toneRose
```

## Components

### Entrypoints

Unified launch scripts and recipe wrappers

- [train.py](https://github.com/radixark/miles/blob/main/train.py): launcher · Primary synchronous training entrypoint · `train.py`
- [train async](https://github.com/radixark/miles/blob/main/train_async.py): launcher · Asynchronous training entrypoint · `train_async.py`
- [Scripts](https://github.com/radixark/miles/tree/main/scripts): run scripts · Model- and topology-specific launch wrappers · `scripts`

### Core runtime

Shared framework packages for rollout, training, routing, and orchestration

- [Utils](https://github.com/radixark/miles/tree/main/miles/utils): shared utilities · Argument parsing, async/process control, logging, metrics, tokenization, and shared helpers · `miles/utils`
- [Rollout](https://github.com/radixark/miles/tree/main/miles/rollout): generation plane · Prompting, sampling, sessions, filters, and rollout modes · `miles/rollout`
- [Training](https://github.com/radixark/miles/tree/main/miles/backends/training_utils): training backend · Losses, data shaping, replay, metrics, and distributed training helpers · `miles/backends/training_utils`
- [Megatron](https://github.com/radixark/miles/tree/main/miles/backends/megatron_utils): training backend · Feature-rich distributed training path and checkpoint/weight handling · `miles/backends/megatron_utils`
- [FSDP](https://github.com/radixark/miles/tree/main/miles/backends/experimental/fsdp_utils): training backend · Experimental FSDP training path · `miles/backends/experimental/fsdp_utils`
- [SGLang](https://github.com/radixark/miles/tree/main/miles/backends/sglang_utils): serving bridge · Configuration and engine bridge for high-throughput generation · `miles/backends/sglang_utils`
- [Ray](https://github.com/radixark/miles/tree/main/miles/ray): distributed control plane · Actors, placement, rollout servers, router management, and worker orchestration · `miles/ray`
- [Router](https://github.com/radixark/miles/tree/main/miles/router): routing layer · Routing-aware execution and replay alignment boundary · `miles/router`
- [On-policy](https://github.com/radixark/miles/tree/main/miles/true_on_policy): schema/contracts · Contracts, configs, and schemas for strict rollout-training consistency · `miles/true_on_policy`

### Extensions

Model-specific bridges, kernels, and plugins

- [Plugins](https://github.com/radixark/miles/tree/main/miles_plugins): model extensions · Model bridges, custom kernels, and backend-specific extensions · `miles_plugins`

### Ops & delivery

Packaging, containers, examples, tools, and validation

- [Tools](https://github.com/radixark/miles/tree/main/tools): conversion tools · Checkpoint and precision conversion utilities · `tools`
- [Examples](https://github.com/radixark/miles/tree/main/examples): recipes · Runnable training, rollout, and evaluation recipes · `examples`
- [Docker](https://github.com/radixark/miles/tree/main/docker): build images · Container builds and vendor-specific patch sets · `docker`
- [Tests](https://github.com/radixark/miles/tree/main/tests): validation · Unit, integration, and end-to-end validation suites · `tests`
- [Docs](https://github.com/radixark/miles/tree/main/docs): documentation · Architecture, user guide, examples, and model/platform docs · `docs`

## Connections

- train.py → Utils: parse/configure · Loads shared args, runtime config, and utilities
- train async → Utils: parse/configure · Uses shared async and argument utilities
- train.py → Rollout: dispatch · Selects rollout mode and generation path
- train async → Rollout: dispatch · Runs async rollout/inference flow
- train.py → Training: train loop · Feeds trajectories into losses and optimization helpers
- train async → Training: train loop · Uses the same training-side data and loss logic
- Training → Megatron: backend · Megatron is the main large-scale implementation
- Training → FSDP: backend · Experimental FSDP implementation path
- Rollout → SGLang: serve/generate · Uses SGLang for high-throughput token generation
- Rollout → Ray: coordinate · Distributed rollout workers and servers are managed via Ray
- Ray → Rollout: data plane · Routes rollout requests through servers and sessions
- Ray → Training: workers · Coordinates training-side distributed actors and placement
- Rollout → Router: route-aware · Supports replay and mismatch-aware routing flows
- Router → Ray: orchestrate · Routing decisions are carried through distributed worker control
- On-policy → Training: contracts · Provides stricter data and model contracts for aligned training
- Plugins → Megatron: model kernels · Extends training with model-specific bridges and custom ops
- Plugins → SGLang: model bridges · Adapts model backends for serving/generation
- Tools → Megatron: convert · Supports checkpoint and parameter mapping workflows
- Examples → train.py: invoke · Recipe scripts launch the main training entrypoint
- Examples → Rollout: compose · Examples demonstrate rollout and evaluation patterns
- Docker → Plugins: patches · Container builds carry vendor and kernel-specific patches
- Tests → Ray: validate · Covers distributed orchestration and rollout server behavior
- Tests → Rollout: validate · Covers generation, sessions, filters, and inference rollout
- Tests → Training: validate · Covers loss, replay, and training-side numerics
- Docs → Examples: describe · Documents the intended recipes and workflows
- Docs → Router: describe · Documents routing and mismatch mitigation concepts
