# xdit-project/xdit architecture

> Architecture diagram of the GitHub repository xdit-project/xdit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xdit-project/xdit
- Repository: https://github.com/xdit-project/xdit

## Overview

##### Project Type and Purpose
- xDiT is an open‐source Python library and inference engine (not a web application) for running large Diffusion Transformer (DiT) models in parallel across multiple GPUs or machines.
- It provides:
- A core Python API (“xfuser” package) to wrap, configure, and execute DiT models using various parallel strategies (USP, PipeFusion, hybrid, etc.).
- Command‐line entrypoints and example scripts to launch jobs in single‐ or multi‐GPU settings (via torchrun or Ray).
- An optional HTTP service for real‐time model serving.
- Benchmarks, developer guides, and Docker support for reproducible experiments.

##### Key Directories and Their Roles
- entrypoints/
- CLI launch scripts (launch.py, curl.sh)
- HTTP service integration
- xfuser/ (core library)
- config/args.py, config.py – parses CLI and programmatic settings
- core/ – distributed state management, group coordination, cache manager
- model_executor/ – model‐wrappers, pipelines, schedulers, layers, and cache adapters for HuggingFace Diffusers
- fast_attention/, long_ctx_attention/ – custom attention kernels and utilities
- envs.py, logger.py – environment detection and logging
- parallel.py, ray/ – multi‐GPU orchestration via torch.distributed or Ray
- examples/ & benchmark/
- Usage examples for various models and parallel setups
- Benchmark scripts for latency and FID measurements
- docs/ & README.md
- Architecture explanations, developer guides, and performance reports

##### High‐Level System Components
A. Client Layer
- CLI (examples/*.py, run.sh via torchrun)
- HTTP Service (entrypoints/Http_Service.md → launch.py + FastAPI or similar)
- Ray submission scripts (examples/ray/)
B. Configuration & Initialization
- config/args.py, config/config.py → parse CLI flags or HTTP payload
- envs.py → detect device topology
- logger.py → centralized logging
C. Core Orchestration
- Distributed Manager (xfuser/core/distributed)
- Parallel Controller (xfuser/parallel.py)
- Ray Worker (xfuser/ray/worker) if using Ray
D. Model Execution Stack
1. Pipelines (xfuser/model_executor/pipelines)
2. Scheduler (timesteps via xfuser/model_executor/schedulers)
3. Model Wrappers (xfuser/model_executor/models + patches)
4. Layers & Attention (xfuser/model_executor/layers + fast/long_ctx_attention)
5. Cache Manager (xfuser/core/cache_manager or model_executor/cache)
E. External Libraries
- PyTorch (torch)
- Optional Diffusers (huggingface)
- FlashAttention, onediff, torch.compile
F. Hardware & Deployment
- GPU nodes (via torch.distributed or Ray)
- Docker container (docker/Dockerfile)

##### Relationships and Data Flow
- Clients → Configuration Loader → Parallel Manager → Pipeline Executor → Scheduler → Model → Attention/Cache → GPU Devices
- Pipelines trigger schedulers to compute diffusion timesteps, then dispatch through model layers.
- The distributed manager orchestrates all‐reduce or ring communications for USP, PipeFusion, etc.

##### Architectural Patterns and Principles
- Modular Pipeline Pattern: Separate pipeline, scheduler, model, layers, and cache into swappable modules.
- Plugin Registry: central register.py files allow dynamic discovery of pipelines, models, and schedulers.
- Wrapper/Adapter: “base_wrapper.py” adapts raw nn.Modules (HuggingFace or custom) to xDiT’s execution engine.
- Hybrid Parallelism: composition of sequence‐parallel, pipeline‐parallel, data‐parallel strategies.
- Dependency Inversion: high‐level orchestration depends on abstract interfaces (e.g., BasePipeline, BaseScheduler).

##### Essential Elements to Capture
- Clear component names reflecting directory structure

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client Layer"
        LAUNCH["entrypoints/launch.py"]:::frontend
        CURL["entrypoints/curl.sh"]:::frontend
        RUNSH["examples/run.sh"]:::frontend
        HTTPDOC["docs/developer/Http_Service.md"]:::frontend
        RAYEX["examples/ray/"]:::frontend
    end

    %% Configuration & Initialization
    subgraph "Configuration & Initialization"
        ARGS["xfuser/config/args.py"]:::config
        CFG["xfuser/config/config.py"]:::config
        ENVS["xfuser/envs.py"]:::config
        LOG["xfuser/logger.py"]:::config
    end

    %% Core Orchestration
    subgraph "Core Orchestration"
        DIST["xfuser/core/distributed/"]:::orchestration
        PAR["xfuser/parallel.py"]:::orchestration
        RAYWORKER["xfuser/ray/worker/"]:::orchestration
        RAYPIPE["xfuser/ray/pipeline/"]:::orchestration
    end

    %% Model Execution Stack
    subgraph "Model Execution Stack"
        PIPE["xfuser/model_executor/pipelines/"]:::execution
        SCHED["xfuser/model_executor/schedulers/"]:::execution
        BASEWRAP["xfuser/model_executor/base_wrapper.py"]:::execution
        MODELS["xfuser/model_executor/models/"]:::execution
        LAYERS["xfuser/model_executor/layers/"]:::execution
        FASTATTN["xfuser/core/fast_attention/"]:::execution
        LONGATTN["xfuser/core/long_ctx_attention/"]:::execution
        CACHE1["xfuser/core/cache_manager/cache_manager.py"]:::execution
        CACHE2["xfuser/model_executor/cache/"]:::execution
    end

    %% External Libraries
    subgraph "External Libraries"
        TORCH["PyTorch"]:::external
        DIFF["Diffusers"]:::external
        FLASH["FlashAttention"]:::external
        ONEDIFF["onediff"]:::external
        COMPILE["torch.compile"]:::external
    end

    %% Hardware & Deployment
    subgraph "Hardware & Deployment"
        GPU["GPU Nodes"]:::hardware
        DOCKER["Docker Container"]:::hardware
    end

    %% Connections
    LAUNCH -->|parse args| ARGS
    CURL -->|parse args| ARGS
    RUNSH -->|parse args| ARGS
    HTTPDOC -->|load docs| ARGS
    RAYEX -->|submit job| ARGS

    ARGS --> CFG
    ARGS --> ENVS
    ARGS --> LOG

    CFG --> PAR
    ENVS --> PAR
    LOG --> PAR

    PAR --> PIPE
    PAR --> DIST
    DIST --> PAR

    PAR --> RAYWORKER
    PAR --> RAYPIPE

    PIPE --> SCHED
    SCHED --> BASEWRAP
    BASEWRAP --> MODELS
    MODELS --> LAYERS
    LAYERS --> GPU
    FASTATTN --> LAYERS
    LONGATTN --> LAYERS
    CACHE1 --> GPU
    CACHE2 --> GPU
    BASEWRAP --> CACHE2

    TORCH --> BASEWRAP
    DIFF -.-> PIPE
    FLASH -.-> LAYERS
    ONEDIFF -.-> LAYERS
    COMPILE -.-> BASEWRAP

    GPU --> DOCKER

    %% Click Events
    click LAUNCH "https://github.com/xdit-project/xdit/blob/main/entrypoints/launch.py"
    click CURL "https://github.com/xdit-project/xdit/blob/main/entrypoints/curl.sh"
    click RUNSH "https://github.com/xdit-project/xdit/blob/main/examples/run.sh"
    click HTTPDOC "https://github.com/xdit-project/xdit/blob/main/docs/developer/Http_Service.md"
    click RAYEX "https://github.com/xdit-project/xdit/tree/main/examples/ray/"
    click ARGS "https://github.com/xdit-project/xdit/blob/main/xfuser/config/args.py"
    click CFG "https://github.com/xdit-project/xdit/blob/main/xfuser/config/config.py"
    click ENVS "https://github.com/xdit-project/xdit/blob/main/xfuser/envs.py"
    click LOG "https://github.com/xdit-project/xdit/blob/main/xfuser/logger.py"
    click DIST "https://github.com/xdit-project/xdit/tree/main/xfuser/core/distributed/"
    click PAR "https://github.com/xdit-project/xdit/blob/main/xfuser/parallel.py"
    click RAYWORKER "https://github.com/xdit-project/xdit/tree/main/xfuser/ray/worker/"
    click RAYPIPE "https://github.com/xdit-project/xdit/tree/main/xfuser/ray/pipeline/"
    click PIPE "https://github.com/xdit-project/xdit/tree/main/xfuser/model_executor/pipelines/"
    click SCHED "https://github.com/xdit-project/xdit/tree/main/xfuser/model_executor/schedulers/"
    click BASEWRAP "https://github.com/xdit-project/xdit/blob/main/xfuser/model_executor/base_wrapper.py"
    click MODELS "https://github.com/xdit-project/xdit/tree/main/xfuser/model_executor/models/"
    click LAYERS "https://github.com/xdit-project/xdit/tree/main/xfuser/model_executor/layers/"
    click FASTATTN "https://github.com/xdit-project/xdit/tree/main/xfuser/core/fast_attention/"
    click LONGATTN "https://github.com/xdit-project/xdit/tree/main/xfuser/core/long_ctx_attention/"
    click CACHE1 "https://github.com/xdit-project/xdit/blob/main/xfuser/core/cache_manager/cache_manager.py"
    click CACHE2 "https://github.com/xdit-project/xdit/tree/main/xfuser/model_executor/cache/"
    click DOCKER "https://github.com/xdit-project/xdit/tree/main/docker/Dockerfile"

    %% Styles
    classDef frontend fill:#D0E6FF,stroke:#0366d6,color:#0366d6,stroke-width:1px;
    classDef config fill:#DFFFD0,stroke:#28a745,color:#28a745,stroke-width:1px;
    classDef orchestration fill:#DFFFE0,stroke:#22863a,color:#22863a,stroke-width:1px;
    classDef execution fill:#FFF5D0,stroke:#d97706,color:#d97706,stroke-width:1px;
    classDef external fill:#E2E2E2,stroke:#6a737d,color:#6a737d,stroke-width:1px,stroke-dasharray: 5 5;
    classDef hardware fill:#F0E6FF,stroke:#6f42c1,color:#6f42c1,stroke-width:1px;
```
