How xdit-project/xdit works
Project Type and Purpose
• xDiT is an open‐source Python library and inference engine (not a web application) for running large Diffusion Transformer (DiT) models in parallel across multiple GPUs or machines.
• It provides:
• A core Python API (“xfuser” package) to wrap, configure, and execute DiT models using various parallel strategies (USP, PipeFusion, hybrid, etc.).
• Command‐line entrypoints and example scripts to launch jobs in single‐ or multi‐GPU settings (via torchrun or Ray).
• An optional HTTP service for real‐time model serving.
• Benchmarks, developer guides, and Docker support for reproducible experiments.
Key Directories and Their Roles
• entrypoints/
• CLI launch scripts (launch.py, curl.sh)
• HTTP service integration
• xfuser/ (core library)
• config/args.py, config.py – parses CLI and programmatic settings
• core/ – distributed state management, group coordination, cache manager
• model_executor/ – model‐wrappers, pipelines, schedulers, layers, and cache adapters for HuggingFace Diffusers
• fast_attention/, long_ctx_attention/ – custom attention kernels and utilities
• envs.py, logger.py – environment detection and logging
• parallel.py, ray/ – multi‐GPU orchestration via torch.distributed or Ray
• examples/ & benchmark/
• Usage examples for various models and parallel setups
• Benchmark scripts for latency and FID measurements
• docs/ & README.md
• Architecture explanations, developer guides, and performance reports
High‐Level System Components
A. Client Layer
• CLI (examples/*.py, run.sh via torchrun)
• HTTP Service (entrypoints/Http_Service.md → launch.py + FastAPI or similar)
• Ray submission scripts (examples/ray/)
B. Configuration & Initialization
• config/args.py, config/config.py → parse CLI flags or HTTP payload
• envs.py → detect device topology
• logger.py → centralized logging
C. Core Orchestration
• Distributed Manager (xfuser/core/distributed)
• Parallel Controller (xfuser/parallel.py)
• Ray Worker (xfuser/ray/worker) if using Ray
D. Model Execution Stack
1. Pipelines (xfuser/model_executor/pipelines)
2. Scheduler (timesteps via xfuser/model_executor/schedulers)
3. Model Wrappers (xfuser/model_executor/models + patches)
4. Layers & Attention (xfuser/model_executor/layers + fast/long_ctx_attention)
5. Cache Manager (xfuser/core/cache_manager or model_executor/cache)
E. External Libraries
• PyTorch (torch)
• Optional Diffusers (huggingface)
• FlashAttention, onediff, torch.compile
F. Hardware & Deployment
• GPU nodes (via torch.distributed or Ray)
• Docker container (docker/Dockerfile)
Relationships and Data Flow
• Clients → Configuration Loader → Parallel Manager → Pipeline Executor → Scheduler → Model → Attention/Cache → GPU Devices
• Pipelines trigger schedulers to compute diffusion timesteps, then dispatch through model layers.
• The distributed manager orchestrates all‐reduce or ring communications for USP, PipeFusion, etc.
Architectural Patterns and Principles
• Modular Pipeline Pattern: Separate pipeline, scheduler, model, layers, and cache into swappable modules.
• Plugin Registry: central register.py files allow dynamic discovery of pipelines, models, and schedulers.
• Wrapper/Adapter: “base_wrapper.py” adapts raw nn.Modules (HuggingFace or custom) to xDiT’s execution engine.
• Hybrid Parallelism: composition of sequence‐parallel, pipeline‐parallel, data‐parallel strategies.
• Dependency Inversion: high‐level orchestration depends on abstract interfaces (e.g., BasePipeline, BaseScheduler).
Essential Elements to Capture
• Clear component names reflecting directory structure