# alpa-projects/alpa architecture

> Architecture diagram of the GitHub repository alpa-projects/alpa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alpa-projects/alpa
- Repository: https://github.com/alpa-projects/alpa

## Overview

##### Main Components
- Core API and Parallelization Layer – This is the “alpa” package itself (e.g., alpa/api.py, alpa/parallel_method.py, alpa/parallel_plan.py). It exposes the decorator (@parallelize) and other high-level interfaces used by the client code.
- Distributed Training Modules – Break these down further:
– Collective Communication Modules: The “alpa/collective” folder (plus subfolders for various collective groups like nccl, gloo, and XLA-specific implementations) represents the components that manage low-level communication across devices.
– Pipeline Parallel Modules: Located in “alpa/pipeline_parallel”, these modules (e.g., layer_construction, schedules, runtime_emitter) set up and run the pipelined execution of model layers.
– Shard Parallel Modules: Found under “alpa/shard_parallel”, these files manage auto and manual sharding of model parameters.
– Device Mesh & Execution: “alpa/device_mesh.py”, “alpa/mesh_executable.py”, and “alpa/mesh_profiling.py” are involved in mapping computations to the physical devices.
- Model Definitions and Utilities – The “alpa/model” folder contains example neural network models (BERT, GPT, Conformer, etc.) and related utilities that can be used in training.
- Serving Components – Under “alpa/serve”, components like the “controller”, “http_util”, and “run” files enable the system’s large-scale model serving capabilities.
- Integration with Frameworks – The “alpa/torch” folder is for integration with PyTorch, enabling support for Torch-based training/inference.
- Auxiliary Components – Other folders (e.g., “benchmark”, “docs”, “examples”, “tests”, “docker”, and “build_jaxlib”) support testing, benchmarking, documentation, containerized deployment, and building external dependencies like jaxlib.

##### Relationships and Interactions
- Client/Front-end Interface: Users write training or serving code that uses the Alpa API. Their code calls functions decorated with @parallelize.
- Parallel Execution Engine: This layer (composed of the parallel plan, collective modules, and device mesh components) takes a single-device computation, performs graph analysis and partitioning, and outputs a distributed execution plan.
- Serving Pipeline: For inference, the serving modules accept requests (e.g., via HTTP as indicated by “http_util” and “controller”) that are routed into the distributed backend for large model inference.
- Integration with Third-Party Libraries: Note the dependencies on JAX/XLA, Ray, and possibly PyTorch. These integrations should be represented as external systems that interact with the core engine.

##### Architectural Patterns and Design Principles
- Modular and Layered Architecture: The source code is organized into clear modules (collective, pipeline_parallel, shard_parallel, etc.) that separately handle distinct responsibilities.
- Separation of Concerns: There is a clear divide between training, serving, and supporting utilities (benchmarking, testing, Docker deployments, and JAXLib building).
- Distributed and Data-Parallel Patterns: The system leverages data, operator, and pipeline parallelism, with dedicated modules to handle sharding and device communication.
- Extensibility: The roadmap and multi-framework integration (JAX, PyTorch) suggest a design that can be extended to accommodate varied backend requirements.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Layer"
        CC["Client Code"]:::client
    end

    subgraph "Core API Layer"
        API["Alpa API (@parallelize)"]:::core
        PM["Parallelization Methods"]:::core
        PP["Parallel Plan Engine"]:::core
    end

    CC -->|"calls"| API
    API -->|"invokes"| PM
    API -->|"invokes"| PP

    subgraph "Parallel Computation Engine"
        COL["Collective Communication (NCCL/Gloo/XLA)"]:::engine
        PIPE["Pipeline Parallel Engine"]:::engine
        SHARD["Shard Parallel Engine"]:::engine
    end

    PP -->|"distributes to"| COL
    PP -->|"distributes to"| PIPE
    PP -->|"distributes to"| SHARD

    subgraph "Device Mesh & Execution"
        DEV_MESH["Device Mesh Manager"]:::execution
        MESH_EXEC["Mesh Executable"]:::execution
        MESH_PROFILE["Mesh Profiling"]:::execution
    end

    COL -->|"routes to"| DEV_MESH
    PIPE -->|"routes to"| DEV_MESH
    SHARD -->|"routes to"| DEV_MESH

    subgraph "Model and Serving"
        MODELS["Model Repository & Utilities"]:::model
        SERVE["Serving Components (Controller, HTTP Utils, Run)"]:::serve
    end

    DEV_MESH -->|"feeds into"| MODELS
    DEV_MESH -->|"feeds into"| SERVE

    subgraph "Integrations & External"
        PYTORCH["PyTorch Integration"]:::integration
        JAX["JAX/XLA"]:::external
        RAY["Ray"]:::external
    end

    API -->|"integrates with"| PYTORCH
    DEV_MESH -->|"executes with"| JAX
    DEV_MESH -->|"executes with"| RAY
    PYTORCH -->|"leverages"| JAX

    subgraph "Auxiliary"
        TEST["Tests/Benchmark"]:::aux
        DEPLOY["Deployment (Docker/Build Tools)"]:::aux
        DOCS["Docs/Examples"]:::aux
    end

    SERVE -->|"supported by"| TEST
    SERVE -->|"supported by"| DEPLOY
    SERVE -->|"supported by"| DOCS

    %% Click Events
    click API "https://github.com/alpa-projects/alpa/blob/main/alpa/api.py"
    click PM "https://github.com/alpa-projects/alpa/blob/main/alpa/parallel_method.py"
    click PP "https://github.com/alpa-projects/alpa/blob/main/alpa/parallel_plan.py"
    click COL "https://github.com/alpa-projects/alpa/tree/main/alpa/collective"
    click PIPE "https://github.com/alpa-projects/alpa/tree/main/alpa/pipeline_parallel"
    click SHARD "https://github.com/alpa-projects/alpa/tree/main/alpa/shard_parallel"
    click DEV_MESH "https://github.com/alpa-projects/alpa/blob/main/alpa/device_mesh.py"
    click MESH_EXEC "https://github.com/alpa-projects/alpa/blob/main/alpa/mesh_executable.py"
    click MESH_PROFILE "https://github.com/alpa-projects/alpa/blob/main/alpa/mesh_profiling.py"
    click MODELS "https://github.com/alpa-projects/alpa/tree/main/alpa/model"
    click SERVE "https://github.com/alpa-projects/alpa/tree/main/alpa/serve"
    click PYTORCH "https://github.com/alpa-projects/alpa/tree/main/alpa/torch"

    %% Styles
    classDef client fill:#FFEDB3,stroke:#B8860B,stroke-width:2px;
    classDef core fill:#FFD700,stroke:#B8860B,stroke-width:2px;
    classDef engine fill:#C0C0C0,stroke:#808080,stroke-width:2px;
    classDef execution fill:#ADD8E6,stroke:#00008B,stroke-width:2px;
    classDef model fill:#90EE90,stroke:#006400,stroke-width:2px;
    classDef serve fill:#FFB6C1,stroke:#FF69B4,stroke-width:2px;
    classDef integration fill:#E6E6FA,stroke:#8A2BE2,stroke-width:2px;
    classDef external fill:#FFEFD5,stroke:#FFA500,stroke-width:2px;
    classDef aux fill:#D3D3D3,stroke:#696969,stroke-width:2px;
```
