# microsoft/vidur architecture

> Architecture diagram of the GitHub repository microsoft/vidur, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/vidur
- Repository: https://github.com/microsoft/vidur

## Overview

##### Main Components Identification

a) Core Simulation Components:
- Request Generator System
- Scheduler System (Global, Replica, and Stage schedulers)
- Execution Time Predictor
- Event Management System
- Metrics Collection System

b) Configuration Components:
- Model Configurations
- Device SKU Configurations
- Node SKU Configurations
- Simulation Parameters

##### Component Relationships

- Blue: Core simulation components
- Green: Input/configuration components
- Orange: Scheduling components
- Purple: Monitoring and metrics
- Gray: External integrations

##### Specific Implementation Details

- Supported model implementations (Llama, CodeLlama, etc.)
- Hardware configurations (A100, H100, multi-GPU setups)
- Scheduler types (SARATHI, VLLM, etc.)
- Profiling capabilities (attention, MLP, network)

##### Key Features to Highlight

- Pipeline parallelism support
- Tensor parallelism configurations
- Maximum context length handling
- Multi-GPU support
- Trace replay capabilities

##### Integration Points

External integration points for:
- Wandb logging
- Chrome tracing
- Profiling tools
- Configuration management

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        Config["Configuration System"]:::config
        click Config "https://github.com/microsoft/vidur/tree/main/vidur/config/"
        ModelConfig["Model Configs"]:::config
        HWConfig["Hardware SKUs"]:::config
        WorkloadParams["Workload Parameters"]:::config
        
        Profiling["Profiling System"]:::profiling
        click Profiling "https://github.com/microsoft/vidur/tree/main/vidur/profiling/"
        CompProfiles["Compute Profiles"]:::profiling
        click CompProfiles "https://github.com/microsoft/vidur/tree/main/data/profiling/compute/"
        NetProfiles["Network Profiles"]:::profiling
        click NetProfiles "https://github.com/microsoft/vidur/tree/main/data/profiling/network/"
    end

    subgraph Core
        ReqGen["Request Generator"]:::core
        click ReqGen "https://github.com/microsoft/vidur/tree/main/vidur/request_generator/"
        
        EventSystem["Event Management System"]:::core
        click EventSystem "https://github.com/microsoft/vidur/tree/main/vidur/events/"
        EventQueue["Event Queue"]:::core
        click EventQueue "https://github.com/microsoft/vidur/blob/main/vidur/utils/event_queue.py"
        
        subgraph Scheduler["Scheduler System"]
            GlobalSched["Global Scheduler"]:::scheduler
            click GlobalSched "https://github.com/microsoft/vidur/tree/main/vidur/scheduler/global_scheduler/"
            ReplicaSched["Replica Scheduler"]:::scheduler
            click ReplicaSched "https://github.com/microsoft/vidur/tree/main/vidur/scheduler/replica_scheduler/"
            StageSched["Stage Scheduler"]:::scheduler
            click StageSched "https://github.com/microsoft/vidur/tree/main/vidur/scheduler/replica_stage_scheduler/"
        end
        
        ExecPredictor["Execution Time Predictor"]:::core
        click ExecPredictor "https://github.com/microsoft/vidur/tree/main/vidur/execution_time_predictor/"
        
        SimCore["Simulator Core"]:::core
        click SimCore "https://github.com/microsoft/vidur/blob/main/vidur/simulator.py"
        
        Entities["Entity Definitions"]:::core
        click Entities "https://github.com/microsoft/vidur/tree/main/vidur/entities/"
        
        Types["Type Definitions"]:::core
        click Types "https://github.com/microsoft/vidur/tree/main/vidur/types/"
    end

    subgraph Output
        Metrics["Metrics Collection"]:::output
        click Metrics "https://github.com/microsoft/vidur/tree/main/vidur/metrics/"
        ChromeTraces["Chrome Traces"]:::output
        WandB["WandB Integration"]:::output
        Analytics["Performance Analytics"]:::output
    end

    %% Input Relationships
    Config --> ModelConfig
    Config --> HWConfig
    Config --> WorkloadParams
    Profiling --> CompProfiles
    Profiling --> NetProfiles

    %% Core Flow
    ModelConfig & HWConfig & WorkloadParams --> ReqGen
    CompProfiles & NetProfiles --> ExecPredictor
    
    ReqGen --> EventQueue
    EventQueue --> EventSystem
    EventSystem --> GlobalSched
    GlobalSched --> ReplicaSched
    ReplicaSched --> StageSched
    StageSched --> ExecPredictor
    ExecPredictor --> SimCore
    
    Entities --> SimCore
    Types --> SimCore

    %% Output Flow
    SimCore --> Metrics
    Metrics --> ChromeTraces
    Metrics --> WandB
    Metrics --> Analytics

    %% Styling
    classDef config fill:#90EE90,stroke:#333,stroke-width:2px
    classDef profiling fill:#FFB6C1,stroke:#333,stroke-width:2px
    classDef core fill:#87CEEB,stroke:#333,stroke-width:2px
    classDef scheduler fill:#FFA500,stroke:#333,stroke-width:2px
    classDef output fill:#DDA0DD,stroke:#333,stroke-width:2px

    %% Legend
    subgraph Legend
        L1["Configuration"]:::config
        L2["Profiling"]:::profiling
        L3["Core Components"]:::core
        L4["Scheduler"]:::scheduler
        L5["Output"]:::output
    end
```
