# oumi-ai/oumi architecture

> Architecture diagram of the GitHub repository oumi-ai/oumi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oumi-ai/oumi
- Repository: https://github.com/oumi-ai/oumi

## Overview

##### Main Components
The system architecture should be divided into these key components:

A. Core Platform Components:
- Training System
- Evaluation System
- Inference System
- Judge System
- Launch/Deployment System

B. Supporting Infrastructure:
- Configuration Management
- Data Pipeline
- Model Registry
- Distributed Computing
- Cloud Integration

##### Component Relationships

**┌─────────────────────────────────────────── Oumi Platform ───────────────────────────────────────────┐**

- ┌─────────────── Core Components ───────────────┐    ┌──────── Infrastructure Layer ──────────┐   │

- ┌─────────┐    ┌──────────┐   ┌────────┐   │    │   ┌─────────────┐   ┌──────────────┐  │   │
- Train   │◄──►│ Evaluate │◄─►│ Infer  │   │    │   │ Config      │◄─►│ Model        │  │   │
- ┘    └──────────┘   └────────┘   │    │   │ Management  │   │ Registry     │  │   │
- ▲              ▲             ▲        │    │   └─────────────┘   └──────────────┘  │   │
- ▲                  ▲         │   │

- ┌─────────┐    ┌──────────┐   ┌────────┐ │    │   ┌─────────────┐   ┌──────────────┐  │   │
- Judge   │◄──►│ Launch   │◄─►│ Deploy │ │    │   │ Data        │◄─►│ Distributed  │  │   │
- ┘    └──────────┘   └────────┘ │    │   │ Pipeline    │   │ Computing    │  │   │
- ┘   └──────────────┘  │   │
- ┘    └─────────────────────────────────────┘   │

- ┌─────────────── External Integrations ────────────────────────────────────────────────────┐ │

- ┌─────────┐    ┌──────────┐    ┌────────┐    ┌────────┐    ┌────────┐    ┌────────┐  │ │
- AWS     │    │ GCP      │    │ Azure  │    │ Lambda │    │ vLLM   │    │ Other  │  │ │
- ┘    └──────────┘    └────────┘    └────────┘    └────────┘    └────────┘  │ │
- ┘ │
- ┘

##### Key Design Elements

a. Core Components:
- Training system with support for different methods (SFT, LoRA, QLoRA, DPO)
- Evaluation system with benchmark integration
- Inference system with multiple engine support (vLLM, SGLang)
- Judge system for data curation
- Launch/deployment system for cloud platforms

b. Data Flow:
- How data moves between components
- Configuration flow
- Model artifact paths

c. Integration Points:
- Cloud platform connections (AWS, GCP, Azure, Lambda)
- External service integrations (vLLM, SGLang)
- Model hub interactions

##### Implementation Details

- Relationship between the src/oumi directory structure and components
- Configuration management through the configs directory
- Test integration via the tests directory
- Documentation system integration

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Main Platform Container
    subgraph Oumi["Oumi Platform"]
        %% Core Components Section
        subgraph Core["Core Components"]
            Train["Training System"]
            Eval["Evaluation System"]
            Infer["Inference System"]
            Judge["Judge System"]
            Launch["Launch System"]
            
            Train <--> Eval
            Eval <--> Infer
            Train <--> Judge
            Judge <--> Launch
            Launch <--> Infer
        end

        %% Infrastructure Layer
        subgraph Infra["Infrastructure Layer"]
            Config["Configuration Management"]
            DataPipe["Data Pipeline"]
            ModelReg["Model Registry"]
            DistComp["Distributed Computing"]
            
            Config <--> ModelReg
            DataPipe <--> DistComp
            ModelReg <--> DistComp
            Config <--> DataPipe
        end

        %% External Integrations
        subgraph External["Cloud & External Services"]
            AWS["AWS"]
            GCP["GCP"]
            Azure["Azure"]
            Lambda["Lambda"]
            VLLM["vLLM"]
            Other["Other Services"]
        end

        %% Connections between sections
        Core --> Infra
        Infra --> External
    end

    %% Styling
    classDef core fill:#2374ab,stroke:#2374ab,color:#fff
    classDef infra fill:#48a9a6,stroke:#48a9a6,color:#fff
    classDef external fill:#e4572e,stroke:#e4572e,color:#fff
    classDef container fill:none,stroke:#666,stroke-width:2px,color:#666

    %% Apply styles
    class Train,Eval,Infer,Judge,Launch core
    class Config,DataPipe,ModelReg,DistComp infra
    class AWS,GCP,Azure,Lambda,VLLM,Other external
    class Oumi,Core,Infra,External container

    %% Click events for component mapping
    click Train "https://github.com/oumi-ai/oumi/blob/main/src/oumi/train.py"
    click Eval "https://github.com/oumi-ai/oumi/blob/main/src/oumi/evaluate.py"
    click Infer "https://github.com/oumi-ai/oumi/blob/main/src/oumi/infer.py"
    click Judge "https://github.com/oumi-ai/oumi/blob/main/src/oumi/judge.py"
    click Launch "https://github.com/oumi-ai/oumi/tree/main/src/oumi/launcher/"
    click Config "https://github.com/oumi-ai/oumi/tree/main/configs/"
    click DataPipe "https://github.com/oumi-ai/oumi/tree/main/src/oumi/core/datasets/"
    click ModelReg "https://github.com/oumi-ai/oumi/tree/main/src/oumi/models/"
    click DistComp "https://github.com/oumi-ai/oumi/blob/main/src/oumi/core/distributed.py"
    click GCP "https://github.com/oumi-ai/oumi/blob/main/configs/examples/bulk_inference/gcp_job.yaml"
```
