How oumi-ai/oumi works
Main Components
The system architecture should be divided into these key components:
A. Core Platform Components:
• Training System
• Evaluation System
• Inference System
• Judge System
• Launch/Deployment System
B. Supporting Infrastructure:
• Configuration Management
• Data Pipeline
• Model Registry
• Distributed Computing
• Cloud Integration
Component Relationships
┌─────────────────────────────────────────── Oumi Platform ───────────────────────────────────────────┐
• ┌─────────────── Core Components ───────────────┐ ┌──────── Infrastructure Layer ──────────┐ │
• ┌─────────┐ ┌──────────┐ ┌────────┐ │ │ ┌─────────────┐ ┌──────────────┐ │ │
• Train │◄──►│ Evaluate │◄─►│ Infer │ │ │ │ Config │◄─►│ Model │ │ │
• ┘ └──────────┘ └────────┘ │ │ │ Management │ │ Registry │ │ │
• ▲ ▲ ▲ │ │ └─────────────┘ └──────────────┘ │ │
• ▲ ▲ │ │
• ┌─────────┐ ┌──────────┐ ┌────────┐ │ │ ┌─────────────┐ ┌──────────────┐ │ │
• Judge │◄──►│ Launch │◄─►│ Deploy │ │ │ │ Data │◄─►│ Distributed │ │ │
• ┘ └──────────┘ └────────┘ │ │ │ Pipeline │ │ Computing │ │ │
• ┘ └──────────────┘ │ │
• ┘ └─────────────────────────────────────┘ │
• ┌─────────────── External Integrations ────────────────────────────────────────────────────┐ │
• ┌─────────┐ ┌──────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │
• AWS │ │ GCP │ │ Azure │ │ Lambda │ │ vLLM │ │ Other │ │ │
• ┘ └──────────┘ └────────┘ └────────┘ └────────┘ └────────┘ │ │
• ┘ │
• ┘
Key Design Elements
a. Core Components:
• Training system with support for different methods (SFT, LoRA, QLoRA, DPO)
• Evaluation system with benchmark integration
• Inference system with multiple engine support (vLLM, SGLang)
• Judge system for data curation
• Launch/deployment system for cloud platforms
b. Data Flow:
• How data moves between components
• Configuration flow
• Model artifact paths
c. Integration Points:
• Cloud platform connections (AWS, GCP, Azure, Lambda)
• External service integrations (vLLM, SGLang)
• Model hub interactions
Implementation Details
• Relationship between the src/oumi directory structure and components
• Configuration management through the configs directory
• Test integration via the tests directory
• Documentation system integration