# hfailab/hai-platform architecture

> Architecture diagram of the GitHub repository hfailab/hai-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hfailab/hai-platform
- Repository: https://github.com/hfailab/hai-platform

## Overview

##### Project type and purpose
- Project Type: A full‐stack, time-sharing training orchestration platform with a Python REST API server, a CLI client, a scheduling engine, task managers, and k8s integration.
- Purpose: Provide multi-tenant training task scheduling on Kubernetes, manage Jupyter development containers, expose a CLI and UI, and support pluggable storage backends.

##### File structure
- api/ & uvicorn_server.py – the FastAPI HTTP server (routes, dependency injection, query/operation modules)
- client/ & plugins/ – the CLI (hai-cli) and its extension points (haienv, haiworkspace)
- scheduler/ & roman_parliament/ & server_model/ – the scheduling engine, message broker abstractions, server‐side models and implementations
- k8s/ & k8s_watcher/ – Kubernetes API client and event watchers to create/delete pods, configmaps, deployments
- experiment_manager/ – long-running processes that manage task lifecycle (checking logs, resource release, client handling)
- cloud_storage/ – pluggable storage provider interface (NFS, Ceph, OSS)
- db/ & db_schemas/ – database access layer for PostgreSQL and Redis, plus SQL schema files
- one/ & marsv2/ – container entrypoints and scripts used inside the all-in-one Docker/K8s image
- conf/ – configuration modules and toml overrides
- docs/ – static documentation site (not part of runtime)

##### README for architectural insights
- Components to deploy: webservice, Postgres, Redis, shared filesystem, Kubernetes cluster
- Deployment methods: docker-compose or k8s (one/up scripts)
- Users interact via hai-cli or the Studio UI (separate repo)
- override.toml drives runtime configuration for DB, scheduler, launcher, jupyter ingress

##### Main components & their interactions
- Client Layer
– hai-cli (Python CLI) and optional Studio UI
– communicates over HTTP/HTTPS to the API server
- API Server (FastAPI + Uvicorn)
– Exposes REST endpoints under api/app.py
– Uses server_model/* to implement domain logic
– Persists state in Postgres (user, quota, storage, etc.) and Redis (caching, locking)
– Publishes scheduling requests on an internal bus (roman_parliament)
- Scheduler Engine
– scheduler/base_model + modules configure assigners, matchers, feedbackers
– Subscribes to tasks on the bus, decides next resource assignments
– Feeds back to the API and launches via launcher or k8s client
- Experiment Manager
– A set of worker processes that watch task status, collect logs, free resources
– Subscribes/publishes via the same message broker abstractions
- Kubernetes Integration
– k8s/* and k8s_watcher/* handle pod creation, deletion, state watching
– Uses ConfigMaps (for scripts in marsv2), secrets, and RBAC to manage job pods
- Storage Layer
– cloud_storage/provider implements interfaces to NFS, Ceph, OSS
– Shared filesystem mounted into pods for code, logs, and config
- Database & Cache
– PostgreSQL stores system state, user info, quota, task metadata
– Redis used for locks, lightweight queues, and pub/sub in roman_parliament
- Configuration & Orchestration
– one/ and marsv2/ scripts prepare the container image, supervisord config, and entrypoints
– deploy/ scripts initialize DB schemas and override.toml templates

##### Architectural patterns & design principles
- Modular Layering: clear separation between HTTP API, domain logic (server_model), scheduler, and infrastructure (k8s, storage, DB)
- Adapter Pattern: cloud_storage/provider, db/implement, server_model/task_impl implement pluggable back-ends
- Event-Driven: roman_parliament and Redis pub/sub for decoupled communication between API, scheduler, and managers
- Dependency Injection: FastAPI dependencies in api/depends
- Plugin Architecture: client/commands and plugins/haienv, haiworkspace allow extending CLI functionality
- Infrastructure as Code: one/ and deploy/ generate k8s manifests and docker images

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        Client["hai-cli"]:::frontend
        Env["haienv Plugin"]:::frontend
        Workspace["haiworkspace Plugin"]:::frontend
        Studio["Studio UI"]:::frontend
    end

    %% API Server
    subgraph "API Server"
        FastAPI["FastAPI Server"]:::control-plane
        Uvicorn["Uvicorn"]:::control-plane
        APIModules["API Modules"]:::control-plane
    end

    %% Domain Logic
    subgraph "Domain Logic (Server Model)"
        DomainLogic["Server Model"]:::control-plane
    end

    %% Scheduler & Message Bus
    subgraph "Scheduler & Message Bus"
        Scheduler["Scheduler Engine"]:::control-plane
        Bus["Redis Pub/Sub & Roman_Parliament"]:::control-plane
    end

    %% Experiment Manager
    subgraph "Experiment Manager"
        Manager["Experiment Manager"]:::control-plane
    end

    %% Kubernetes Integration
    subgraph "Kubernetes Integration"
        K8sApi["K8s API Client"]:::infra
        K8sWatcher["K8s Watchers"]:::infra
    end

    %% Storage Layer
    subgraph "Storage Layer"
        Storage["Cloud Storage Providers"]:::infra
    end

    %% Databases & Cache
    subgraph "Databases & Cache"
        Postgres["PostgreSQL"]:::infra
        Redis["Redis"]:::infra
    end

    %% Container Entrypoints & Orchestration
    subgraph "Container Entrypoints & Orchestration"
        Entrypoints["one/ scripts"]:::infra
        Marsv2["marsv2 scripts"]:::infra
    end

    %% Configuration & Deployment
    subgraph "Configuration & Deployment"
        Config["Configuration (conf/)"]:::infra
        Deploy["Deployment Scripts"]:::infra
    end

    %% Connections
    Client -->|"HTTP"| FastAPI
    Studio -->|"HTTP"| FastAPI
    Client -->|"uses"| Env
    Client -->|"uses"| Workspace
    FastAPI -->|"loads"| APIModules
    FastAPI -->|"calls"| DomainLogic
    FastAPI -->|"writes"| Postgres
    FastAPI -->|"reads/writes"| Redis
    FastAPI -->|"publishes"| Bus
    Scheduler <-->|"subscribes/publishes"| Bus
    Scheduler -->|"launches"| K8sApi
    Manager <-->|"subscribes/publishes"| Bus
    Manager -->|"organizes pods"| K8sApi
    K8sApi -->|"watches events"| K8sWatcher
    Storage -->|"mounted on pods"| K8sApi
    Config -->|"drives"| FastAPI
    Config -->|"drives"| Scheduler
    Config -->|"drives"| Manager
    Deploy -->|"provides templates"| Config

    %% Click Events
    click Client "https://github.com/hfailab/hai-platform/tree/main/client/"
    click Env "https://github.com/hfailab/hai-platform/tree/main/plugins/haienv/"
    click Workspace "https://github.com/hfailab/hai-platform/tree/main/plugins/haiworkspace/"
    click FastAPI "https://github.com/hfailab/hai-platform/blob/main/api/app.py"
    click Uvicorn "https://github.com/hfailab/hai-platform/blob/main/uvicorn_server.py"
    click APIModules "https://github.com/hfailab/hai-platform/tree/main/api/depends/"
    click DomainLogic "https://github.com/hfailab/hai-platform/tree/main/server_model/"
    click Scheduler "https://github.com/hfailab/hai-platform/tree/main/scheduler/"
    click Bus "https://github.com/hfailab/hai-platform/tree/main/roman_parliament/"
    click Manager "https://github.com/hfailab/hai-platform/tree/main/experiment_manager/manager/"
    click K8sApi "https://github.com/hfailab/hai-platform/tree/main/k8s/"
    click K8sWatcher "https://github.com/hfailab/hai-platform/tree/main/k8s_watcher/"
    click Storage "https://github.com/hfailab/hai-platform/tree/main/cloud_storage/"
    click Postgres "https://github.com/hfailab/hai-platform/tree/main/db/"
    click Redis "https://github.com/hfailab/hai-platform/tree/main/db/"
    click Entrypoints "https://github.com/hfailab/hai-platform/tree/main/one/"
    click Marsv2 "https://github.com/hfailab/hai-platform/tree/main/marsv2/"
    click Config "https://github.com/hfailab/hai-platform/tree/main/conf/"
    click Deploy "https://github.com/hfailab/hai-platform/tree/main/deploy/"

    %% Styles
    classDef frontend fill:#BBDEFB,stroke:#1E88E5,stroke-width:1px
    classDef control-plane fill:#C8E6C9,stroke:#2E7D32,stroke-width:1px
    classDef infra fill:#FFE0B2,stroke:#EF6C00,stroke-width:1px
    classDef optional stroke-dasharray: 5 5
    class Studio optional
```
