# phu142857/ml-air architecture

> Architecture diagram of the GitHub repository phu142857/ml-air, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/phu142857/ml-air
- Repository: https://github.com/phu142857/ml-air

## Overview

MLAir is a lifecycle control plane for governed ML training. The operator workflow pins an immutable dataset version, evaluates readiness and policy eligibility, starts a gated run, executes its tasks, and promotes a model; the Hub surfaces lifecycle and execution status. Pipelines provide the execution substrate. The README also describes identity/security, domain-event audit and timeline, observability, plugins, and realtime capabilities. Some implementation paths are unsampled, so documented branches are shown without inferred internal call chains.

Main entry points: [`api/app/api/routes/v1.py`](https://github.com/phu142857/ml-air/blob/main/api/app/api/routes/v1.py) (REST control plane), [`scheduler/main.py`](https://github.com/phu142857/ml-air/blob/main/scheduler/main.py) (DAG scheduler), [`api/app/domains/orchestration/run_service.py`](https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/run_service.py) (Run lifecycle), [`executor/main.py`](https://github.com/phu142857/ml-air/blob/main/executor/main.py) (Task executor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_hub["Hub experience"]
  node_hub_ui["MLAir Hub<br/>[page.tsx]"]
  node_run_ui["Runs and models<br/>[page.tsx]"]
end

subgraph group_lifecycle["Lifecycle governance"]
  node_api["REST control plane<br/>[v1.py]"]
  node_dataset["Dataset versions"]
  node_readiness["Readiness gates"]
  node_policy["Policy evaluation<br/>[policy_engine.py]"]
  node_models["Model governance"]
  node_lineage["Lineage tracking<br/>[lineage_service.py]"]
end

subgraph group_execution["Execution platform"]
  node_run_service["Run lifecycle<br/>[run_service.py]"]
  node_pipeline["Pipeline versions"]
  node_scheduler["DAG scheduler<br/>[main.py]"]
  node_task_service["Task coordination"]
  node_executor["Task executor<br/>[main.py]"]
  node_plugins["Plugin runtime<br/>[loader.py]"]
end

subgraph group_identity["Identity and access"]
  node_identity["Identity services"]
  node_auth["Authentication and RBAC<br/>[auth_service.py]"]
  node_identity_store[("Identity repository")]
end

subgraph group_insight["Audit and observability"]
  node_domain_events["Domain event dispatch"]
  node_audit["Domain audit"]
  node_audit_query["Audit queries"]
  node_projections["Timeline projections"]
  node_tracing["Trace services<br/>[trace_service.py]"]
  node_realtime["Hub realtime<br/>[main.py]"]
end

node_operator(("ML operator"))
node_postgres[("PostgreSQL")]
node_redis["Redis queues"]

node_operator -->|"uses"| node_hub_ui
node_hub_ui -->|"requests lifecycle"| node_api
node_api -.->|"manages versions"| node_dataset
node_api -.->|"evaluates readiness"| node_readiness
node_api -.->|"checks eligibility"| node_policy
node_api -->|"creates runs"| node_run_service
node_run_service -->|"schedules run"| node_scheduler
node_scheduler -->|"coordinates tasks"| node_task_service
node_scheduler -->|"enqueues tasks"| node_executor
node_executor -.->|"runs plugins"| node_plugins
node_executor -->|"submits lineage"| node_api
node_api -.->|"versions pipelines"| node_pipeline
node_api -.->|"governs promotion"| node_models
node_api -.->|"tracks lineage"| node_lineage
node_run_service -->|"reports run status"| node_run_ui
node_models -->|"shows model stages"| node_run_ui
node_api -->|"authenticates requests"| node_auth
node_auth -->|"resolves identity"| node_identity
node_identity -->|"reads and writes"| node_identity_store
node_domain_events -->|"publishes events"| node_audit
node_audit -.->|"stores audit events"| node_audit_query
node_audit_query -->|"feeds timeline"| node_projections
node_api -.->|"exposes observability"| node_tracing
node_scheduler -->|"publishes run updates"| node_realtime
node_dataset -.->|"persists versions"| node_postgres
node_identity_store -->|"persists identity"| node_postgres
node_identity -->|"limits MFA attempts"| node_redis
node_operator -->|"calls REST API"| node_api

click node_hub_ui "https://github.com/phu142857/ml-air/blob/main/frontend/app/(dashboard)/datasets/%5BdatasetId%5D/page.tsx"
click node_run_ui "https://github.com/phu142857/ml-air/blob/main/frontend/app/(dashboard)/runs/%5BrunId%5D/page.tsx"
click node_api "https://github.com/phu142857/ml-air/blob/main/api/app/api/routes/v1.py"
click node_dataset "https://github.com/phu142857/ml-air/blob/main/api/app/domains/lifecycle/dataset_aggregate.py"
click node_readiness "https://github.com/phu142857/ml-air/blob/main/api/app/domains/lifecycle/readiness_service.py"
click node_policy "https://github.com/phu142857/ml-air/blob/main/api/app/domains/policy/policy_engine.py"
click node_run_service "https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/run_service.py"
click node_pipeline "https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/pipeline_version_service.py"
click node_scheduler "https://github.com/phu142857/ml-air/blob/main/scheduler/main.py"
click node_task_service "https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/worker_task_service.py"
click node_executor "https://github.com/phu142857/ml-air/blob/main/executor/main.py"
click node_plugins "https://github.com/phu142857/ml-air/blob/main/api/app/plugins/loader.py"
click node_models "https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/model_registry_service.py"
click node_lineage "https://github.com/phu142857/ml-air/blob/main/api/app/domains/lifecycle/lineage_service.py"
click node_identity "https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/identity_service.py"
click node_auth "https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/auth_service.py"
click node_identity_store "https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/identity_repository.py"
click node_domain_events "https://github.com/phu142857/ml-air/blob/main/api/app/domains/shared/events/domain_event_dispatch.py"
click node_audit "https://github.com/phu142857/ml-air/blob/main/api/app/domains/audit/domain_audit_subscriber.py"
click node_audit_query "https://github.com/phu142857/ml-air/blob/main/api/app/domains/audit/domain_audit_query_repository.py"
click node_projections "https://github.com/phu142857/ml-air/blob/main/api/app/domains/projections/projection_query_service.py"
click node_tracing "https://github.com/phu142857/ml-air/blob/main/api/app/domains/observability/trace_service.py"
click node_realtime "https://github.com/phu142857/ml-air/blob/main/realtime/app/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_hub_ui,node_run_ui toneBlue
class node_api,node_dataset,node_readiness,node_policy,node_models,node_lineage,node_postgres,node_redis toneAmber
class node_run_service,node_pipeline,node_scheduler,node_task_service,node_executor,node_plugins toneMint
class node_identity,node_auth,node_identity_store toneRose
class node_domain_events,node_audit,node_audit_query,node_projections,node_tracing,node_realtime,node_operator toneIndigo
```

## Components

### Hub experience

- [MLAir Hub](https://github.com/phu142857/ml-air/blob/main/frontend/app/(dashboard)/datasets/%5BdatasetId%5D/page.tsx): `frontend/app/(dashboard)/datasets/[datasetId]/page.tsx`
- [Runs and models](https://github.com/phu142857/ml-air/blob/main/frontend/app/(dashboard)/runs/%5BrunId%5D/page.tsx): `frontend/app/(dashboard)/runs/[runId]/page.tsx`

### Lifecycle governance

- [REST control plane](https://github.com/phu142857/ml-air/blob/main/api/app/api/routes/v1.py): `api/app/api/routes/v1.py`
- [Dataset versions](https://github.com/phu142857/ml-air/blob/main/api/app/domains/lifecycle/dataset_aggregate.py): `api/app/domains/lifecycle/dataset_aggregate.py`
- [Readiness gates](https://github.com/phu142857/ml-air/blob/main/api/app/domains/lifecycle/readiness_service.py): `api/app/domains/lifecycle/readiness_service.py`
- [Policy evaluation](https://github.com/phu142857/ml-air/blob/main/api/app/domains/policy/policy_engine.py): `api/app/domains/policy/policy_engine.py`
- [Model governance](https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/model_registry_service.py): `api/app/domains/governance/model_registry_service.py`
- [Lineage tracking](https://github.com/phu142857/ml-air/blob/main/api/app/domains/lifecycle/lineage_service.py): `api/app/domains/lifecycle/lineage_service.py`

### Execution platform

- [Run lifecycle](https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/run_service.py): `api/app/domains/orchestration/run_service.py`
- [Pipeline versions](https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/pipeline_version_service.py): `api/app/domains/orchestration/pipeline_version_service.py`
- [DAG scheduler](https://github.com/phu142857/ml-air/blob/main/scheduler/main.py): `scheduler/main.py`
- [Task coordination](https://github.com/phu142857/ml-air/blob/main/api/app/domains/orchestration/worker_task_service.py): `api/app/domains/orchestration/worker_task_service.py`
- [Task executor](https://github.com/phu142857/ml-air/blob/main/executor/main.py): `executor/main.py`
- [Plugin runtime](https://github.com/phu142857/ml-air/blob/main/api/app/plugins/loader.py): `api/app/plugins/loader.py`

### Identity and access

- [Identity services](https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/identity_service.py): `api/app/domains/governance/identity_service.py`
- [Authentication and RBAC](https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/auth_service.py): `api/app/domains/governance/auth_service.py`
- [Identity repository](https://github.com/phu142857/ml-air/blob/main/api/app/domains/governance/identity_repository.py): `api/app/domains/governance/identity_repository.py`

### Audit and observability

- [Domain event dispatch](https://github.com/phu142857/ml-air/blob/main/api/app/domains/shared/events/domain_event_dispatch.py): `api/app/domains/shared/events/domain_event_dispatch.py`
- [Domain audit](https://github.com/phu142857/ml-air/blob/main/api/app/domains/audit/domain_audit_subscriber.py): `api/app/domains/audit/domain_audit_subscriber.py`
- [Audit queries](https://github.com/phu142857/ml-air/blob/main/api/app/domains/audit/domain_audit_query_repository.py): `api/app/domains/audit/domain_audit_query_repository.py`
- [Timeline projections](https://github.com/phu142857/ml-air/blob/main/api/app/domains/projections/projection_query_service.py): `api/app/domains/projections/projection_query_service.py`
- [Trace services](https://github.com/phu142857/ml-air/blob/main/api/app/domains/observability/trace_service.py): `api/app/domains/observability/trace_service.py`
- [Hub realtime](https://github.com/phu142857/ml-air/blob/main/realtime/app/main.py): `realtime/app/main.py`

### Other components

- **ML operator**
- **PostgreSQL**
- **Redis queues**

## Connections

- ML operator → MLAir Hub: uses (evidence: `README.md`)
- MLAir Hub → REST control plane: requests lifecycle (evidence: `README.md`)
- REST control plane → Dataset versions: manages versions
- REST control plane → Readiness gates: evaluates readiness
- REST control plane → Policy evaluation: checks eligibility
- REST control plane → Run lifecycle: creates runs (evidence: `README.md`)
- Run lifecycle → DAG scheduler: schedules run (evidence: `README.md`)
- DAG scheduler → Task coordination: coordinates tasks (evidence: `scheduler/main.py`)
- DAG scheduler → Task executor: enqueues tasks (evidence: `README.md`)
- Task executor → Plugin runtime: runs plugins
- Task executor → REST control plane: submits lineage (evidence: `executor/main.py`)
- REST control plane → Pipeline versions: versions pipelines
- REST control plane → Model governance: governs promotion
- REST control plane → Lineage tracking: tracks lineage
- Run lifecycle → Runs and models: reports run status (evidence: `README.md`)
- Model governance → Runs and models: shows model stages (evidence: `README.md`)
- REST control plane → Authentication and RBAC: authenticates requests
- Authentication and RBAC → Identity services: resolves identity (evidence: `api/app/domains/governance/auth_service.py`)
- Identity services → Identity repository: reads and writes (evidence: `api/app/domains/governance/identity_service.py`)
- Domain event dispatch → Domain audit: publishes events (evidence: `api/app/domains/audit/domain_audit_subscriber.py`)
- Domain audit → Audit queries: stores audit events
- Audit queries → Timeline projections: feeds timeline (evidence: `README.md`)
- REST control plane → Trace services: exposes observability
- DAG scheduler → Hub realtime: publishes run updates (evidence: `README.md`)
- Dataset versions → PostgreSQL: persists versions
- Identity repository → PostgreSQL: persists identity (evidence: `api/app/domains/governance/identity_repository.py`)
- Identity services → Redis queues: limits MFA attempts (evidence: `api/app/domains/governance/identity_service.py`)
- ML operator → REST control plane: calls REST API (evidence: `README.md`)