How phu142857/ml-air works
MLAir is a lifecycle control plane for governed ML training. The operator workflow pins an immutable dataset version, evaluates readiness and policy eligibility, starts a gated run, executes its tasks, and promotes a model; the Hub surfaces lifecycle and execution status. Pipelines provide the execution substrate. The README also describes identity/security, domain-event audit and timeline, observability, plugins, and realtime capabilities. Some implementation paths are unsampled, so documented branches are shown without inferred internal call chains.
Main entry points: v1.py REST control plane · scheduler/main.py DAG scheduler · run_service.py Run lifecycle · executor/main.py Task executor
Components 26
Hub experience
- MLAir Hub
frontend/app/(dashboard)/datasets/[datasetId]/page.tsx - Runs and models
frontend/app/(dashboard)/runs/[runId]/page.tsx
Lifecycle governance
- REST control plane
api/app/api/routes/v1.py - Dataset versions
api/app/domains/lifecycle/dataset_aggregate.py - Readiness gates
api/app/domains/lifecycle/readiness_service.py - Policy evaluation
api/app/domains/policy/policy_engine.py - Model governance
api/app/domains/governance/model_registry_service.py - Lineage tracking
api/app/domains/lifecycle/lineage_service.py
Execution platform
- Run lifecycle
api/app/domains/orchestration/run_service.py - Pipeline versions
api/app/domains/orchestration/pipeline_version_service.py - DAG scheduler
scheduler/main.py - Task coordination
api/app/domains/orchestration/worker_task_service.py - Task executor
executor/main.py - Plugin runtime
api/app/plugins/loader.py
Identity and access
- Identity services
api/app/domains/governance/identity_service.py - Authentication and RBAC
api/app/domains/governance/auth_service.py - Identity repository
api/app/domains/governance/identity_repository.py
Audit and observability
- Domain event dispatch
api/app/domains/shared/events/domain_event_dispatch.py - Domain audit
api/app/domains/audit/domain_audit_subscriber.py - Audit queries
api/app/domains/audit/domain_audit_query_repository.py - Timeline projections
api/app/domains/projections/projection_query_service.py - Trace services
api/app/domains/observability/trace_service.py - Hub realtime
realtime/app/main.py
Other components
- ML operator
- PostgreSQL
- Redis queues
Connections 28
- ML operator → MLAir Hub: uses
README.md - MLAir Hub → REST control plane: requests lifecycle
README.md - REST control plane → Dataset versions: manages versions
- REST control plane → Readiness gates: evaluates readiness
- REST control plane → Policy evaluation: checks eligibility
- REST control plane → Run lifecycle: creates runs
README.md - Run lifecycle → DAG scheduler: schedules run
README.md - DAG scheduler → Task coordination: coordinates tasks
scheduler/main.py - DAG scheduler → Task executor: enqueues tasks
README.md - Task executor → Plugin runtime: runs plugins
- Task executor → REST control plane: submits lineage
executor/main.py - REST control plane → Pipeline versions: versions pipelines
- REST control plane → Model governance: governs promotion
- REST control plane → Lineage tracking: tracks lineage
- Run lifecycle → Runs and models: reports run status
README.md - Model governance → Runs and models: shows model stages
README.md - REST control plane → Authentication and RBAC: authenticates requests
- Authentication and RBAC → Identity services: resolves identity
api/app/domains/governance/auth_service.py - Identity services → Identity repository: reads and writes
api/app/domains/governance/identity_service.py - Domain event dispatch → Domain audit: publishes events
api/app/domains/audit/domain_audit_subscriber.py - Domain audit → Audit queries: stores audit events
- Audit queries → Timeline projections: feeds timeline
README.md - REST control plane → Trace services: exposes observability
- DAG scheduler → Hub realtime: publishes run updates
README.md - Dataset versions → PostgreSQL: persists versions
- Identity repository → PostgreSQL: persists identity
api/app/domains/governance/identity_repository.py - Identity services → Redis queues: limits MFA attempts
api/app/domains/governance/identity_service.py - ML operator → REST control plane: calls REST API
README.md