How dfds-data/mlflow-infrastructure works
Project type and purpose
• This is an infrastructure‐as‐code repository for centrally managed MLflow deployments.
• “/infrastructure” defines Terraform for a shared PostgreSQL backend in a central AWS account.
• “/mlflow-image” builds and publishes versioned Docker images of MLflow to ECR.
• “/mlflow-terraform” provides Terraform modules for client teams to provision S3 buckets and IAM roles in their AWS accounts so their Kubernetes‐hosted MLflow servers can store artifacts.
File structure
• Top‐level CI/CD configurations:
• .github/workflows/check-mlflow-release.yml (checks MLflow versions)
• azure-pipelines.yml (build & push MLflow Docker images, apply central Terraform)
• infrastructure/ (central AWS account Terraform for PostgreSQL RDS)
• mlflow-image/ (Dockerfile, docker-compose, pipeline scripts for image build & push to ECR)
• mlflow-terraform/ (client‐side Terraform to create S3 bucket & IAM role for MLflow artifact store)
README for additional insights
• Central DB to avoid per-project servers
• Automated version detection via GitHub Actions → PR → on-merge triggers Azure Pipelines
• Clients directly reference source code of Terraform modules for their releases
Components
a. Version Management & CI/CD
• GitHub repo (with Infra, Image, Terraform modules)
• GitHub Actions (daily MLflow version check)
• Azure Pipelines (on‐merge build & deploy)
b. Container Registry
• AWS ECR (shared repository for MLflow images)
c. Central AWS Account (managed by “/infrastructure”)
• PostgreSQL RDS instance (Terraform)
d. Client AWS Accounts (via “/mlflow-terraform”)
• S3 Bucket (artifact storage)
• IAM Role (assumed by K8s)
e. Kubernetes Clusters (outside this repo, per client)
• Deploy MLflow server pods (pull image from ECR, connect to central RDS, write artifacts to S3 via IAM)
Relationships and data flows
• Build Docker image → Push to ECR
• Apply “/infrastructure” Terraform → Create/Update RDS
• Client team runs “/mlflow-terraform” → Terraform applies in their AWS → S3 + IAM
• K8s cluster pulls MLflow image from ECR, connects to RDS (TCP), writes artifacts to S3 (via IAM)
Architectural patterns and design principles
• Centralized shared‐backend for MLflow state (avoid duplicate RDS)
• Module‐based Terraform for reusability (central vs. client modules)
• GitOps/CI‐driven automation for releases and infra apply
• Containerization of MLflow for uniform deployment