# rajdeep712/mlops-capstone-project architecture

> Architecture diagram of the GitHub repository rajdeep712/mlops-capstone-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rajdeep712/mlops-capstone-project
- Repository: https://github.com/rajdeep712/mlops-capstone-project

## Overview

#### Purpose and Entry Points
This repository implements an end-to-end Python MLOps workflow: reproducible model training, experiment/model tracking, containerized Flask inference, and AWS/Kubernetes delivery. Primary local entry points are DVC reproduction via `dvc.yaml`, the prediction service in `flask_app/app.py`, and model promotion in `scripts/promote_model.py`.

#### Reproducible Training Orchestration
DVC defines the training pipeline and dependency graph in `dvc.yaml`, with run parameters centralized in `params.yaml` and locked outputs/dependencies in `dvc.lock`. DVC configuration in `.dvc/config` establishes versioned data/pipeline state and is intended to use an S3 remote.

#### Data Acquisition and Preparation
The data boundary begins with ingestion and external source access: `src/data/data_ingestion.py`, `src/connections/s3_connection.py`, and `src/connections/ssms_connection.py`. Raw data then passes through preprocessing and transformation stages in `src/data/data_preprocessing.py` and `src/features/feature_engineering.py`, producing model-ready features.

#### Model Lifecycle
Training, evaluation, and registration are separated into dedicated pipeline stages: `src/model/model_building.py`, `src/model/model_evaluation.py`, and `src/model/register_model.py`. The intended control flow is features → trained scikit-learn model → metrics/artifacts logged to MLflow/Dagshub → selected model registered or promoted. `scripts/promote_model.py` is the explicit operational promotion boundary.

#### Online Inference Service
`flask_app/app.py` is the serving runtime and HTTP entry point. It loads the deployed model and delegates request/input handling or transformations to `flask_app/processing_utility.py`; `flask_app/templates/index.html` supplies the browser-facing interface. Training code and online serving are separate boundaries: serving consumes an already-selected model rather than executing DVC stages.

#### Packaging, Deployment, and CI/CD
`Dockerfile` packages the Flask service for deployment. `deployment.yaml` describes Kubernetes deployment/service infrastructure, intended for Amazon EKS and external exposure. GitHub Actions at `.github/workflows/ci.yaml` automates validation and the delivery path described by the project: tests/DVC work, image build, ECR publication, and EKS deployment. AWS integrations span S3 for DVC data, ECR for images, and EKS for runtime orchestration.

#### Observability and Validation
The deployed Flask API is intended to expose metrics for Prometheus, with Grafana consuming Prometheus data for dashboards; their infrastructure definitions are not present in the tree. Tests focus on the main execution boundaries: inference behavior in `tests/test_flask_app.py` and model behavior in `tests/test_model.py`. Python 3.10 is the primary runtime; dependency manifests are `requirements.txt` and `flask_app/requirements.txt`.

Main entry points: [`flask_app/app.py`](https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/app.py) (Flask API), [`dvc.yaml`](https://github.com/rajdeep712/mlops-capstone-project/blob/main/dvc.yaml) (DVC pipeline), [`.github/workflows/ci.yaml`](https://github.com/rajdeep712/mlops-capstone-project/blob/main/.github/workflows/ci.yaml) (GitHub Actions), [`src/model/model_building.py`](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/model_building.py) (Model training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Training Pipeline"]
  node_dvc{{"DVC pipeline<br/>orchestrator<br/>[dvc.yaml]"}}
  node_params["Run parameters<br/>[params.yaml]"]
  node_dvc_state["DVC state<br/>pipeline state<br/>[dvc.lock]"]
  node_s3[("S3 data remote<br/>AWS storage")]
  node_ingestion["Data ingestion<br/>pipeline stage<br/>[data_ingestion.py]"]
  node_data_connections["AWS connections<br/>data access<br/>[s3_connection.py]"]
  node_preprocessing["Preprocessing<br/>pipeline stage"]
  node_features["Feature engineering<br/>pipeline stage"]
  node_training_stage["Model training<br/>pipeline stage<br/>[model_building.py]"]
  node_evaluation["Model evaluation<br/>pipeline stage"]
  node_mlflow[("MLflow / Dagshub<br/>experiment tracking")]
  node_registration["Model registration<br/>pipeline stage<br/>[register_model.py]"]
  node_promotion["Model promotion<br/>operations script<br/>[promote_model.py]"]
end

subgraph group_serving["Inference Service"]
  node_flask_app{{"Flask API<br/>HTTP inference runtime<br/>[app.py]"}}
  node_processing["Request processing<br/>serving utility"]
  node_ui["Browser interface<br/>HTML template<br/>[index.html]"]
end

subgraph group_delivery["Delivery"]
  node_docker["Service image<br/>container build"]
  node_ci{{"GitHub Actions<br/>CI/CD workflow<br/>[ci.yaml]"}}
  node_ecr[("Amazon ECR<br/>container registry")]
  node_eks{{"Amazon EKS<br/>Kubernetes runtime<br/>[deployment.yaml]"}}
end

node_prometheus[("Prometheus<br/>metrics collection")]
node_grafana["Grafana<br/>dashboards"]

node_params -->|"configures stages"| node_dvc
node_dvc_state -->|"locks runs"| node_dvc
node_dvc -->|"runs"| node_ingestion
node_s3 -->|"source data"| node_ingestion
node_data_connections -->|"accesses"| node_s3
node_ingestion -->|"raw data"| node_preprocessing
node_preprocessing -->|"prepared data"| node_features
node_features -->|"features"| node_training_stage
node_training_stage -->|"trained model"| node_evaluation
node_training_stage -->|"artifacts"| node_mlflow
node_evaluation -->|"metrics"| node_mlflow
node_evaluation -->|"validated model"| node_registration
node_registration -->|"registered model"| node_promotion
node_promotion -.->|"selected model"| node_flask_app
node_ui -->|"requests"| node_flask_app
node_flask_app -->|"delegates inputs"| node_processing
node_docker -->|"packages"| node_flask_app
node_ci -.->|"validates pipeline"| node_dvc
node_ci -->|"builds image"| node_docker
node_docker -->|"publishes image"| node_ecr
node_ci -->|"deploys"| node_eks
node_ecr -->|"image pull"| node_eks
node_eks -->|"hosts"| node_flask_app
node_prometheus -.->|"scrapes metrics"| node_flask_app
node_grafana -->|"queries metrics"| node_prometheus

click node_dvc "https://github.com/rajdeep712/mlops-capstone-project/blob/main/dvc.yaml"
click node_params "https://github.com/rajdeep712/mlops-capstone-project/blob/main/params.yaml"
click node_dvc_state "https://github.com/rajdeep712/mlops-capstone-project/blob/main/dvc.lock"
click node_ingestion "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/data/data_ingestion.py"
click node_data_connections "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/connections/s3_connection.py"
click node_preprocessing "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/data/data_preprocessing.py"
click node_features "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/features/feature_engineering.py"
click node_training_stage "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/model_building.py"
click node_evaluation "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/model_evaluation.py"
click node_registration "https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/register_model.py"
click node_promotion "https://github.com/rajdeep712/mlops-capstone-project/blob/main/scripts/promote_model.py"
click node_flask_app "https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/app.py"
click node_processing "https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/processing_utility.py"
click node_ui "https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/templates/index.html"
click node_docker "https://github.com/rajdeep712/mlops-capstone-project/blob/main/Dockerfile"
click node_ci "https://github.com/rajdeep712/mlops-capstone-project/blob/main/.github/workflows/ci.yaml"
click node_eks "https://github.com/rajdeep712/mlops-capstone-project/blob/main/deployment.yaml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dvc,node_params,node_dvc_state,node_s3,node_ingestion,node_data_connections,node_preprocessing,node_features,node_training_stage,node_evaluation,node_mlflow,node_registration,node_promotion toneBlue
class node_flask_app,node_processing,node_ui toneAmber
class node_docker,node_ci,node_ecr,node_eks toneMint
class node_prometheus,node_grafana toneNeutral
```

## Components

### Training Pipeline

Reproducible DVC-managed model lifecycle.

- [DVC pipeline](https://github.com/rajdeep712/mlops-capstone-project/blob/main/dvc.yaml): orchestrator · Defines reproducible training stages. · `dvc.yaml`
- [Run parameters](https://github.com/rajdeep712/mlops-capstone-project/blob/main/params.yaml): parameters · `params.yaml`
- [DVC state](https://github.com/rajdeep712/mlops-capstone-project/blob/main/dvc.lock): pipeline state · Locks dependencies and configures versioned data state. · `dvc.lock`
- **S3 data remote**: AWS storage · DVC remote and external data source.
- [Data ingestion](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/data/data_ingestion.py): pipeline stage · `src/data/data_ingestion.py`
- [AWS connections](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/connections/s3_connection.py): data access · S3 and SSM access adapters. · `src/connections/s3_connection.py`
- [Preprocessing](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/data/data_preprocessing.py): pipeline stage · `src/data/data_preprocessing.py`
- [Feature engineering](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/features/feature_engineering.py): pipeline stage · Produces model-ready features. · `src/features/feature_engineering.py`
- [Model training](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/model_building.py): pipeline stage · Builds a scikit-learn model. · `src/model/model_building.py`
- [Model evaluation](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/model_evaluation.py): pipeline stage · `src/model/model_evaluation.py`
- **MLflow / Dagshub**: experiment tracking · Receives metrics and model artifacts.
- [Model registration](https://github.com/rajdeep712/mlops-capstone-project/blob/main/src/model/register_model.py): pipeline stage · `src/model/register_model.py`
- [Model promotion](https://github.com/rajdeep712/mlops-capstone-project/blob/main/scripts/promote_model.py): operations script · Selects the model for deployment. · `scripts/promote_model.py`

### Inference Service

Separate online Flask runtime.

- [Flask API](https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/app.py): HTTP inference runtime · Loads the selected model and serves predictions. · `flask_app/app.py`
- [Request processing](https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/processing_utility.py): serving utility · Handles inference inputs and transformations. · `flask_app/processing_utility.py`
- [Browser interface](https://github.com/rajdeep712/mlops-capstone-project/blob/main/flask_app/templates/index.html): HTML template · `flask_app/templates/index.html`

### Delivery

Container build and AWS Kubernetes release path.

- [Service image](https://github.com/rajdeep712/mlops-capstone-project/blob/main/Dockerfile): container build · `Dockerfile`
- [GitHub Actions](https://github.com/rajdeep712/mlops-capstone-project/blob/main/.github/workflows/ci.yaml): CI/CD workflow · Validates, builds, publishes, and deploys. · `.github/workflows/ci.yaml`
- **Amazon ECR**: container registry
- [Amazon EKS](https://github.com/rajdeep712/mlops-capstone-project/blob/main/deployment.yaml): Kubernetes runtime · Runs the externally exposed Flask service. · `deployment.yaml`

### Other components

- **Prometheus**: metrics collection · Scrapes deployed API metrics.
- **Grafana**: dashboards

## Connections

- Run parameters → DVC pipeline: configures stages
- DVC state → DVC pipeline: locks runs
- DVC pipeline → Data ingestion: runs
- S3 data remote → Data ingestion: source data
- AWS connections → S3 data remote: accesses
- Data ingestion → Preprocessing: raw data
- Preprocessing → Feature engineering: prepared data
- Feature engineering → Model training: features
- Model training → Model evaluation: trained model
- Model training → MLflow / Dagshub: artifacts
- Model evaluation → MLflow / Dagshub: metrics
- Model evaluation → Model registration: validated model
- Model registration → Model promotion: registered model
- Model promotion → Flask API: selected model
- Browser interface → Flask API: requests
- Flask API → Request processing: delegates inputs
- Service image → Flask API: packages
- GitHub Actions → DVC pipeline: validates pipeline
- GitHub Actions → Service image: builds image
- Service image → Amazon ECR: publishes image
- GitHub Actions → Amazon EKS: deploys
- Amazon ECR → Amazon EKS: image pull
- Amazon EKS → Flask API: hosts
- Prometheus → Flask API: scrapes metrics
- Grafana → Prometheus: queries metrics