# vucongtuanduong/heart-disease-prediction-mlops architecture

> Architecture diagram of the GitHub repository vucongtuanduong/heart-disease-prediction-mlops, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vucongtuanduong/heart-disease-prediction-mlops
- Repository: https://github.com/vucongtuanduong/heart-disease-prediction-mlops

## Overview

##### Project Type & Purpose
- The project is a data science and machine learning application focused on predicting heart disease. It involves data ingestion, EDA, model training and evaluation, and both batch and web-service model deployments.
- The README emphasizes a full ML lifecycle including data preprocessing, model training (using libraries like scikit-learn, xgboost, catboost), deployment (batch and real-time), orchestration with Docker and MLflow, and monitoring using Evidently.

##### Analyzing the File Structure
- Top-level directories indicate clear segregation of concerns:
- "data": Contains data files and scripts for generating random test data.
- "best-practices": Contains code for testing, building, and automated scripts with Makefiles.
- "deployment": Separates the batch deployment and web-service (Flask/Gunicorn) deployment, each with their own Dockerfiles and configuration (Pipfile, requirements.txt) along with serialized model artifacts.
- "model": Stores the serialized artifacts (rf_model.pkl, scaler.pkl, dict_vectorizer.pkl) produced by the training process.
- "notebooks": Provides an interactive environment for EDA and model development.
- "orchestration": Contains scripts, Docker configuration, and orchestrates workflows for data ingestion, transformation, and model training using pipelines, MLFlow integration, and custom scripts.
- "monitoring": Contains everything related to monitoring the deployed model's performance, including Docker-compose files and notebooks for Evidently reports.
- Additional directories and files (e.g., .github/workflows, code best practices tests, and orchestration charts) confirm a comprehensive, multi-component architecture.

##### Main Components to Identify
a. Data & Preprocessing:
- Data ingestion scripts (in orchestration, data_loaders/ingest.py, and data/create_random_test_data.py).
- Notebooks for exploratory data analysis.

b. Model Training & Lifecycle Management:
- Components in "best-practices/code" and "orchestration/heart-disease-prediction" (for training, pipelines, and transformers).
- Use of MLflow (check orchestration/mlflow and MLFlow-related files) to manage experiments, tracking and model registry.

c. Model Artifact Storage:
- The "model" directory holds the final model, scaler, and vectorizer artifacts.

d. Model Deployment:
- Batch deployment: Under "deployment/batch" where a Docker container is used to perform batch predictions.
- Real-time web service: Under "deployment/web-service" with a Flask/Gunicorn setup to serve predictions.

e. Workflow Orchestration:
- The "orchestration" directory contains Docker-compose, custom scripts, and pipelines that orchestrate model training and data transformation.

f. Model Monitoring:
- The "monitoring" directory shows the integration of Evidently (or similar monitoring tools) to review performance metrics of the deployed model.

g. CI/CD and Best Practices:
- GitHub workflows for CI (in .github/workflows) and testing scripts within best-practices code.

##### Relationships & Data Flow
- Data originates from the "data" directory then flows into preprocessing (notebooks and orchestration/data_loaders), where it may be transformed by the pipelines (orchestration/heart-disease-prediction/pipelines).
- The transformed data is used in model training (best-practices and orchestration training scripts). The trained model and related artifacts are stored in the "model" directory.
- Deployment components (batch and web-service) load these artifacts to serve predictions.
- The orchestration layer integrates model retraining with the ML lifecycle tooling (MLflow) and passes new or updated models to the deployment workflows.
- Monitoring components receive inputs from the deployed services, analyze predictions, and generate performance reports.

##### Architectural Patterns & Technologies
- Microservices/Containerized Architecture: Each deployment scenario (batch, web-service, orchestration, monitoring) has its own Docker environment and configurations.
- Pipeline/Workflow Orchestration: Clear use of pipelines (data ingestion, preparation, and transformation), as seen in orchestration/heart-disease-prediction.
- ML Lifecycle Management: Integration with MLflow for experiment tracking and model registry.
- CI/CD Pipeline: GitHub actions configured under .github/workflows for continuous integration and testing.

##### Specific Elements
- Data Source (data directory and Kaggle dataset)
- EDA/Preprocessing (notebooks, orchestration/data_loaders)
- Model Training (best-practices, orchestration/train scripts)
- Model Artifact Repository (model directory)
- Batch Deployment Service (deployment/batch) and associated Docker container
- Real-Time Prediction Service (deployment/web-service) with Flask/Gunicorn
- Orchestration Framework (orchestration directory’s components, pipelines, MLflow integrated container)
- Monitoring Dashboard (monitoring directory and reports)
- Data flow from ingestion → preprocessing → training
- Model artifact movement from training to deployment
- Feedback loop from monitoring back to retraining or model update processes
- Optionally, note external integrations like Docker, MLflow, and Evidently in a separate annotation block.

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Preprocessing Layer
    subgraph "Data & Preprocessing Layer"
        D1["Ingest Data Script"]:::data
        D2["Random Test Data"]:::data
        D3["EDA/Preprocessing"]:::data
    end

    %% Model Training & MLflow Layer
    subgraph "Model Training & MLflow Layer"
        M1["Model Training Script"]:::training
        M2["Training Pipeline"]:::training
        M3["MLflow Tracking"]:::training
    end

    %% Artifact & Deployment Layer
    subgraph "Artifact & Deployment Layer"
        A1["Model Artifact Repository"]:::deployment
        Dp1["Batch Deployment Service"]:::deployment
        Dp2["Real-Time Prediction Service"]:::deployment
    end

    %% Orchestration & CI/CD Layer
    subgraph "Orchestration & CI/CD Layer"
        O1["Workflow Orchestration"]:::orchestration
        O2["Data Pipelines"]:::orchestration
        C1["CI/CD Pipeline"]:::cicd
        C2["Best Practices Code"]:::cicd
    end

    %% Monitoring Layer
    subgraph "Monitoring Layer"
        Mon1["Model Monitoring"]:::monitoring
    end

    %% Data Flow Connections
    D1 -->|"flows to"| D3
    D2 -->|"flows to"| D3

    D3 -->|"triggers"| M1
    D3 -->|"triggers"| M2

    M1 -->|"logs to"| M3
    M2 -->|"logs to"| M3

    M1 -->|"produces"| A1
    M2 -->|"produces"| A1

    A1 -->|"serves"| Dp1
    A1 -->|"serves"| Dp2

    Dp1 -->|"sends data to"| Mon1
    Dp2 -->|"sends data to"| Mon1

    Mon1 -->|"feedback"| O1
    O1 -->|"triggers retraining"| M1
    O1 -->|"feeds"| O2
    O2 -->|"supports"| M2

    C1 -->|"orchestrates"| O1
    C2 -->|"supports"| M1

    %% Click Events
    click D1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/blob/main/orchestration/heart-disease-prediction/data_loaders/ingest.py"
    click D2 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/blob/main/data/create_random_test_data.py"
    click D3 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/blob/main/notebooks/model.ipynb"
    click M1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/blob/main/best-practices/code/model.py"
    click M2 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/blob/main/orchestration/heart-disease-prediction/data_exporters/train.py"
    click M3 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/orchestration/mlflow/"
    click A1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/model/"
    click Dp1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/deployment/batch/"
    click Dp2 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/deployment/web-service/"
    click O1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/orchestration/"
    click O2 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/orchestration/heart-disease-prediction/pipelines/"
    click C1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/blob/main/.github/workflows/ci-tests.yml"
    click C2 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/best-practices/code/"
    click Mon1 "https://github.com/vucongtuanduong/heart-disease-prediction-mlops/tree/main/monitoring/"

    %% Class Definitions for Styling
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef training fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef deployment fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef orchestration fill:#D8BFD8,stroke:#000,stroke-width:1px;
    classDef monitoring fill:#FF6961,stroke:#000,stroke-width:1px;
    classDef cicd fill:#FFFF99,stroke:#000,stroke-width:1px;
```
