# datatalksclub/mlops-zoomcamp architecture

> Architecture diagram of the GitHub repository datatalksclub/mlops-zoomcamp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datatalksclub/mlops-zoomcamp
- Repository: https://github.com/datatalksclub/mlops-zoomcamp

## Overview

##### Main Components
- Module Components – Each module (intro, experiment tracking, orchestration, deployment, monitoring, best practices, final project) forms a primary “node” of the learning journey.
- Experiment Tracking – Highlight parts that use MLflow (and even WandB in some cohort solutions).
- Model Deployment – Subdivide this into:
– Batch deployment (with notebooks, Python scripts, and Pipenv dependencies),
– Streaming deployment (including Dockerfiles, AWS Lambda functions, and test scripts),
– Web-service deployment (using Flask, Docker, and standalone prediction scripts).
- Orchestration and Pipelines – Identify orchestration components and notebooks/code that demonstrate workflow management (e.g., Prefect, Mage, or custom orchestration scripts).
- Monitoring – Represent both web service monitoring (with Prometheus, Grafana dashboards, Evidently) and batch job monitoring components.
- Best Practices / CI-CD / Infrastructure – Include components that touch on unit/integration testing, linting, pre-commit hooks, GitHub Actions CI/CD workflows, and the Terraform configurations for infrastructure.

##### Relationships and Interactions
- From experiment tracking (with model saving/registry) move to “Orchestration/Pipelines,” which would be the process of managing data workflows and training.
- Next, link “Model Deployment,” splitting into the three deployment types (batch, streaming, web-service) where the trained model is put into production.
- The “Best Practices” module can be drawn as an overlay that touches each component (CI/CD, testing, code quality) and supports the entire pipeline.

##### Architectural Patterns & Technologies
- Modular design – each module encapsulates a stage of the ML lifecycle.
- Microservice-like separation in deployment – distinguish between batch jobs, streaming services, and web API calls.
- CI/CD Integration – include the GitHub workflows (.github/workflows) that trigger tests and deployments.
- Infrastructure as Code – show a side-chain where Terraform scripts (in the best practices module) ensure reproducible deployments.
- Tools/Technologies – Label components with the corresponding technologies (e.g., MLflow, Flask, Docker, AWS Lambda, Prefect, Terraform, Python and Pipfile-based dependency management).

##### Final Touches
- Use clear, legible labels on every node and connection.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Learning Journey"
        intro["Introduction Module"]:::learning
        exp["Experiment Tracking (MLflow & WandB)"]:::learning
        orch["Orchestration & Pipelines"]:::learning
        final["Final Project Module"]:::learning
    end

    subgraph "Model Deployment"
        depBatch["Deployment - Batch Processing"]:::deployment
        depStream["Deployment - Streaming (AWS Lambda)"]:::deployment
        depWebML["Deployment - Web Service (MLflow)"]:::deployment
        depWebAlt["Deployment - Web Service (Alternative)"]:::deployment
    end

    mon["Monitoring (Web & Batch)"]:::monitoring
    best["Best Practices / Terraform"]:::infra
    workflows["CI/CD Workflows"]:::infra

    %% Main flow of the MLOps lifecycle
    intro -->|"next"| exp
    exp -->|"next"| orch
    orch -->|"deploy"| depBatch
    depBatch -->|"next"| depStream
    depStream -->|"next"| depWebML
    depWebML -->|"next"| depWebAlt
    depWebAlt -->|"monitor"| mon
    mon -->|"conclude"| final

    %% Operational support (Best Practices & CI/CD) overlays supporting all stages with dashed lines
    best -.-> intro
    best -.-> exp
    best -.-> orch
    best -.-> depBatch
    best -.-> depStream
    best -.-> depWebML
    best -.-> depWebAlt
    best -.-> mon
    best -.-> final

    workflows -.-> intro
    workflows -.-> exp
    workflows -.-> orch
    workflows -.-> depBatch
    workflows -.-> depStream
    workflows -.-> depWebML
    workflows -.-> depWebAlt
    workflows -.-> mon
    workflows -.-> final

    %% Click Events for Component Mapping
    click intro "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/01-intro"
    click exp "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/02-experiment-tracking"
    click orch "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/03-orchestration"
    click depBatch "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/04-deployment/batch"
    click depStream "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/04-deployment/streaming"
    click depWebML "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/04-deployment/web-service-mlflow"
    click depWebAlt "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/04-deployment/web-service"
    click mon "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/05-monitoring"
    click best "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/06-best-practices"
    click workflows "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/.github/workflows"
    click final "https://github.com/datatalksclub/mlops-zoomcamp/tree/main/07-project"

    %% Styles
    classDef learning fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef deployment fill:#A9DFBF,stroke:#196F3D,stroke-width:2px;
    classDef monitoring fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef infra fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
```
