# matheusham/minimo_mlops architecture

> Architecture diagram of the GitHub repository matheusham/minimo_mlops, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matheusham/minimo_mlops
- Repository: https://github.com/matheusham/minimo_mlops

## Overview

##### Project type and purpose
- This is a minimal MLOps demo that combines data engineering, model training, and model serving in one Docker‐ized service.
- The goal: consume a CSV of credit‐card transactions, build a “revenge spending” classifier, persist the model, and expose a REST API for predictions.

##### File structure
- Top level:
- Dockerfile / Makefile / requirements.txt → container build & execution
- assets/ → raw input data (MiBolsillo.csv)
- data_engineering/ → ETL scripts & exploratory notebook
- model_pipeline/ → training pipeline script & exploration notebook
- model_serving/ → FastAPI app (main.py) + Makefile
- No external microservices—everything lives in one container, but logically it’s three stages.

##### README for extra insight
- Describes a minimal MLOps ecosystem: data ingestion → feature engineering → training → evaluation → serving → monitoring.
- Mentions technologies: Python, PySpark (hinted), scikit‐learn, FastAPI, Docker, optionally DVC/MLflow.

1) Data Source
- CSV file (assets/MiBolsillo.csv)
2) Data Engineering Stage
- prod_data_eng_pipeline.py (pyspark or pandas transforms)
3) Model Training Stage
- prod_model_pipeline.py (scikit‐learn pipeline, hyperparam tuning, model serialization)
4) Model Serving Stage
- FastAPI app (main.py), /predict endpoint
5) Docker Container
- Builds image, runs the three stages at container startup, then hosts FastAPI
6) Client
- Browser or HTTP client hitting the Swagger UI /docs and the prediction API

b. Relationships & data flow:
- Surround Data Eng, Training, Serving with a “Docker Container” boundary

c. Architectural patterns & design principles:
- Pipeline pattern: sequential ETL → train → serve
- Single‐container MLOps demo (for simplicity)
- Separation of concerns: three logical modules in code
- Potential extension points: swap out data source for streaming, external feature store, separate microservices

d. Key technologies / libraries:
- Python 3.x
- PySpark/Pandas (data_engineering)
- scikit‐learn (model_pipeline)
- FastAPI + Uvicorn (model_serving)
- Docker + Make (or direct docker CLI)
- Optionally DVC/MLflow for model versioning (mentioned in README)

##### Extra details
- The Dockerfile as the orchestration mechanism: annotate that it builds the Python environment, installs dependencies, then runs data_eng → model_train → uvicorn main:app.
- Mention the Makefile targets (make build, make run) as convenience scripts.
- Where model artifacts live (e.g., /app/model.pkl inside container).

- Data Ingestion → Feature Engineering → Model Training → Model Serving
- All steps containerized and orchestrated by Docker/Make
- Client interaction via REST API

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Docker Container"  
        direction TB
        DS["MiBolsillo.csv\n(assets)"]:::data
        subgraph "Data Engineering Stage"
            direction TB
            DE_PY["prod_data_eng_pipeline.py\n(PySpark)"]:::compute
            DE_NOTE["exploratory_analysis.ipynb"]:::compute
        end
        subgraph "Model Training Stage"
            direction TB
            MT_PY["prod_model_pipeline.py\n(scikit-learn)"]:::compute
            MT_NOTE["sklearn_pipeline_explore.ipynb"]:::compute
        end
        subgraph "Model Serving Stage"
            direction TB
            MS_APP["main.py\n(FastAPI)"]:::serving
            MS_MAKE["Makefile"]:::infra
        end
        DOCKERFILE["Dockerfile"]:::infra
        MAKE_ROOT["Makefile"]:::infra
        REQ["requirements.txt"]:::infra
        DS --> DE_PY --> MT_PY --> MS_APP
        MS_APP -->|"exposed port"| Client
    end

    Client["Client\nHTTP Request"]:::external
    DOC["README.md"]:::docs

    click DS "https://github.com/matheusham/minimo_mlops/blob/main/assets/MiBolsillo.csv"
    click DE_PY "https://github.com/matheusham/minimo_mlops/blob/main/data_engineering/prod_data_eng_pipeline.py"
    click DE_NOTE "https://github.com/matheusham/minimo_mlops/blob/main/data_engineering/exploratory_analysis.ipynb"
    click MT_PY "https://github.com/matheusham/minimo_mlops/blob/main/model_pipeline/prod_model_pipeline.py"
    click MT_NOTE "https://github.com/matheusham/minimo_mlops/blob/main/model_pipeline/sklearn_pipeline_explore.ipynb"
    click MS_APP "https://github.com/matheusham/minimo_mlops/blob/main/model_serving/main.py"
    click MS_MAKE "https://github.com/matheusham/minimo_mlops/tree/main/model_serving/Makefile"
    click DOCKERFILE "https://github.com/matheusham/minimo_mlops/tree/main/Dockerfile"
    click MAKE_ROOT "https://github.com/matheusham/minimo_mlops/tree/main/Makefile"
    click REQ "https://github.com/matheusham/minimo_mlops/blob/main/requirements.txt"
    click DOC "https://github.com/matheusham/minimo_mlops/blob/main/README.md"

    classDef data fill:#cce5ff,stroke:#66b0ff,color:#004085;
    classDef compute fill:#d4edda,stroke:#81c784,color:#155724;
    classDef serving fill:#fff3cd,stroke:#ffecb5,color:#856404;
    classDef infra fill:#e2e3e5,stroke:#c6c8ca,color:#383d41;
    classDef external fill:#f5e6ff,stroke:#d1a3ff,color:#4a0f6b;
    classDef docs fill:#fdfdfe,stroke:#d6d8db,color:#1b1e21;
```
