# equipo59/absenteeism_at_work architecture

> Architecture diagram of the GitHub repository equipo59/absenteeism_at_work, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/equipo59/absenteeism_at_work
- Repository: https://github.com/equipo59/absenteeism_at_work

## Overview

##### Project type and purpose
- This is an MLOps‐driven machine learning service (“Absenteeism at Work”) that spans from data versioning and model experimentation to automated deployment of a prediction API.
- It follows the Cookiecutter Data Science pattern, uses DVC for data pipelines, MLflow for experiment tracking, and AWS ECS/Fargate (via CloudFormation) for serving.

##### Main architectural components
a. Data layer
- Raw data (data/raw) versioned with DVC and stored in a remote DVC backend (e.g., S3).
- Intermediate data (data/interim) and processed data (data/processed) artifacts produced by the pipeline.
b. Compute / pipeline layer
- Preprocessing module (`preprocess_data.py` → data/processed)
- Feature engineering (`features.py`)
- Model training pipeline (`modeling/train.py`, Jupyter notebooks)
- Experiment tracking server (MLflow `mlruns/`) logging params, metrics, artifacts
c. Model registry / artifact store
- Persisted model files in `models/` and `mlruns/models/...`
d. Serving layer
- Flask/FastAPI app (`app.py`) plus static web UI (`static/index.html`, `script.js`, `style.css`)
- Containerized via `Dockerfile` and `docker-compose.yml`
e. CI/CD & infrastructure
- GitHub Actions workflow (`.github/workflows/deploy.yml`)
- Infrastructure as code (CloudFormation template in `infrastructure/cloudformation/infrastructure.yaml`)
- AWS resources: ECR repository, ECS Fargate cluster & service, ALB, auto‐scaling, CloudWatch logs
f. External systems
- GitHub (source + CI/CD trigger)
- DVC remote storage (e.g., S3)
- AWS services (ECS, ECR, ALB, CloudWatch)

##### Relationships & data flow
1. **Code commit (master → web)** triggers GitHub Actions
2. **DVC pipeline** (`make data-pull` / `dvc repro`) reads raw data → preprocessing → processed data
3. **Training module** consumes processed data → produces model artifacts + logs to MLflow
4. **GitHub Actions** uses CloudFormation to provision/update AWS infra
5. **Docker build** packs `app.py`, `absenteeism_at_work/`, selected model artifact → pushes image to ECR
6. **ECS service** (behind ALB) pulls new image → runs API container
7. **Clients** (browser or other services) call API endpoint via ALB → Flask app loads model from local container FS
8. **CloudWatch** aggregates logs and health‐check metrics

##### Architectural patterns & design principles
- Pipeline as code: DVC + `dvc.yaml` define reproducible ETL & training steps
- Modular code: separate modules for config, dataset, features, modeling, API
- Experiment tracking: MLflow for metrics, params, artifact versioning
- Infrastructure as code: CloudFormation for full-stack AWS resources
- Containerization & microservice: API container deployed on ECS Fargate behind ALB
- CI/CD: GitHub Actions driving infra provision, image build/push, service update

##### Essential elements
- Layer separation (Data → Pipeline → Registry → Serving → External)
- Clear labels for each artifact, service, and interaction
- Notes on health checks and logging (CloudWatch)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Data Layer"
        Raw["Raw Data: work_absenteeism_raw.csv"]:::storage
        Interm["Intermediate Data: data/interim"]:::storage
        Proc["Processed Data: work_absenteeism_processed.csv"]:::storage
    end

    subgraph "Pipeline Layer"
        DVC["DVC Pipeline: dvc.yaml"]:::doc
        Preprocess["Preprocessing Module"]:::compute
        Features["Feature Engineering Module"]:::compute
        Dataset["Dataset Abstraction"]:::compute
        Train["Model Training Script"]:::compute
        MLflowSmoke["MLflow Smoke-Test Script"]:::compute
        MLflowExp["MLflow Experiment Script"]:::compute
        subgraph "Jupyter Notebooks"
            Notebooks1["data_preparation.ipynb"]:::doc
            Notebooks2["eda_fe.ipynb"]:::doc
            Notebooks3["model_train.ipynb"]:::doc
        end
    end

    subgraph "Model Registry / Artifact Store"
        MLflowStore["MLflow Tracking Store"]:::storage
        ModelsReg["Model Artifacts (best_model.joblib)"]:::storage
    end

    subgraph "Serving Layer"
        API["Flask/FastAPI App (app.py)"]:::compute
        APIModule["API Module"]:::compute
        subgraph "Static Web UI"
            UIIndex["index.html"]:::compute
            UIScript["script.js"]:::compute
            UIStyle["style.css"]:::compute
        end
        DockerfileNode["Dockerfile"]:::doc
        Compose["docker-compose.yml"]:::doc
        Req["requirements-api.txt"]:::doc
    end

    subgraph "CI/CD & Infrastructure"
        CI["GitHub Actions Workflow"]:::doc
        CF["CloudFormation Template"]:::doc
        DeployScript["Deployment Script"]:::doc
        Env[".env.example"]:::doc
        Config["config.py"]:::doc
        Makefile["Makefile"]:::doc
    end

    subgraph "AWS Infrastructure"
        ECR["ECR Repository"]:::aws
        ECS["ECS Fargate Service"]:::aws
        ALB["Application Load Balancer"]:::aws
        CW["CloudWatch Logs"]:::aws
    end

    subgraph "External"
        GitHub["GitHub"]:::doc
        DVCRemote["DVC Remote (S3)"]:::aws
        Clients["Clients (Browser/Services)"]:::doc
    end

    GitHub -->|"on git push"| CI
    CI -->|"provision/update"| CF
    CF -->|"creates"| ECR
    CF -->|"creates"| ECS
    CF -->|"creates"| ALB
    CF -->|"creates"| CW
    DVCRemote -->|"dvc pull"| Raw
    Raw -->|"dvc repro"| Preprocess
    Preprocess --> Features
    Features --> Dataset
    Features --> Train
    Dataset --> Train
    Train -->|"mlflow log"| MLflowStore
    Train --> ModelsReg
    DVC -->|"defines pipeline"| Preprocess
    DVC --> Features
    DVC --> Train
    Notebooks1 --> Preprocess
    Notebooks2 --> Features
    Notebooks3 --> Train
    ModelsReg -->|"included in image"| DockerfileNode
    CI -->|"docker build & push"| ECR
    ECS -->|"pulls image"| ECR
    ALB -->|"routes HTTP"| ECS
    Clients -->|"HTTP request /predict"| ALB
    ECS -->|"logs"| CW
    API --> ModelsReg
    APIModule --> API
    UIIndex --> API
    UIScript --> API
    UIStyle --> API
    DockerfileNode --> Req
    CI --> DeployScript

    click Raw "https://github.com/equipo59/absenteeism_at_work/blob/master/data/raw/work_absenteeism_raw.csv"
    click Interm "https://github.com/equipo59/absenteeism_at_work/tree/master/data/interim/"
    click Proc "https://github.com/equipo59/absenteeism_at_work/blob/master/data/processed/work_absenteeism_processed.csv"
    click DVC "https://github.com/equipo59/absenteeism_at_work/blob/master/dvc.yaml"
    click Preprocess "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/preprocess_data.py"
    click Features "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/features.py"
    click Dataset "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/dataset.py"
    click Train "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/modeling/train.py"
    click MLflowSmoke "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/modeling/mlflow_smoke_test.py"
    click MLflowExp "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/modeling/mlflow_absenteeism_train.py"
    click Notebooks1 "https://github.com/equipo59/absenteeism_at_work/blob/master/notebooks/Phase1/data_preparation.ipynb"
    click Notebooks2 "https://github.com/equipo59/absenteeism_at_work/blob/master/notebooks/Phase1/eda_fe.ipynb"
    click Notebooks3 "https://github.com/equipo59/absenteeism_at_work/blob/master/notebooks/Phase1/model_train.ipynb"
    click MLflowStore "https://github.com/equipo59/absenteeism_at_work/tree/master/mlruns/"
    click ModelsReg "https://github.com/equipo59/absenteeism_at_work/blob/master/models/best_model.joblib"
    click API "https://github.com/equipo59/absenteeism_at_work/blob/master/app.py"
    click APIModule "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/api.py"
    click UIIndex "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/static/index.html"
    click UIScript "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/static/script.js"
    click UIStyle "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/static/style.css"
    click DockerfileNode "https://github.com/equipo59/absenteeism_at_work/tree/master/Dockerfile"
    click Compose "https://github.com/equipo59/absenteeism_at_work/blob/master/docker-compose.yml"
    click Req "https://github.com/equipo59/absenteeism_at_work/blob/master/requirements-api.txt"
    click CI "https://github.com/equipo59/absenteeism_at_work/blob/master/.github/workflows/deploy.yml"
    click CF "https://github.com/equipo59/absenteeism_at_work/blob/master/infrastructure/cloudformation/infrastructure.yaml"
    click DeployScript "https://github.com/equipo59/absenteeism_at_work/blob/master/scripts/deploy.sh"
    click Env "https://github.com/equipo59/absenteeism_at_work/blob/master/.env.example"
    click Config "https://github.com/equipo59/absenteeism_at_work/blob/master/absenteeism_at_work/config.py"
    click Makefile "https://github.com/equipo59/absenteeism_at_work/tree/master/Makefile"

    classDef storage fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef compute fill:#90ee90,stroke:#333,stroke-width:1px
    classDef aws fill:#ffcc99,stroke:#333,stroke-width:1px
    classDef doc fill:#d3d3d3,stroke:#333,stroke-width:1px
```
