# iamrajatroy/data-science-lab architecture

> Architecture diagram of the GitHub repository iamrajatroy/data-science-lab, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iamrajatroy/data-science-lab
- Repository: https://github.com/iamrajatroy/data-science-lab

## Overview

##### Project type and purpose
- This is a Data Science Experimentation repository (“Data-Science-Lab”) that contains:
- A variety of Jupyter notebooks implementing ML use‐cases (classification, computer vision explainability, etc.)
- Standalone Python scripts for training (“codecarbon/train.py”, “mlflow/train.py”)
- Emissions tracking via CodeCarbon
- Experiment tracking via MLflow
- A small “translator-app” Flask service as a demo web application

##### Top-level components (based on the file tree)
- /notebook: Jupyter notebooks (interactive experiments)
- /codecarbon: scripts that wrap training with carbon‐emission tracking
- /mlflow: scripts that wrap training with MLflow experiment tracking
- /translator-app: Flask web service (Procfile for Heroku, app.py, translator module)
- Data files: emissions.csv, temp/*.csv

##### Key architectural patterns
- Modular separation: notebooks vs. script-based pipelines
- Sidecar tracking services: CodeCarbon and MLflow integrate as wrappers around training
- A simple web front end for demonstration (translator-app)

##### Main components
a. Data Source(s)
– Public datasets (CSV on Kaggle or other open sources)
– Temporary extracted data under codecarbon/temp/
b. Experiment Orchestration
– Jupyter Notebook environment (interactive)
– Python training scripts:
- codecarbon/train.py (with CodeCarbon SDK, writes emissions.csv)
- mlflow/train.py (with MLflow SDK, logs parameters/metrics/models)
c. Experiment Tracking & Logging
– CodeCarbon service (local CSV)
– MLflow Tracking Server (local filesystem or remote server)
d. Results Storage & Visualization
– CSVs, MLflow UI, Jupyter cell outputs
e. Demo Web Application
– Translator-app (Flask)
– External translation model or service (e.g., a local model in translator.py or third-party API)
– Deployment environment (Heroku via Procfile)

##### Relationships / Data Flow
1. Data Source → Notebook or Training Script
2. Training Script ↔ CodeCarbon (wraps run, emits carbon usage to emissions.csv)
3. Training Script ↔ MLflow (logs run metadata to MLflow store)
4. Outputs → local storage (models, CSVs) + MLflow server
5. Models (optionally) deployed into translator-app
6. Translator-app front end → translator.py → model/service → response

- Callouts for key frameworks (TensorFlow, Scikit-learn, CodeCarbon SDK, MLflow SDK, Flask)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources"
        direction TB
        PublicData["Public Datasets (CSV)"]:::storage
        TempCSV["Temp Data CSV"]:::storage
    end

    %% Experimentation
    subgraph "Experimentation" 
        direction TB
        NotebookEnv["Jupyter Notebook Environment"]:::compute
        Scripts["Training Scripts"]:::compute
    end

    %% Notebook files directory
    click NotebookEnv "https://github.com/iamrajatroy/data-science-lab/tree/main/notebook/"

    %% Scripts
    subgraph "Training Scripts"
        direction TB
        CCTrain["codecarbon/train.py"]:::compute
        CCAPI["codecarbon/api_run.py"]:::compute
        MLTrain["mlflow/train.py"]:::compute
    end

    click CCTrain "https://github.com/iamrajatroy/data-science-lab/blob/main/codecarbon/train.py"
    click CCAPI "https://github.com/iamrajatroy/data-science-lab/blob/main/codecarbon/api_run.py"
    click MLTrain "https://github.com/iamrajatroy/data-science-lab/blob/main/mlflow/train.py"

    %% Tracking Services
    subgraph "Tracking & Logging" 
        direction TB
        CodeCarbon["CodeCarbon Tracker"]:::tracking
        MLflowSvc["MLflow Tracking Server"]:::tracking
    end

    %% Storage
    subgraph "Results Storage"
        direction TB
        EmissionsCSV["emissions.csv"]:::storage
        MLflowStore["MLflow Artifact Store"]:::storage
    end

    click EmissionsCSV "https://github.com/iamrajatroy/data-science-lab/blob/main/codecarbon/emissions.csv"

    %% Deployment
    subgraph "Demo Web Application" 
        direction TB
        TranslatorApp["translator-app (Flask)"]:::web
        ExternalModel["External Model/Service"]:::compute
    end

    click TranslatorApp "https://github.com/iamrajatroy/data-science-lab/blob/main/translator-app/app.py"
    click TranslatorApp "https://github.com/iamrajatroy/data-science-lab/blob/main/translator-app/translator.py"
    click TranslatorApp "https://github.com/iamrajatroy/data-science-lab/tree/main/translator-app/Procfile"

    %% Data flow connections
    PublicData -->|ingest data| NotebookEnv
    PublicData -->|ingest data| CCTrain
    PublicData -->|ingest data| MLTrain
    TempCSV -->|ingest data| NotebookEnv

    NotebookEnv -->|runs experiments| CodeCarbon
    NotebookEnv -->|runs experiments| MLflowSvc

    CCTrain <-->|"wraps run"| CodeCarbon
    CCTrain -->|outputs emissions| EmissionsCSV
    CCTrain -->|logs params/metrics| MLflowSvc

    CCAPI <-->|"wraps run"| CodeCarbon

    MLTrain <-->|"wraps run"| MLflowSvc
    MLTrain -->|outputs emissions| EmissionsCSV
    MLTrain -->|logs params/metrics| MLflowSvc

    CodeCarbon -->|writes CSV| EmissionsCSV
    MLflowSvc -->|stores artifacts| MLflowStore

    EmissionsCSV -->|view results| NotebookEnv
    MLflowStore -->|view in UI| NotebookEnv

    MLflowStore -->|artifacts| TranslatorApp
    TranslatorApp -->|loads model| ExternalModel
    TranslatorApp -->|HTTP request| ExternalModel
    ExternalModel -->|response| TranslatorApp

    %% Legend
    subgraph Legend
        direction TB
        A[Compute Components]:::compute
        B[Tracking Services]:::tracking
        C[Data Storage]:::storage
        D[Web Application]:::web
    end

    classDef compute fill:#AED6F1,stroke:#1F618D,color:#1B2631;
    classDef tracking fill:#A9DFBF,stroke:#27AE60,color:#145A32;
    classDef storage fill:#FCF3CF,stroke:#F1C40F,color:#7D6608;
    classDef web fill:#F5CBA7,stroke:#D35400,color:#641E16;
```
