How iamrajatroy/data-science-lab works
Project type and purpose
• This is a Data Science Experimentation repository (“Data-Science-Lab”) that contains:
• A variety of Jupyter notebooks implementing ML use‐cases (classification, computer vision explainability, etc.)
• Standalone Python scripts for training (“codecarbon/train.py”, “mlflow/train.py”)
• Emissions tracking via CodeCarbon
• Experiment tracking via MLflow
• A small “translator-app” Flask service as a demo web application
Top-level components (based on the file tree)
• /notebook: Jupyter notebooks (interactive experiments)
• /codecarbon: scripts that wrap training with carbon‐emission tracking
• /mlflow: scripts that wrap training with MLflow experiment tracking
• /translator-app: Flask web service (Procfile for Heroku, app.py, translator module)
• Data files: emissions.csv, temp/*.csv
Key architectural patterns
• Modular separation: notebooks vs. script-based pipelines
• Sidecar tracking services: CodeCarbon and MLflow integrate as wrappers around training
• A simple web front end for demonstration (translator-app)
Main components
a. Data Source(s)
– Public datasets (CSV on Kaggle or other open sources)
– Temporary extracted data under codecarbon/temp/
b. Experiment Orchestration
– Jupyter Notebook environment (interactive)
– Python training scripts:
• codecarbon/train.py (with CodeCarbon SDK, writes emissions.csv)
• mlflow/train.py (with MLflow SDK, logs parameters/metrics/models)
c. Experiment Tracking & Logging
– CodeCarbon service (local CSV)
– MLflow Tracking Server (local filesystem or remote server)
d. Results Storage & Visualization
– CSVs, MLflow UI, Jupyter cell outputs
e. Demo Web Application
– Translator-app (Flask)
– External translation model or service (e.g., a local model in translator.py or third-party API)
– Deployment environment (Heroku via Procfile)
Relationships / Data Flow
1. Data Source → Notebook or Training Script
2. Training Script ↔ CodeCarbon (wraps run, emits carbon usage to emissions.csv)
3. Training Script ↔ MLflow (logs run metadata to MLflow store)
4. Outputs → local storage (models, CSVs) + MLflow server
5. Models (optionally) deployed into translator-app
6. Translator-app front end → translator.py → model/service → response
• Callouts for key frameworks (TensorFlow, Scikit-learn, CodeCarbon SDK, MLflow SDK, Flask)