How mithun-j/mini-project works
This repository is a batch crop-yield ETL and exploratory-analysis application. A user runs the ETL entry point to read and audit the historical CSV, clean and validate rows, engineer agricultural features, then persist a prepared dataset and summary reports. A separate EDA entry point reads that dataset and coordinates structure, outlier, correlation, and visualization analyses. The README documents the broader reporting and visualization capabilities; the sampled code shows the main orchestrators and several outputs, but not every report-writing detail.
Main entry points: eda.py EDA Runner · pipeline.py ETL Runner · load.py Persist and Report · reports ETL and EDA Reports
Components 15
ETL Processing
- ETL Runner
pipeline.py - Extract and Audit
src/etl/extract.py - Clean and Validate
src/etl/clean.py - Engineer Features
src/etl/features.py - Persist and Report
src/etl/load.py
Exploratory Analysis
- EDA Runner
eda.py - Structure Analysis
src/eda/structure.py - Outlier Analysis
src/eda/outliers.py - Correlation Analysis
src/eda/correlations.py - EDA Visualizations
src/eda/visuals.py
Data Artifacts
- Raw Crop Data
raw_data/crop_yield.csv - Prepared Dataset
output/preprocessed_data.csv - ETL and EDA Reports
reports - EDA Figures
reports/figures
Other components
- User
Connections 17
- User → ETL Runner: runs
pipeline.py - ETL Runner → Extract and Audit: calls
pipeline.py - ETL Runner → Clean and Validate: calls
pipeline.py - ETL Runner → Engineer Features: calls
pipeline.py - ETL Runner → Persist and Report: calls
pipeline.py - Extract and Audit → Raw Crop Data: reads
src/etl/extract.py - Persist and Report → Prepared Dataset: writes
README.md - Persist and Report → ETL and EDA Reports: generates
src/etl/load.py - User → EDA Runner: runs
eda.py - EDA Runner → Prepared Dataset: reads
eda.py - EDA Runner → Structure Analysis: calls
eda.py - EDA Runner → Outlier Analysis: calls
eda.py - EDA Runner → Correlation Analysis: calls
eda.py - EDA Runner → EDA Visualizations: calls
eda.py - Outlier Analysis → ETL and EDA Reports: writes summary
src/eda/outliers.py - EDA Visualizations → EDA Figures: writes plots
README.md - EDA Runner → ETL and EDA Reports: writes summary
eda.py