How kumarpranavv/yieldwise works
Project type and purpose
• YieldWise is an end-to-end data-science application with a Python/Streamlit web UI.
• Its goal is to forecast Remaining Safe Storage Days (RSSD) for grain using a Gradient Boosting Regressor, provide what-if analyses, and display rich visualizations on a Streamlit dashboard.
File structure
• Top-level directories/files:
• data/ (train.csv, test.csv) → raw datasets
• models/ (grain_predictor_pipeline.joblib) → persisted ML pipeline
• output/ (metrics.json) → evaluation results
• src/ → all core scripts:
– data_preprocessing.py
– model_utils.py
– train.py (batch training pipeline)
– evaluate.py (standalone evaluation & plots)
– predict.py (inference logic)
• main.py → Streamlit application entrypoint
• requirements.txt, .gitignore
• Architectural pattern: a clear separation between batch pipelines (data prep, train, evaluate) and real-time/inference (Streamlit front end invoking predict).
Pulling in README insights
• Tech stack: Python 3.12, Pandas/NumPy, Scikit-learn, Streamlit, Plotly/Matplotlib/Seaborn
• Full DS lifecycle: synthetic data gen, feature engineering, train, eval, deploy as interactive tool.
a. Main components:
1. Data Storage
– raw CSV files in data/
2. Batch Pipeline
– Data Preprocessing (src/data_preprocessing.py)
– Model Training & Serialization (src/train.py + model_utils.py → models/)
– Model Evaluation (src/evaluate.py → output/metrics.json & evaluation plots)
3. Inference & Serving
– Inference Engine (src/predict.py loads joblib pipeline)
– Streamlit Web App (main.py)
4. Visualization Libraries
– Plotly/Matplotlib/Seaborn used by evaluate.py & Streamlit UI
5. External Dependencies
– Scikit-learn, Pandas, NumPy
b. Relationships & data flow:
• Raw data → Data Preprocessing → Features
• Features → Model Training → Trained Pipeline (.joblib)
• Trained Pipeline → Model Evaluation → metrics.json & plots
• Trained Pipeline + User Input → Predict → Streamlit UI
• Streamlit UI → Visualization Components → Browser
c. Architectural patterns & principles:
• Separation of concerns: batch vs interactive
• Pipeline modularity: each script addresses a single phase
• File-based artifact storage for reproducibility
d. Key technologies/frameworks:
• Streamlit (UI & app server)
• scikit-learn Pipeline (model + preprocessing)
• Plotly/Matplotlib/Seaborn (charts)
• joblib (model persistence)