How krish-kunjadiya/supplychain_forecasting_and_optimization works
1. This repository is an offline, notebook-orchestrated analytics pipeline rather than a deployed application. The architecture is a linear sequence of 8 Jupyter notebooks that transform raw supply-chain data into forecasts, inventory policies, and a stakeholder-facing dashboard.
2. The primary boundary is the input dataset in data/supply_chain_dataset1.csv, which is intentionally excluded from version control. Everything else in the repo is either a derived artifact, a trained model, a metrics file, or a visualization output.
3. Notebook 01 performs data loading, cleaning, validation, missing-value handling, and type/range checks. Its output, supply_chain_validated.csv, becomes the trusted base dataset for the rest of the pipeline.
4. Notebook 02 is the exploratory analysis layer. It reshapes the validated data into supply_chain_eda_ready.csv and is the point where descriptive statistics, seasonality inspection, and feature relationships are established before modeling.
5. Notebook 03 is the feature engineering boundary between analysis and modeling. It produces two model-specific datasets: features_lstm.csv for sequence modeling and features_xgboost.csv for tabular learning, plus feature_registry.json to document the engineered feature set and transformation metadata.
6. The forecasting layer splits into two parallel model tracks. Notebook 04 trains the LSTM time-series model and persists TensorFlow/Keras artifacts (lstm_best_model.keras, lstm_final_model.keras) plus evaluation outputs. Notebook 05 trains the XGBoost model, saving tuned parameters, the final model, feature importance outputs, and test predictions. This is the main modeling fork in the architecture.
7. Notebook 06 is the ensemble integration layer. It consumes LSTM and XGBoost predictions and compares multiple blending strategies: simple average, weighted blend, meta-learner stacking, and per-SKU dynamic weighting. The outputs include hybrid_predictions.csv, hybrid_metrics.json, all_model_comparison.csv, and sku_dynamic_weights.csv.
8. Notebook 07 converts forecasts into operational decisions. It uses the selected/hybrid demand forecasts to calculate reorder points, EOQ, and safety stock, then simulates policy performance and quantifies business impact. This stage produces inventory_policy.csv, policy_comparison.csv, simulation_results.csv, and business_value.json.
9. Notebook 08 is the presentation and reporting layer. It aggregates model metrics, inventory outcomes, and business value into seven static chart images and a self-contained supply_chain_dashboard.html report. This is the final consumable artifact for stakeholders.
10. The repo’s persisted artifacts form a clear data lineage: raw data → validated data → EDA-ready data → engineered features → model outputs → ensemble outputs → inventory policy → dashboard/report. The JSON, CSV, Keras, PKL, and HTML files are all checkpoints in that lineage rather than independent products.
11. The main technologies that materially shape the architecture are Python notebooks, pandas/numpy for data processing, scikit-learn for preprocessing and metrics, TensorFlow/Keras for LSTM training, XGBoost for gradient-boosted forecasting, and matplotlib/seaborn for reporting. The optional LightGBM meta-learner appears only within the hybrid notebook.
12. Conceptually, the system has three important layers: data preparation, predictive modeling, and decision optimization. The final business value comes from connecting forecast accuracy to inventory policy simulation, not from forecasting alone.
Main entry points: 08_results_dashboard.ipynb 08 Dashboard · 03_feature_engineering.ipynb 03 Features · 04_lstm_model.ipynb 04 LSTM · 05_xgboost_model.ipynb 05 XGBoost
Components 23
Data prep
Raw data boundary, validation, EDA, and feature engineering.
- Raw data · Excluded source CSV boundary for the pipeline.
- 01 Validate · Loads, cleans, validates, and standardizes the source data.
01_data_loading_and_validation.ipynb - Validated data · Cleaned base dataset used by the rest of the pipeline.
- 02 EDA · Exploratory analysis, descriptive statistics, and feature inspection.
02_eda.ipynb - EDA-ready data · Reshaped dataset prepared for feature engineering.
- 03 Features · Builds model-specific features and records feature metadata.
03_feature_engineering.ipynb - LSTM features · Engineered sequence features for time-series forecasting.
- XGB features · Engineered tabular features for gradient-boosted forecasting.
- Feature registry · Feature definitions and transformation metadata.
feature_registry.json
Forecasting
Parallel model tracks and ensemble integration.
- 04 LSTM · Trains the Keras time-series model and evaluates forecasts.
04_lstm_model.ipynb - LSTM model · Persisted best and final LSTM models.
lstm_final_model.keras - LSTM metrics · Model evaluation outputs for the LSTM track.
lstm_metrics.json - LSTM scaler · Scaling artifact used with the LSTM workflow.
lstm_scaler.pkl - 05 XGBoost · Trains the XGBoost forecasting model and captures tuning results.
05_xgboost_model.ipynb - XGB model · Final trained XGBoost predictor.
xgb_final_model.pkl - Best params · Selected XGBoost hyperparameters.
xgb_best_params.json - XGB metrics · XGBoost tuning, importance, and test-set metrics.
xgb_metrics.json - 06 Hybrid · Blends model forecasts with averaging, stacking, and dynamic weighting.
06_hybrid_model.ipynb - Hybrid outputs · Blended forecasts, comparisons, and meta-model artifacts.
hybrid_metrics.json
Optimization
Forecasts turned into inventory decisions and business impact.
- 07 Inventory · Turns forecasts into reorder points, EOQ, safety stock, and simulations.
07_inventory_optimization.ipynb - Policy outputs · Inventory policy, comparisons, simulations, and business value.
business_value.json
Reporting
Dashboard assembly and stakeholder-facing outputs.
- 08 Dashboard · Assembles charts and the final stakeholder report.
08_results_dashboard.ipynb - Dashboard · Self-contained HTML summary of results and charts.
supply_chain_dashboard.html
Connections 26
- Raw data → 01 Validate: source data · Notebook 01 ingests the raw CSV boundary.
- 01 Validate → Validated data: writes · Validated dataset becomes the trusted base.
- Validated data → 02 EDA: input · EDA starts from the validated dataset.
- 02 EDA → EDA-ready data: writes · Produces EDA-ready reshaped data.
- EDA-ready data → 03 Features: input · Feature engineering builds on analysis output.
- 03 Features → LSTM features: writes · Creates sequence features for LSTM training.
- 03 Features → XGB features: writes · Creates tabular features for XGBoost training.
- 03 Features → Feature registry: documents · Records engineered feature definitions and lineage.
- LSTM features → 04 LSTM: input · LSTM training consumes sequence features.
- 04 LSTM → LSTM model: persists · Stores best and final Keras model artifacts.
- 04 LSTM → LSTM metrics: evaluates · Outputs forecast accuracy and diagnostics.
- 04 LSTM → LSTM scaler: persists · Stores scaling state used by the LSTM workflow.
- XGB features → 05 XGBoost: input · XGBoost training consumes tabular features.
- 05 XGBoost → Best params: writes · Persists the tuned parameter set.
- 05 XGBoost → XGB model: persists · Stores the final boosted-tree model.
- 05 XGBoost → XGB metrics: evaluates · Outputs tuning and test-set metrics.
- LSTM model → 06 Hybrid: forecast input · Hybrid layer consumes LSTM predictions.
- XGB model → 06 Hybrid: forecast input · Hybrid layer consumes XGBoost predictions.
- 06 Hybrid → Hybrid outputs: writes · Produces blended forecasts and ensemble comparisons.
- Hybrid outputs → 07 Inventory: selected forecast · Inventory optimization uses the chosen hybrid demand signal.
- 07 Inventory → Policy outputs: writes · Generates policy, simulation, and business impact artifacts.
- Policy outputs → 08 Dashboard: input · Reporting aggregates inventory outcomes and value.
- LSTM metrics → 08 Dashboard: input · Dashboard includes model evaluation summaries.
- XGB metrics → 08 Dashboard: input · Dashboard includes model evaluation summaries.
- Hybrid outputs → 08 Dashboard: input · Dashboard includes ensemble comparison results.
- 08 Dashboard → Dashboard: writes · Final stakeholder-facing HTML report.