How 22amitdeo/dwdmproject works
This repository analyzes agricultural crop prices to characterize volatility and farmer income risk, and also provides a separate modal-price prediction app. The analysis workflow cleans raw records, then runs risk mining, OLAP summaries, visualizations, and insight generation over the cleaned dataset. A warehouse builder creates dimensional tables, while a separate training-and-serving path predicts prices from market details. The README documents these analysis capabilities; the sampled sources show the main orchestration and several analysis, warehouse, and prediction paths, but not every subsystem’s implementation.
Main entry points: cleaned_crop_prices.csv Cleaned Prices · run_analysis.py Analysis Runner · build_warehouse.py Warehouse Builder · model.pkl Trained Model
Components 26
Data Preparation
- Raw Prices
data/crop_prices.csv - Data Cleaner
preprocessing/clean_data.py - Cleaned Prices
data/cleaned_crop_prices.csv - Warehouse Builder
warehouse/build_warehouse.py - Time Dimension
warehouse/time_dimension.csv - Location Dimension
warehouse/location_dimension.csv - Crop Dimension
warehouse/crop_dimension.csv - Price Facts
warehouse/fact_table.csv
Risk Analysis
- Analysis Runner
run_analysis.py - Risk Mining
analysis/data_mining.py - OLAP Operations
analysis/olap_operations.py - Core Charts
analysis/visualizations.py - Advanced Charts
analysis/advanced_visualizations.py - Insight Generator
analysis/insights.py - Price Analysis
analysis/price_analysis.py
Price Prediction
- Model Trainer
prediction/train_model.py - Trained Model
prediction/model.pkl - Prediction App
prediction/app.py - Prediction Function
prediction/predict.py - Model Evaluator
prediction/evaluate_model.py
Analysis Outputs
- Mining Results
results/data_mining - OLAP Results
results/olap - Chart Results
results/visualizations - Advanced Results
results/advanced_visualizations - Insight Summary
results/insights_summary.txt
Other components
- App User
Connections 30
- Raw Prices → Data Cleaner: loads
preprocessing/clean_data.py - Data Cleaner → Cleaned Prices: writes
preprocessing/clean_data.py - Cleaned Prices → Warehouse Builder: reads
warehouse/build_warehouse.py - Warehouse Builder → Time Dimension: writes
warehouse/build_warehouse.py - Warehouse Builder → Location Dimension: writes
warehouse/build_warehouse.py - Warehouse Builder → Crop Dimension: writes
warehouse/build_warehouse.py - Warehouse Builder → Price Facts: writes
warehouse/build_warehouse.py - Analysis Runner → Risk Mining: runs
run_analysis.py - Analysis Runner → OLAP Operations: runs
run_analysis.py - Analysis Runner → Core Charts: runs
run_analysis.py - Analysis Runner → Advanced Charts: runs
run_analysis.py - Analysis Runner → Insight Generator: runs
run_analysis.py - Risk Mining → Cleaned Prices: reads
analysis/data_mining.py - Risk Mining → Mining Results: writes
analysis/data_mining.py - OLAP Operations → Cleaned Prices: reads
analysis/olap_operations.py - OLAP Operations → OLAP Results: writes
analysis/olap_operations.py - Core Charts → Cleaned Prices: reads
analysis/visualizations.py - Core Charts → Chart Results: writes
analysis/visualizations.py - Advanced Charts → Cleaned Prices: reads
analysis/advanced_visualizations.py - Advanced Charts → Advanced Results: writes
analysis/advanced_visualizations.py - Insight Generator → Cleaned Prices: reads
analysis/insights.py - Insight Generator → Insight Summary: writes
analysis/insights.py - Model Trainer → Cleaned Prices: reads
prediction/train_model.py - Model Trainer → Trained Model: saves
prediction/train_model.py - App User → Prediction App: enters details
prediction/app.py - Prediction App → Cleaned Prices: loads options
prediction/app.py - Prediction App → Trained Model: loads
prediction/app.py - Prediction Function → Trained Model: loads
prediction/predict.py - Model Evaluator → Trained Model: loads
prediction/evaluate_model.py - Model Evaluator → Cleaned Prices: reads
prediction/evaluate_model.py