How neuromancer-1984/mcx-commodity-dashboard works
The product is a daily commodity-price ETL pipeline and interactive Streamlit dashboard. Users select commodities and a date range, then explore MCX-equivalent INR prices, volatility, return correlations, and spread signals. The documented pipeline fetches global futures and USD/INR data, transforms it into derived prices and indicators, and stores both raw and derived records in PostgreSQL. Dashboard query and connection layers are shown separately. The scheduled workflow and database schema are documented, but their implementation details were not sampled.
Main entry points: app.py Streamlit dashboard · queries.py Dashboard queries · extract_load.py Extract and load · transform.py Transform and derive
Components 14
ETL pipeline
- Extract and load
etl/extract_load.py - Transform and derive
etl/transform.py
Data storage
- Database schema
sql/02_create_tables.sql - PostgreSQL
Dashboard
- Streamlit dashboard
dashboard/app.py - DB connection
dashboard/db.py - Dashboard queries
dashboard/queries.py - Price trends
dashboard/app.py - Volatility view
dashboard/app.py - Correlation view
dashboard/app.py - Spread analysis
dashboard/app.py
Other components
- Dashboard user
- Daily scheduler
- Yahoo Finance
Connections 19
- Daily scheduler → Extract and load: runs daily
- Daily scheduler → Transform and derive: runs daily
- Extract and load → Yahoo Finance: fetches prices
- Extract and load → PostgreSQL: loads raw prices and FX
- Extract and load → PostgreSQL: reads commodity metadata
- Transform and derive → PostgreSQL: reads raw prices, FX, metadata
- Transform and derive → PostgreSQL: writes derived prices and returns
- Database schema → PostgreSQL: defines tables
- Dashboard user → Streamlit dashboard: selects and explores
- Streamlit dashboard → DB connection: creates engine
- Streamlit dashboard → Dashboard queries: requests dashboard data
- Dashboard queries → PostgreSQL: queries data
- Streamlit dashboard → Price trends: renders
- Streamlit dashboard → Volatility view: renders
- Streamlit dashboard → Correlation view: renders
- Streamlit dashboard → Spread analysis: renders
- Dashboard queries → PostgreSQL: reads price history and indicators
- Dashboard queries → PostgreSQL: reads returns for correlation
- Dashboard queries → PostgreSQL: reads prices for ratio