# neuromancer-1984/mcx-commodity-dashboard architecture

> Architecture diagram of the GitHub repository neuromancer-1984/mcx-commodity-dashboard, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/neuromancer-1984/mcx-commodity-dashboard
- Repository: https://github.com/neuromancer-1984/mcx-commodity-dashboard

## Overview

The product is a daily commodity-price ETL pipeline and interactive Streamlit dashboard. Users select commodities and a date range, then explore MCX-equivalent INR prices, volatility, return correlations, and spread signals. The documented pipeline fetches global futures and USD/INR data, transforms it into derived prices and indicators, and stores both raw and derived records in PostgreSQL. Dashboard query and connection layers are shown separately. The scheduled workflow and database schema are documented, but their implementation details were not sampled.

Main entry points: [`dashboard/app.py`](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py) (Streamlit dashboard), [`dashboard/queries.py`](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/queries.py) (Dashboard queries), [`etl/extract_load.py`](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/etl/extract_load.py) (Extract and load), [`etl/transform.py`](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/etl/transform.py) (Transform and derive)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["ETL pipeline"]
  node_extract["Extract and load<br/>[extract_load.py]"]
  node_transform["Transform and derive<br/>[transform.py]"]
end

subgraph group_storage["Data storage"]
  node_schema["Database schema"]
  node_postgres[("PostgreSQL")]
end

subgraph group_dashboard["Dashboard"]
  node_app["Streamlit dashboard<br/>[app.py]"]
  node_db["DB connection<br/>[db.py]"]
  node_queries["Dashboard queries<br/>[queries.py]"]
  node_price["Price trends<br/>[app.py]"]
  node_volatility["Volatility view<br/>[app.py]"]
  node_correlation["Correlation view<br/>[app.py]"]
  node_spread["Spread analysis<br/>[app.py]"]
end

node_user(("Dashboard user"))
node_scheduler(("Daily scheduler"))
node_yahoo["Yahoo Finance"]

node_scheduler -->|"runs daily"| node_extract
node_scheduler -->|"runs daily"| node_transform
node_extract -->|"fetches prices"| node_yahoo
node_extract -->|"loads raw prices and FX"| node_postgres
node_extract -->|"reads commodity metadata"| node_postgres
node_transform -->|"reads raw prices, FX, metadata"| node_postgres
node_transform -->|"writes derived prices and returns"| node_postgres
node_schema -->|"defines tables"| node_postgres
node_user -->|"selects and explores"| node_app
node_app -->|"creates engine"| node_db
node_app -->|"requests dashboard data"| node_queries
node_queries -->|"queries data"| node_postgres
node_app -->|"renders"| node_price
node_app -->|"renders"| node_volatility
node_app -->|"renders"| node_correlation
node_app -->|"renders"| node_spread
node_queries -->|"reads price history and indicators"| node_postgres
node_queries -->|"reads returns for correlation"| node_postgres
node_queries -->|"reads prices for ratio"| node_postgres

click node_extract "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/etl/extract_load.py"
click node_transform "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/etl/transform.py"
click node_schema "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/sql/02_create_tables.sql"
click node_app "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py"
click node_db "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/db.py"
click node_queries "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/queries.py"
click node_price "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py"
click node_volatility "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py"
click node_correlation "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py"
click node_spread "https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_extract,node_transform,node_user toneBlue
class node_schema,node_postgres toneAmber
class node_app,node_db,node_queries,node_price,node_volatility,node_correlation,node_spread toneMint
class node_scheduler toneRose
class node_yahoo toneIndigo
```

## Components

### ETL pipeline

- [Extract and load](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/etl/extract_load.py): `etl/extract_load.py`
- [Transform and derive](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/etl/transform.py): `etl/transform.py`

### Data storage

- [Database schema](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/sql/02_create_tables.sql): `sql/02_create_tables.sql`
- **PostgreSQL**

### Dashboard

- [Streamlit dashboard](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py): `dashboard/app.py`
- [DB connection](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/db.py): `dashboard/db.py`
- [Dashboard queries](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/queries.py): `dashboard/queries.py`
- [Price trends](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py): `dashboard/app.py`
- [Volatility view](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py): `dashboard/app.py`
- [Correlation view](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py): `dashboard/app.py`
- [Spread analysis](https://github.com/neuromancer-1984/mcx-commodity-dashboard/blob/main/dashboard/app.py): `dashboard/app.py`

### Other components

- **Dashboard user**
- **Daily scheduler**
- **Yahoo Finance**

## Connections

- Daily scheduler → Extract and load: runs daily
- Daily scheduler → Transform and derive: runs daily
- Extract and load → Yahoo Finance: fetches prices
- Extract and load → PostgreSQL: loads raw prices and FX
- Extract and load → PostgreSQL: reads commodity metadata
- Transform and derive → PostgreSQL: reads raw prices, FX, metadata
- Transform and derive → PostgreSQL: writes derived prices and returns
- Database schema → PostgreSQL: defines tables
- Dashboard user → Streamlit dashboard: selects and explores
- Streamlit dashboard → DB connection: creates engine
- Streamlit dashboard → Dashboard queries: requests dashboard data
- Dashboard queries → PostgreSQL: queries data
- Streamlit dashboard → Price trends: renders
- Streamlit dashboard → Volatility view: renders
- Streamlit dashboard → Correlation view: renders
- Streamlit dashboard → Spread analysis: renders
- Dashboard queries → PostgreSQL: reads price history and indicators
- Dashboard queries → PostgreSQL: reads returns for correlation
- Dashboard queries → PostgreSQL: reads prices for ratio