# akshat-rai0/ntpc-nvvn-mp architecture

> Architecture diagram of the GitHub repository akshat-rai0/ntpc-nvvn-mp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/akshat-rai0/ntpc-nvvn-mp
- Repository: https://github.com/akshat-rai0/ntpc-nvvn-mp

## Overview

1. This repository is a Django-based forecasting system centered on a state-specific prediction pipeline. The runtime is split into three main Django apps: `states` for domain data, `predictions` for ingestion/inference/API, and `dashboard` for the human-facing UI. The project package `demand_predictor/` wires these apps together through Django settings, URL routing, and WSGI/ASGI entrypoints.

2. The core business object is “state.” Each state is configured in `config/states/*.yaml` with a short code, MERIT India demand URL, model path, timezone, fallback demand, and a set of cities with weather weights. That YAML drives registration in the database and determines which external source, model file, and weather locations are used for that state.

3. `states/` defines the persistent domain layer. It holds the Django models and admin registration for state metadata and time-series records such as live demand readings and prediction records. This is the main boundary between the business data model and everything else in the system.

4. `predictions/services/` is the engine room. `merit_client.py` fetches live demand from MERIT India, `weather.py` in `utils/` fetches weather from Open-Meteo, `time_features.py` and `lag_store.py` derive model inputs, `predictor.py` runs single-step inference, and `forecaster.py` handles day-ahead/multi-slot forecasts. `registry.py` loads YAML state definitions, while `csv_sync.py` mirrors database changes into CSV files.

5. The model boundary is explicit and file-based. Each state has a trained LightGBM model stored under `models/{state_code}/lgbm_final.txt` or, for a few states, a pickle artifact. Inference expects a fixed feature vector assembled from weather, calendar, and lagged demand inputs; the system is therefore tightly coupled to these model assets and their feature order.

6. Data flows from external services into the pipeline, then into storage and visualization. On each refresh, the app pulls live demand and weather, computes features, performs prediction, saves both actual and predicted values to SQLite, and syncs them into CSV logs under `data/states/{code}/`. Historical CSVs in `data/` are also used to backfill lag features and bootstrap new states.

7. The scheduling boundary is handled inside `predictions/scheduler.py` and app startup logic. A background job refreshes active states every five minutes when enabled, so the system can operate continuously without manual intervention. The same refresh workflow is also exposed as a management command for manual execution and operational control.

8. The public API sits in `predictions/api/views.py` and `predictions/urls.py`. It exposes state listing plus per-state “today,” “tomorrow,” “forecast,” and “history” endpoints, returning JSON for consumption by the dashboard or external systems. This API reads from the same persistence layer used by the scheduler, so it is not a separate data source.

9. The dashboard in `dashboard/` is a presentation layer over the API/database. `templates/dashboard/index.html` and the static Chart.js frontend render actual vs predicted demand, forecast views, and summary metrics. It depends on the prediction data being current, but it does not own the forecasting logic itself.

10. Persistence is dual-layered. SQLite (`db.sqlite3`) is the primary system-of-record for Django models, while CSV exports provide portability, backup, and offline analysis. The CSV layer is not authoritative; it is a synchronized projection of database state.

11. Operational tooling is first-class. Management commands under `predictions/management/commands/` support seeding state definitions, refreshing demand, and importing historical demand. The standalone `prediction.py` script is a lightweight CLI entrypoint for running a single-state prediction without the full server stack.

12. The repository also contains a substantial data/science workspace under `data/` with notebooks and intermediate CSVs. This area is separate from the application runtime: it supports model development, feature engineering, and data preparation, while the deployed app consumes only the curated model files, YAML configs, and runtime logs.

13. In architectural terms, the main dependency direction is: external services and historical data → feature engineering and model inference → SQLite/CSV persistence → API and dashboard. The key boundaries are between state configuration, prediction services, persistence, and presentation, with Django acting as the integration framework across all of them.

Main entry points: [`predictions`](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/predictions) (predictions app), [`db.sqlite3`](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/db.sqlite3) (SQLite DB), [`predictions/services/predictor.py`](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/predictor.py) (Predictor), [`demand_predictor`](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/demand_predictor) (demand_predictor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_project["Django project"]
  node_node_demand_predictor["demand_predictor<br/>django project"]
end

subgraph group_group_domain["Domain data"]
  node_node_states_app["states app<br/>domain app"]
  node_node_states_models[("State models<br/>django models<br/>[models.py]")]
  node_node_states_admin["State admin<br/>[admin.py]"]
  node_node_sqlite[("SQLite DB<br/>primary storage<br/>[db.sqlite3]")]
end

subgraph group_group_engine["Prediction engine"]
  node_node_predictions_app["predictions app<br/>service app"]
  node_node_registry["Registry<br/>state config loader<br/>[registry.py]"]
  node_node_merit_client["MERIT client<br/>external fetch<br/>[merit_client.py]"]
  node_node_weather["Weather fetch<br/>external fetch<br/>[weather.py]"]
  node_node_features["Features<br/>feature engineering<br/>[time_features.py]"]
  node_node_lag_store["Lag store<br/>historical cache<br/>[lag_store.py]"]
  node_node_predictor["Predictor<br/>single-step inference<br/>[predictor.py]"]
  node_node_forecaster["Forecaster<br/>multi-slot forecast<br/>[forecaster.py]"]
  node_node_csv_sync["CSV sync<br/>projection<br/>[csv_sync.py]"]
end

subgraph group_group_ui["Presentation"]
  node_node_api["API views<br/>json api<br/>[views.py]"]
  node_node_dashboard["Dashboard<br/>ui layer<br/>[index.html]"]
end

subgraph group_group_assets["Runtime assets"]
  node_node_config["State YAML<br/>config set"]
  node_node_models["Model files<br/>trained models"]
  node_node_data["Historical data<br/>data workspace"]
end

subgraph group_group_ops["Operations"]
  node_node_scheduler["Scheduler<br/>background job<br/>[scheduler.py]"]
  node_node_commands["Commands<br/>management cli"]
  node_node_prediction_cli["prediction.py<br/>cli entrypoint<br/>[prediction.py]"]
end

node_node_demand_predictor -->|"registers"| node_node_states_app
node_node_demand_predictor -->|"registers"| node_node_predictions_app
node_node_demand_predictor -->|"serves"| node_node_dashboard
node_node_states_app -->|"defines"| node_node_states_models
node_node_states_app -->|"publishes"| node_node_states_admin
node_node_states_models -->|"persists"| node_node_sqlite
node_node_predictions_app -->|"loads"| node_node_registry
node_node_registry -->|"reads"| node_node_config
node_node_predictions_app -->|"fetches"| node_node_merit_client
node_node_predictions_app -->|"fetches"| node_node_weather
node_node_predictions_app -->|"builds"| node_node_features
node_node_features -->|"uses"| node_node_lag_store
node_node_lag_store -->|"bootstraps"| node_node_data
node_node_predictor -->|"loads"| node_node_models
node_node_predictor -->|"consumes"| node_node_features
node_node_forecaster -->|"orchestrates"| node_node_predictor
node_node_predictions_app -->|"writes"| node_node_sqlite
node_node_csv_sync -->|"reads"| node_node_sqlite
node_node_csv_sync -->|"syncs"| node_node_data
node_node_scheduler -->|"triggers"| node_node_predictions_app
node_node_commands -->|"invokes"| node_node_predictions_app
node_node_prediction_cli -->|"runs"| node_node_predictor
node_node_api -->|"reads"| node_node_sqlite
node_node_api -->|"exposes"| node_node_predictions_app
node_node_dashboard -->|"consumes"| node_node_api
node_node_dashboard -->|"reads"| node_node_sqlite

click node_node_demand_predictor "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/demand_predictor"
click node_node_states_app "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/states"
click node_node_states_models "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/states/models.py"
click node_node_states_admin "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/states/admin.py"
click node_node_predictions_app "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/predictions"
click node_node_registry "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/registry.py"
click node_node_merit_client "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/merit_client.py"
click node_node_weather "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/utils/weather.py"
click node_node_features "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/utils/time_features.py"
click node_node_lag_store "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/utils/lag_store.py"
click node_node_predictor "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/predictor.py"
click node_node_forecaster "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/forecaster.py"
click node_node_csv_sync "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/csv_sync.py"
click node_node_api "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/api/views.py"
click node_node_dashboard "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/dashboard/templates/dashboard/index.html"
click node_node_scheduler "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/scheduler.py"
click node_node_commands "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/predictions/management/commands"
click node_node_prediction_cli "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/prediction.py"
click node_node_config "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/config/states"
click node_node_models "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/models"
click node_node_data "https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/data"
click node_node_sqlite "https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/db.sqlite3"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_demand_predictor toneBlue
class node_node_states_app,node_node_states_models,node_node_states_admin,node_node_sqlite toneAmber
class node_node_predictions_app,node_node_registry,node_node_merit_client,node_node_weather,node_node_features,node_node_lag_store,node_node_predictor,node_node_forecaster,node_node_csv_sync toneMint
class node_node_api,node_node_dashboard toneRose
class node_node_config,node_node_models,node_node_data toneIndigo
class node_node_scheduler,node_node_commands,node_node_prediction_cli toneTeal
```

## Components

### Django project

Framework wiring and entrypoints

- [demand_predictor](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/demand_predictor): django project · Settings, URL routing, ASGI/WSGI integration · `demand_predictor`

### Domain data

State metadata and persisted demand records

- [states app](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/states): domain app · State configuration and persistent demand/prediction data · `states`
- [State models](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/states/models.py): django models · State metadata and time-series records · `states/models.py`
- [State admin](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/states/admin.py): admin · Admin registration for domain objects · `states/admin.py`
- [SQLite DB](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/db.sqlite3): primary storage · System-of-record for persisted Django state and prediction records · `db.sqlite3`

### Prediction engine

State loading, feature building, inference, forecasting, and sync

- [predictions app](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/predictions): service app · Ingestion, inference, scheduling, API, and commands · `predictions`
- [Registry](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/registry.py): state config loader · Loads YAML state definitions and runtime metadata · `predictions/services/registry.py`
- [MERIT client](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/merit_client.py): external fetch · Fetches live demand from MERIT India · `predictions/services/merit_client.py`
- [Weather fetch](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/utils/weather.py): external fetch · Pulls weather inputs from Open-Meteo · `utils/weather.py`
- [Features](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/utils/time_features.py): feature engineering · Calendar, lag, and time features for inference · `utils/time_features.py`
- [Lag store](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/utils/lag_store.py): historical cache · Backfills and persists lagged demand inputs · `utils/lag_store.py`
- [Predictor](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/predictor.py): single-step inference · Runs model inference for one state and slot · `predictions/services/predictor.py`
- [Forecaster](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/forecaster.py): multi-slot forecast · Builds day-ahead and multi-step forecasts · `predictions/services/forecaster.py`
- [CSV sync](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/services/csv_sync.py): projection · Mirrors database changes into state CSV logs · `predictions/services/csv_sync.py`

### Presentation

API and dashboard surfaces

- [API views](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/api/views.py): json api · State listing, today, tomorrow, forecast, and history endpoints · `predictions/api/views.py`
- [Dashboard](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/dashboard/templates/dashboard/index.html): ui layer · Chart.js-based demand visualization and summaries · `dashboard/templates/dashboard/index.html`

### Runtime assets

Config, models, and historical data used by the app

- [State YAML](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/config/states): config set · Per-state sources, model paths, timezone, fallback demand, and city weights · `config/states`
- [Model files](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/models): trained models · State-specific LightGBM and some pickle artifacts · `models`
- [Historical data](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/data): data workspace · CSV logs and notebooks for training, backfill, and analysis · `data`

### Operations

Scheduling and manual execution entrypoints

- [Scheduler](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/predictions/scheduler.py): background job · Periodic refresh of active states · `predictions/scheduler.py`
- [Commands](https://github.com/akshat-rai0/ntpc-nvvn-mp/tree/main/predictions/management/commands): management cli · Seed, refresh, and import operational commands · `predictions/management/commands`
- [prediction.py](https://github.com/akshat-rai0/ntpc-nvvn-mp/blob/main/prediction.py): cli entrypoint · Standalone single-state prediction runner · `prediction.py`

## Connections

- demand_predictor → states app: registers · Project settings and routing include the states app
- demand_predictor → predictions app: registers · Project settings and routing include the predictions app
- demand_predictor → Dashboard: serves · Project URLs expose the dashboard presentation layer
- states app → State models: defines · Persistent state and time-series data model
- states app → State admin: publishes · Registers domain objects in Django admin
- State models → SQLite DB: persists · State metadata and readings are stored in SQLite
- predictions app → Registry: loads · Bootstraps state definitions from YAML configuration
- Registry → State YAML: reads · Parses per-state configuration files
- predictions app → MERIT client: fetches · Pulls live demand from MERIT India
- predictions app → Weather fetch: fetches · Retrieves weather features for configured cities
- predictions app → Features: builds · Derives feature vectors for model input
- Features → Lag store: uses · Consumes lagged demand history for feature creation
- Lag store → Historical data: bootstraps · Uses historical CSVs as backfill source
- Predictor → Model files: loads · Loads the per-state trained artifact
- Predictor → Features: consumes · Uses engineered feature vectors for inference
- Forecaster → Predictor: orchestrates · Builds repeated predictions across future slots
- predictions app → SQLite DB: writes · Stores actuals and predictions in the primary database
- CSV sync → SQLite DB: reads · Projects database state into CSV logs
- CSV sync → Historical data: syncs · Maintains per-state CSV outputs under data/states
- Scheduler → predictions app: triggers · Background refresh workflow runs the service pipeline
- Commands → predictions app: invokes · Management commands reuse the same refresh and import flows
- prediction.py → Predictor: runs · Standalone CLI invokes single-state prediction
- API views → SQLite DB: reads · API reads persisted demand and forecast records
- API views → predictions app: exposes · JSON endpoints surface prediction data to consumers
- Dashboard → API views: consumes · Frontend renders charts from API data
- Dashboard → SQLite DB: reads · Dashboard can also read persisted summaries directly