# d3vil13/btp_targeted_enforcement_intelligence architecture

> Architecture diagram of the GitHub repository d3vil13/btp_targeted_enforcement_intelligence, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/d3vil13/btp_targeted_enforcement_intelligence
- Repository: https://github.com/d3vil13/btp_targeted_enforcement_intelligence

## Overview

1. **Repository shape**
   - This is primarily a **decision-support platform** for Bengaluru traffic enforcement, not a generic CRUD app.
   - It has two major runtime surfaces: an **offline analytics/modeling pipeline** and an **online full-stack web app**.
   - The online app reads **precomputed artifacts** from the repo root; it does not depend on a live database.

2. **Core data source and trust boundary**
   - The system starts from BTP’s **stationary AI edge camera violation dataset**.
   - Raw records flow through cleaning, deduplication, feature engineering, and human-audited fields.
   - Important boundary: the app uses **auditor-corrected outputs** like `updated_vehicle_number` and `updated_vehicle_type`, not raw OCR values.

3. **Offline data engineering pipeline**
   - The notebooks and `scripts/` folder form a batch pipeline that turns raw CSV data into structured analysis tables and geospatial assets.
   - Key outputs include:
     - `hotspots_ranked.csv`
     - `station_performance.csv`
     - `daily_violation_counts.csv`
     - `violations_sample.csv`
     - `camera_coverage.geojson`, `coverage_gaps.geojson`, `station_boundaries.geojson`, `hotspot_hulls.geojson`
   - This layer is the source of truth for almost all dashboard and map content.

4. **Spatial intelligence subsystem**
   - Hotspot detection is based on **H3 hexagonal binning** and geospatial clustering.
   - Camera coverage and station boundary files provide the spatial context for maps and gap analysis.
   - The app’s geospatial views are driven by these static GeoJSON artifacts rather than live GIS services.

5. **Forecasting subsystem**
   - Forecast generation is separate from the web app and lives in `forecast/` plus `models/branch_a/`.
   - The final forecast artifact is an **ensemble** combining a day-of-week baseline and a global LSTM, with per-station Prophet models retained as artifacts.
   - Outputs are materialized as:
     - `forecasts_branch_a_station.csv`
     - `forecasts_branch_a.csv`
   - These feed the forecast page and planning views.

6. **Backend service boundary**
   - `webapp/backend` is a **FastAPI** service that exposes the data to the frontend.
   - `app/services/data_service.py` is the main read/compute layer: it loads the precomputed CSV/GeoJSON/model-derived files and serves response-ready data.
   - `app/api/routes.py` defines the API surface consumed by the UI.
   - `app/models/schemas.py` defines the response contracts, so the backend acts as a schema boundary between static analytics data and the UI.

7. **Frontend application boundary**
   - `webapp/frontend` is a **React + TypeScript + Vite** SPA.
   - It communicates with the backend through `src/services/api.ts`.
   - The frontend is organized by feature pages, not by generic widgets, with pages for Dashboard, Enforcement Map, Priority, Stations, Forecast, Simulator, Analytics, and Settings.

8. **Primary user-facing data flows**
   - **Dashboard flow:** aggregated counts and summaries from processed CSVs → KPI cards and trend charts.
   - **Map flow:** hotspot rankings + camera coverage + station boundaries + sampled violations → Leaflet visualization.
   - **Priority flow:** hotspot metrics + configurable ROI weights → ranked enforcement actions.
   - **Station flow:** station benchmark metrics → performance table and gap warnings.
   - **Forecast flow:** 7-day predictions → heatmap and station detail chart.
   - **Simulator flow:** forecast/priority data + greedy assignment logic on backend → officer deployment recommendations.

9. **Business logic in the backend**
   - The backend is not just a file server; it contains operational logic for:
     - ranking zones by congestion impact,
     - computing simulator assignments,
     - preparing forecast and station summaries,
     - applying tunable weights for priority scoring.
   - That makes `data_service.py` the central orchestration point for derived views.

10. **Configuration and deployment model**
   - The app is intended to be run locally as two independent processes:
     - Uvicorn for FastAPI on port 8000
     - Vite dev server on port 5173
   - The frontend proxies `/api` requests to the backend, which keeps the browser-facing app decoupled from backend hosting details.

11. **Static artifact dependency model**
   - A notable architectural choice is that the repo includes **all generated assets** in source control.
   - The runtime app depends on local files in the repository root rather than on a database, message queue, or object store.
   - This makes the system reproducible and demo-friendly, but also means updates require rerunning the offline pipeline.

12. **Optional enrichment path**
   - `scripts/01_fetch_mapmyindia_traffic.py` and `scripts/02_fetch_osm_roadtype.py` can enrich hotspots with external traffic/road-type context.
   - These are optional inputs to the analytics pipeline, not required for the core app.
   - The architecture therefore supports both a fully offline mode and an enriched mode with external data.

13. **Historical/archived subsystems**
   - `_archive/` contains earlier prototypes, older models, notebooks, and generated outputs.
   - These are not part of the main runtime path, but they show that the current architecture is the result of iterative experimentation with prior Streamlit, SARIMA, autoencoder, and ensemble approaches.

14. **What an architecture diagram should emphasize**
   - At the center: **precomputed analytical artifacts** produced by notebooks/scripts.
   - On the right: **FastAPI backend** as the API and data-preparation boundary.
   - On the far right: **React frontend** as the presentation layer.
   - On the left/bottom: **raw camera violation dataset** feeding the batch pipeline.
   - Around the batch pipeline: **spatial analysis**, **forecasting**, and **ROI/deployment optimization** as separate derived subsystems.

Main entry points: [`scripts/01_fetch_mapmyindia_traffic.py`](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/01_fetch_mapmyindia_traffic.py) (Enrichment scripts), [`webapp/backend/app/services/data_service.py`](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/services/data_service.py) (Data service), [`webapp/backend/app/api/routes.py`](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/api/routes.py) (API routes), [`scripts/05_h3_hotspots.py`](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/05_h3_hotspots.py) (Hotspot build)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_offline_pipeline["Offline pipeline"]
  node_raw_dataset[("Camera data<br/>violation dataset")]
  node_notebooks["Notebooks<br/>feature engineering"]
  node_enrichment_scripts["Enrichment scripts<br/>batch ETL"]
  node_coverage_builder["Coverage build<br/>spatial ETL"]
  node_hotspot_builder["Hotspot build<br/>H3 clustering<br/>[05_h3_hotspots.py]"]
  node_ensemble_builder["Forecast build<br/>ensemble training"]
end

subgraph group_spatial_forecasting["Spatial + forecast assets"]
  node_artifact_tables[("Analytic tables<br/>csv outputs")]
  node_geo_assets[("Geo assets<br/>geojson outputs")]
  node_forecast_artifacts[("Forecasts<br/>prediction outputs")]
  node_model_store[("Model store<br/>saved models<br/>[lstm.keras]")]
  node_roi_config["ROI weights<br/>policy config<br/>[roi_weights.json]"]
end

subgraph group_backend["FastAPI backend"]
  node_data_service["Data service<br/>read/compute layer<br/>[data_service.py]"]
  node_api_routes["API routes<br/>REST surface<br/>[routes.py]"]
  node_schemas["Schemas<br/>response contracts<br/>[schemas.py]"]
  node_backend_main["Backend app<br/>FastAPI entrypoint<br/>[main.py]"]
end

subgraph group_frontend["React frontend"]
  node_frontend_app["SPA shell<br/>React app<br/>[App.tsx]"]
  node_api_client["API client<br/>HTTP client<br/>[api.ts]"]
  node_feature_pages["Feature pages<br/>view layer<br/>[Dashboard.tsx]"]
  node_shared_ui["Shared UI<br/>components<br/>[Layout.tsx]"]
end

subgraph group_legacy["Archived work"]
  node_archive["Archive<br/>legacy prototypes<br/>[app.py]"]
end

node_raw_dataset -->|"inspect/clean"| node_notebooks
node_notebooks -->|"derive pipeline"| node_enrichment_scripts
node_enrichment_scripts -->|"build coverage"| node_coverage_builder
node_enrichment_scripts -->|"rank hotspots"| node_hotspot_builder
node_enrichment_scripts -->|"prepare forecast"| node_ensemble_builder
node_hotspot_builder -->|"write tables"| node_artifact_tables
node_coverage_builder -->|"write geojson"| node_geo_assets
node_hotspot_builder -->|"shape maps"| node_geo_assets
node_ensemble_builder -->|"write forecasts"| node_forecast_artifacts
node_model_store -->|"serve models"| node_ensemble_builder
node_roi_config -->|"score priority"| node_artifact_tables
node_artifact_tables -->|"load summaries"| node_data_service
node_geo_assets -->|"load maps"| node_data_service
node_forecast_artifacts -->|"load forecasts"| node_data_service
node_data_service -->|"serve endpoints"| node_api_routes
node_schemas -->|"shape responses"| node_api_routes
node_backend_main -->|"mount API"| node_api_routes
node_api_routes -->|"/api"| node_api_client
node_api_client -->|"fetch view data"| node_feature_pages
node_feature_pages -->|"compose UI"| node_shared_ui
node_archive -.->|"historical context"| node_notebooks

click node_notebooks "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/dataset_feature_engineering.ipynb"
click node_enrichment_scripts "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/01_fetch_mapmyindia_traffic.py"
click node_coverage_builder "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/03_build_camera_coverage.py"
click node_hotspot_builder "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/05_h3_hotspots.py"
click node_ensemble_builder "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/forecast/branch_a_ensemble.py"
click node_artifact_tables "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/hotspots_ranked.csv"
click node_geo_assets "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/camera_coverage.geojson"
click node_forecast_artifacts "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/forecasts_branch_a.csv"
click node_model_store "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/models/branch_a/lstm.keras"
click node_roi_config "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/roi_weights.json"
click node_data_service "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/services/data_service.py"
click node_api_routes "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/api/routes.py"
click node_schemas "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/models/schemas.py"
click node_backend_main "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/main.py"
click node_frontend_app "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/App.tsx"
click node_api_client "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/services/api.ts"
click node_feature_pages "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/pages/Dashboard.tsx"
click node_shared_ui "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/components/Layout.tsx"
click node_archive "https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/_archive/old_streamlit_app/app/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_raw_dataset,node_notebooks,node_enrichment_scripts,node_coverage_builder,node_hotspot_builder,node_ensemble_builder toneBlue
class node_artifact_tables,node_geo_assets,node_forecast_artifacts,node_model_store,node_roi_config toneAmber
class node_data_service,node_api_routes,node_schemas,node_backend_main toneMint
class node_frontend_app,node_api_client,node_feature_pages,node_shared_ui toneRose
class node_archive toneIndigo
```

## Components

### Offline pipeline

Batch notebooks and scripts that transform raw violation data into static analytical artifacts.

- **Camera data**: violation dataset · Stationary AI edge-camera violation records; the trusted source feeding the batch pipeline.
- [Notebooks](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/dataset_feature_engineering.ipynb): feature engineering · Exploration, cleaning, auditing, and pipeline development notebooks. · `dataset_feature_engineering.ipynb`
- [Enrichment scripts](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/01_fetch_mapmyindia_traffic.py): batch ETL · Optional external enrichment and core pipeline steps for map, coverage, hotspot, and forecast preparation. · `scripts/01_fetch_mapmyindia_traffic.py`
- [Coverage build](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/03_build_camera_coverage.py): spatial ETL · Builds camera coverage and station boundary geospatial layers. · `scripts/03_build_camera_coverage.py`
- [Hotspot build](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/scripts/05_h3_hotspots.py): H3 clustering · Ranks and clusters violations into hotspot outputs using hex/bin spatial logic. · `scripts/05_h3_hotspots.py`
- [Forecast build](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/forecast/branch_a_ensemble.py): ensemble training · Reblends and materializes the branch A forecast artifacts from model components. · `forecast/branch_a_ensemble.py`

### Spatial + forecast assets

Geospatial and prediction outputs materialized in the repository root for downstream consumption.

- [Analytic tables](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/hotspots_ranked.csv): csv outputs · Precomputed summary tables consumed by dashboard, station, priority, and simulator views. · `hotspots_ranked.csv`
- [Geo assets](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/camera_coverage.geojson): geojson outputs · Static geospatial layers for maps, coverage, and gap analysis. · `camera_coverage.geojson`
- [Forecasts](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/forecasts_branch_a.csv): prediction outputs · 7-day predictions and station-level forecast outputs used by planning views. · `forecasts_branch_a.csv`
- [Model store](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/models/branch_a/lstm.keras): saved models · Persisted LSTM, Prophet, and ensemble-weight artifacts supporting forecast generation. · `models/branch_a/lstm.keras`
- [ROI weights](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/roi_weights.json): policy config · Tunable scoring weights used by priority ranking and deployment logic. · `roi_weights.json`

### FastAPI backend

Read/compute API boundary that loads static artifacts and serves response-ready data.

- [Data service](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/services/data_service.py): read/compute layer · Central orchestration layer that loads artifacts, computes summaries, and prepares API payloads. · `webapp/backend/app/services/data_service.py`
- [API routes](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/api/routes.py): REST surface · FastAPI route definitions consumed by the frontend. · `webapp/backend/app/api/routes.py`
- [Schemas](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/app/models/schemas.py): response contracts · Backend response models defining the boundary between static analytics data and the UI. · `webapp/backend/app/models/schemas.py`
- [Backend app](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/backend/main.py): FastAPI entrypoint · Application bootstrap and server entrypoint for the backend process. · `webapp/backend/main.py`

### React frontend

Vite SPA that renders feature pages and talks to the backend API.

- [SPA shell](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/App.tsx): React app · Vite application shell and top-level routing/layout. · `webapp/frontend/src/App.tsx`
- [API client](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/services/api.ts): HTTP client · Frontend service layer for backend communication. · `webapp/frontend/src/services/api.ts`
- [Feature pages](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/pages/Dashboard.tsx): view layer · Dashboard, map, priority, stations, forecast, simulator, analytics, and settings pages. · `webapp/frontend/src/pages/Dashboard.tsx`
- [Shared UI](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/webapp/frontend/src/components/Layout.tsx): components · Reusable layout, cards, tables, and navigation components. · `webapp/frontend/src/components/Layout.tsx`

### Archived work

Older notebooks, models, and prototype apps kept for reference.

- [Archive](https://github.com/d3vil13/btp_targeted_enforcement_intelligence/blob/main/_archive/old_streamlit_app/app/app.py): legacy prototypes · Older notebooks, models, and prototype apps retained for historical context. · `_archive/old_streamlit_app/app/app.py`

## Connections

- Camera data → Notebooks: inspect/clean · Exploration and feature engineering start from the raw violation records.
- Notebooks → Enrichment scripts: derive pipeline · Notebook work is operationalized into batch scripts and reproducible steps.
- Enrichment scripts → Coverage build: build coverage · Spatial preprocessing produces camera coverage and boundary layers.
- Enrichment scripts → Hotspot build: rank hotspots · Violations are cleaned, deduplicated, and aggregated into hotspot metrics.
- Enrichment scripts → Forecast build: prepare forecast · Processed features feed the forecast branch build and reblending steps.
- Hotspot build → Analytic tables: write tables · Hotspot analytics are materialized into ranked and station-level CSV outputs.
- Coverage build → Geo assets: write geojson · Coverage and station geometry outputs are persisted as static map layers.
- Hotspot build → Geo assets: shape maps · Spatial clustering contributes hotspot hulls and map overlays.
- Forecast build → Forecasts: write forecasts · Ensemble outputs are materialized for the forecast page and planning views.
- Model store → Forecast build: serve models · Saved LSTM/Prophet artifacts are inputs to forecast generation.
- ROI weights → Analytic tables: score priority · Priority and deployment rankings use configurable ROI weights.
- Analytic tables → Data service: load summaries · The backend reads summary tables to compute response-ready views.
- Geo assets → Data service: load maps · Static geospatial files are read by the backend for map responses.
- Forecasts → Data service: load forecasts · Forecast payloads are served from precomputed branch A outputs.
- Data service → API routes: serve endpoints · Central service functions back the FastAPI route handlers.
- Schemas → API routes: shape responses · Route handlers return schema-defined payloads to the frontend.
- Backend app → API routes: mount API · The FastAPI app wires the route module into the server process.
- API routes → API client: /api · The SPA consumes backend endpoints through its typed API client.
- API client → Feature pages: fetch view data · Feature pages request dashboard, map, forecast, and simulator data from the backend.
- Feature pages → Shared UI: compose UI · Pages are assembled from shared layout and component primitives.
- Archive → Notebooks: historical context · Archived prototypes show earlier iterations of the same analytical workflow.