How svmtejas84/capstone works
1. System shape
This is a multi-component Python + TypeScript monorepo for “toxicity-aware navigation.” The core architecture is a pipeline: external data sources feed ingestion, ingestion publishes live state to Redis, the graph/risk layer computes exposure-aware edge costs, routing and matching produce candidate corridors, a FastAPI service exposes results, and a Vite/React frontend visualizes them.
2. External data boundary
The system depends on three live data families: Open-Meteo weather, Open-Meteo air quality, and AQICN/CPCB station observations. Road topology comes from OpenStreetMap via OSMnx. These sources are treated as upstream inputs and are normalized into local Parquet datasets under data/raw/ and derived graph artifacts under data/.
3. Ingestion and data fusion layer
The ingestion/ package is the live and batch data boundary. It contains fetchers, bias correction, data fusion, nowcasting, traffic-spike handling, and a Redis publisher. This layer turns heterogeneous source data into a unified stream/state model for downstream consumers. The scripts in scripts/ complement this by pulling archives, repairing gaps, and syncing startup data.
4. State transport and runtime backbone
Redis is the main runtime bus. The README indicates weather, air quality, and sensor streams are published there as near-real-time topics/streams. This makes the system event-driven: ingestion writes state, and the API/routing side reads the latest state without directly coupling to source APIs.
5. Physics and graph learning layer
The gnn/ package is the toxicity estimation engine. It combines physics-informed plume modeling, wind/edge weighting, and a trained PyTorch model (gnn/model_weights/pi_gnn_v1.pt) to estimate edge-level pollutant exposure. Supporting modules like graph_builder.py, edge_weights.py, plume_physics.py, and wake_predictor.py define the model’s core data transformations and priors.
6. Graph artifact boundary
The road graph is a distinct artifact produced from OSM data and stored as projected, city-specific graph files. The README references a UTM Zone 43N Bangalore graph and tensorized graph outputs used for GNN training. This separates static infrastructure geometry from live environmental state, which is important for diagramming the data flow.
7. Routing layer
The router/ package converts toxicity estimates into path costs and routes. It computes inhaled-dose-based edge costs using concentration, respiratory minute volume, and travel time, then runs A* search over the road graph. There is an optional Rust acceleration path in router/rust_astar/, which is a performance boundary rather than a functional one.
8. API service boundary
router/api/main.py exposes the backend as a FastAPI service. The README names endpoints for health, plume, route, and audit lookup. This service is the main synchronous interface between the model/routing stack and the frontend, and it also reads from Redis state and graph/model artifacts.
9. Matching and equilibrium layer
The matcher/ package is a separate decision layer that allocates commuters to corridors using Gale-Shapley stable matching, quotas, segment preferences, and equilibrium checks. This sits above raw pathfinding: routing produces candidate corridors, while matching distributes users across them to avoid congestion or unsafe concentration on a single path.
10. Auditability and reproducibility
router/stake_audit.py and the /audit/{hash} endpoint provide an immutable decision trace. The audit includes request context, model version, graph snapshot, stream state, and timestamps. Architecturally, this is a compliance/reproducibility boundary around route generation.
11. Frontend presentation layer
frontend/ is a separate Vite + React + TypeScript application. Components like PlumeMap, RoutePanel, EquilibriumViz, AgentSim, and AuditViewer suggest the UI is organized around map visualization, route comparison, simulation, and audit inspection. It consumes the FastAPI backend via frontend/src/api/client.ts and stream-like hooks for plume/route updates.
12. Shared utilities and configuration
shared/ holds cross-cutting concerns used by multiple backend packages: config, geo utilities, logging, physics config, Redis client, and schemas. This package is the common contract layer that keeps ingestion, routing, matcher, and API aligned on data shapes and deployment settings.
13. Operational and deployment boundaries
The repo includes docker-compose.yml, GitHub Actions CI/deploy workflows, and startup scripts like scripts/quickstart.sh and scripts/sync_on_entry.py. That indicates the system is designed to run as a small set of cooperating services rather than a single monolith: Redis + ingestion worker + FastAPI backend + frontend, with optional Rust acceleration and offline batch prep.
14. What to emphasize in an architecture diagram
The most important boundaries are: external data sources vs. local processing, batch data preparation vs. live stream ingestion, Redis as the live state bus, GNN/physics as the exposure engine, routing/matching as decision layers, API as the synchronous interface, and the frontend as the visualization/interaction layer. The static graph/model artifacts and the audit trail should be shown as separate storage and traceability components.
Main entry points: ingestion Ingestion · main.py FastAPI · redis_client.py Redis bus · router Router
Components 18
External sources
Upstream environmental and road-data inputs.
- Weather · Open-Meteo weather feeds used for wind and atmospheric context.
- Air quality · Open-Meteo air-quality feeds used as live environmental input.
- Station obs · AQICN/CPCB station observations for ground-truth pollutant measurements.
Data layer
Batch ingestion, fusion, and local artifacts.
- Road graph · Projected road-network artifact derived from OpenStreetMap/OSMnx and stored under data/.
data - Raw data · Normalized Parquet and checkpointed source data under data/raw/.
data/raw - Prep scripts · Archive pulls, gap repair, graph tensor build, and startup sync scripts.
scripts
Core backend
Live state bus, exposure modeling, routing, matching, and API.
- Ingestion · Fetchers, bias correction, data fusion, nowcasting, and traffic-spike handling.
ingestion - Redis bus · Live runtime bus for weather, air quality, and sensor streams.
shared/redis_client.py - Exposure engine · Physics-informed plume modeling and trained GNN edge exposure estimation.
gnn - Edge costs · Turns exposure and inhalation into inhaled-dose-based edge weights.
router/edge_cost.py - Router · A* routing over the road graph, including optional Rust acceleration.
router - Matcher · Stable matching, quotas, route pools, and equilibrium checks for commuter assignment.
matcher - Audit trail · Immutable decision trace with request context, model version, graph snapshot, and stream state.
router/stake_audit.py - FastAPI · Synchronous backend surface for health, plume, route, and audit endpoints.
router/api/main.py - Shared core · Shared config, geo utilities, logging, physics settings, Redis client, and schemas.
shared
Frontend
Browser application for visualization and interaction.
- Web app · Vite/React frontend for plume maps, routes, equilibrium views, simulations, and audits.
frontend/src
Connections 25
- Weather → Ingestion: weather · Weather feeds are fetched and normalized into the live pipeline.
- Air quality → Ingestion: air · Air-quality feeds are fetched and normalized into the live pipeline.
- Station obs → Ingestion: stations · Station observations are fused into the live pipeline.
- Road graph → Exposure engine: graph input · Static road geometry is consumed for exposure modeling.
- Raw data → Ingestion: batch input · Archived and checkpointed data support ingestion and repair.
- Prep scripts → Raw data: prepare · Scripts pull archives, repair gaps, and build derived datasets.
- Prep scripts → Road graph: build · Scripts produce graph tensors and derived graph artifacts.
- Ingestion → Redis bus: publish · Ingestion publishes near-real-time streams and state to Redis.
- Redis bus → Exposure engine: state · The exposure engine reads live environmental state.
- Redis bus → Router: latest state · Routing reads current stream state without direct source coupling.
- Shared core → Ingestion: contracts · Shared config, schemas, and utilities align ingestion with the rest of the system.
- Shared core → Exposure engine: physics config · Shared physics and schema utilities support model computation.
- Shared core → Router: geo/config · Shared geometry and config support routing calculations.
- Exposure engine → Edge costs: exposure · Edge-level exposure estimates feed routing cost calculation.
- Edge costs → Router: weights · Dose-based costs are used by the A* router.
- Router → Matcher: candidate corridors · Routing provides corridor options for commuter allocation.
- Matcher → FastAPI: results · Matching outcomes are exposed through the backend service.
- Router → Audit trail: trace · Route decisions are captured for reproducibility and compliance.
- Audit trail → FastAPI: lookup · Audit records are retrievable through the service API.
- FastAPI → Web app: serve · Frontend consumes route, plume, and audit endpoints from the API.
- Redis bus → FastAPI: live reads · API reads current stream state for synchronous requests.
- Compose → Redis bus: runs · Compose defines the runtime backbone for local deployment.
- Compose → FastAPI: runs · Compose orchestrates the backend service.
- CI/CD → FastAPI: validate · Workflows test and deploy backend changes.
- CI/CD → Web app: validate · Workflows test and deploy frontend changes.