How adt-kmr/flipkart-gridlock works
1. What this repo is
- SPANDANA is a decision-support system for traffic-incident response, not an autonomous traffic-control system.
- The public product is split into a backend API, a React frontend, and a Postgres database, with optional external weather and messaging integrations.
- Internally, the backend Python package is still named clear.
2. Top-level system boundary
- The browser talks only to the frontend, and the frontend talks to the backend over HTTPS.
- The backend is the source of truth for incident storage, model inference, dispatch recommendations, corridor risk, and metrics.
- The database stores incident history, predictions, recommendations, and model/metric state.
3. Frontend subsystem
- The frontend is a Vite + React + TypeScript single-page app deployed separately from the backend.
- Main user-facing surfaces are:
- public landing page,
- operator login/dashboard/planning/stats pages,
- optional citizen view and citizen report form.
- It uses a typed API client and role tokens to call the backend, and it renders operational widgets like incident queues, maps, dispatch panels, SLA metrics, and rain-risk cards.
4. Backend API subsystem
- The backend is a FastAPI service served with Gunicorn/Uvicorn workers.
- It exposes a role-scoped REST API with operator and citizen access control.
- Core responsibilities include incident ingestion, retrieval, per-incident severity and clearance estimates, hotspot clustering, corridor risk nowcasts, dispatch planning, SLA/metrics reporting, and admin/retrain endpoints.
5. Data ingestion and persistence flow
- Incoming incident data is validated against a fixed raw schema, normalized, and idempotently inserted using event_id.
- Failed validation or exhausted retries are written to a dead-letter table rather than silently dropped.
- Canonical storage is in Postgres, with separate tables for incidents, predictions, recommendations, cached corridor risk, model registry entries, and metric history.
6. Modeling and inference layer
- Several models are loaded into backend memory at startup and can fail independently without taking down the whole API.
- Structured severity, text-only severity, clearance-time estimation, and corridor risk each have dedicated model pipelines.
- Hotspots and dispatch are computed on demand rather than persisted as always-on ML models.
- Outputs are validated before persistence or response so the API only serves bounded, sane results.
7. Decision-support workflows
- Incident ingestion can trigger best-effort inference immediately, so newly arrived incidents already have severity and clearance outputs.
- Operator views read from the backend for live queueing and per-incident analysis.
- Dispatch is explicitly human-in-the-loop: the backend returns ranked suggestions and a confirmation endpoint records operator approval.
- Degradation is designed per feature: if a model is missing, the API falls back to stale values, FIFO-style logic, or a route-level 503 instead of crashing globally.
8. Weather and external integration boundary
- The backend can optionally enrich incidents with rainfall data from Open-Meteo.
- A separate live rain-clog feature can combine Weather Union data with corridor history, but it is flag-gated and off by default.
- There is also an optional Telegram bot process that acts as a citizen-facing HTTP client, not as part of the web app.
- These integrations are auxiliary; the core service still works without them.
9. Configuration, auth, and rate limiting
- Behavior is heavily environment-driven: database URL, allowed CORS origins, role tokens, feature flags, and model thresholds all come from config.
- Auth is bearer-token based and resolves two scopes: operator and citizen.
- Rate limiting is handled in-process before normal request handling, with /healthz exempted.
- CORS is a real deployment boundary: the deployed frontend origin must be explicitly allowed or browser calls fail.
10. Build, bootstrap, and deployment
- Backend deployment is containerized; the Docker image bakes in model artifacts at build time so the service boots ready to serve.
- On startup, a bootstrap step initializes schema, seeds data if needed, and can train missing models.
- The frontend is deployed separately to Vercel and rebuilt with environment variables inlined at build time.
- Backend deployment is on Render, using a health check and pooled Postgres connection.
11. Retraining and lifecycle management
- The repo includes a model registry and a retrain lifecycle with stages such as candidate, shadow, canary, production, and archived.
- A retrain command can promote a better model version and archive the previous one.
- This is important architecturally because the serving pointer and the registry are separate: the live .joblib artifact is what the API uses, while the registry tracks promotion history.
12. Operations, testing, and CI/CD
- GitHub Actions runs linting, pytest with a Postgres service, a boot-time health check, and deployment hooks.
- Backend tests focus on idempotency, preprocessing, output validation, planning modules, NLP, and optional integrations.
- A live smoke test script exists for end-to-end API verification.
- These automation pieces are part of the operational boundary, ensuring the separately deployed frontend and backend stay compatible.
13. How to draw the architecture
- Draw three main boxes: browser/frontend, backend API, and Postgres.
- Add side boxes for external weather APIs and optional Telegram.
- Show the backend as the hub for validation, inference, storage, metrics, and recommendation generation.
- Show model artifacts as an internal backend dependency loaded at startup, not as a separate service.
Main entry points: app.py FastAPI App · App.tsx React SPA · __init__.py Model Pipelines · bootstrap.py Bootstrap
Components 19
Frontend
Separate Vite + React SPA for operators and citizens.
- React SPA · Vite + React + TypeScript frontend deployed separately from the API.
frontend/src/App.tsx - API Client · Typed request layer and role-token handling for browser-to-backend calls.
frontend/src/api.ts - Operator Views · Login, dashboard, planning, and stats surfaces for operators.
frontend/src/pages/OperatorDashboard.tsx - Citizen Views · Citizen-facing view and report flow for public incident submission.
frontend/src/pages/CitizenView.tsx - Ops Widgets · Incident queue, dispatch, map, SLA, metrics, and rain-risk widgets.
frontend/src/components/IncidentQueue.tsx
Backend API
FastAPI service that owns ingestion, inference, storage, and planning.
- FastAPI App · Role-scoped REST backend served by Gunicorn/Uvicorn workers.
backend/src/clear/app.py - Auth & Roles · Bearer-token access control for operator and citizen scopes.
backend/src/clear/auth.py - Incident Ingest · Validates raw incident schema, normalizes events, and handles idempotent inserts.
backend/src/clear/ingestion.py - Output Guardrails · Bounds and sanity-checks model outputs before response or persistence.
backend/src/clear/validation.py - Model Pipelines · In-memory severity, clearance, forecast, and text/NLP model pipelines.
backend/src/clear/models/__init__.py - Decision Support · Hotspot clustering, dispatch ranking, diversion aid, and corridor risk workflows.
backend/src/clear/resource_planner.py - Metrics & SLA · Operational metrics, SLA reporting, and corridor/hotspot risk summaries.
backend/src/clear/metrics.py
External Systems
Optional third-party integrations and deployment/runtime boundaries.
- Weather APIs · Optional Open-Meteo and Weather Union enrichment sources.
backend/src/clear/weather.py - Telegram Bot · Optional citizen-facing HTTP client process outside the web app.
backend/src/clear/telegram_bot.py
Operations
Build, bootstrap, and lifecycle management around the live system.
- Bootstrap · Initializes schema, seeds data, and can train missing models at startup.
backend/src/clear/bootstrap.py - Retrain Lifecycle · Promotes candidate models through shadow, canary, production, and archived states.
backend/src/clear/retrain.py - Backend Image · Container image that bakes in model artifacts and runs the API process.
backend/Dockerfile
Other components
- Browser · End users access the public landing page and role-specific views here.
- Postgres · System of record for incidents, predictions, recommendations, metrics, and model registry state.
Connections 24
- Browser → React SPA: uses · Users interact with the deployed frontend in the browser.
- React SPA → API Client: calls · The SPA uses a typed client and role tokens for backend requests.
- API Client → FastAPI App: HTTPS · Frontend communicates with the backend API over HTTPS/CORS.
- React SPA → Operator Views: renders · Operator navigation and workflows are assembled from dedicated pages.
- React SPA → Citizen Views: renders · Citizen-facing entry and report flows are exposed in the SPA.
- React SPA → Ops Widgets: composes · Operational widgets form the live dashboard experience.
- FastAPI App → Auth & Roles: guards · Endpoints resolve operator and citizen access scopes.
- FastAPI App → Incident Ingest: ingests · Incoming incidents are validated, normalized, and inserted idempotently.
- FastAPI App → Model Pipelines: loads · Model artifacts are loaded into memory at startup for inference.
- FastAPI App → Output Guardrails: checks · Model and workflow outputs are bounded before serving.
- FastAPI App → Decision Support: serves · The API computes dispatch, diversion, hotspot, and corridor risk responses.
- FastAPI App → Metrics & SLA: reports · SLA and operational reporting are served from the backend.
- Incident Ingest → Postgres: writes · Canonical incidents and dead-letter records are persisted in Postgres.
- Decision Support → Postgres: reads/writes · Recommendations, cached risk, and planning state are stored or retrieved.
- Metrics & SLA → Postgres: reads · Historical incidents, predictions, and metric history are queried from Postgres.
- Model Pipelines → Postgres: loads state · Model registry entries and persisted training state are backed by the database.
- Bootstrap → Postgres: initializes · Startup bootstraps schema and seed data in the database.
- Bootstrap → Model Pipelines: hydrates · Missing models may be trained or loaded during startup.
- Retrain Lifecycle → Model Pipelines: promotes · Candidate models can be promoted to the live serving pointer.
- Retrain Lifecycle → Postgres: records · Promotion history and registry lifecycle are stored separately from the live artifact.
- FastAPI App → Weather APIs: enriches · Optional rainfall and corridor-history enrichment is pulled from external weather sources.
- FastAPI App → Telegram Bot: optional client · A separate Telegram process can act as a citizen-facing HTTP client.
- Backend Image → FastAPI App: runs · The backend image is the deployable runtime for the API.
- Browser → Backend Image: served by · The separate frontend deployment talks to the independently hosted backend.