How dhanush342/atlas works
1. What this system is
- Bharat Tech Atlas is a two-tier product: a React/Vite frontend for interactive map exploration and a FastAPI backend that serves entity search, analytics, chat, ML, ETL, and map tiles.
- The repo is structured so the backend owns data access and domain logic, while the frontend focuses on visualization and user interaction.
2. User-facing frontend
- frontend/src/App.jsx composes the main UI: map, sidebar, stats, analytics, entity detail, search, and chat.
- Key components include StartupMap, SearchBar, Sidebar, StatsBar, AnalyticsPanel, MLInsightsPanel, EntityDetail, and ChatWidget.
- Hooks such as useMapData, useEntityDetail, useFilters, and useSupabase separate data-fetching and live-update concerns from presentation.
3. Backend application boundary
- backend/main.py is the FastAPI entrypoint that wires routes, middleware, security controls, and health/metrics endpoints.
- The backend is split by capability rather than by CRUD object, which is why there are distinct modules for chat, ETL, ML, analytics, matching, geocoding, and caching.
- backend/routes/* is the public API surface; the rest of backend/* contains the underlying service logic.
4. Core data layer
- The primary entity model is centered on startup ecosystem records such as startups, investors, mentors, incubators, accelerators, corporates, and government bodies.
- The system supports both local SQLite for simpler deployments and Supabase PostgreSQL + PostGIS for the full real-time, spatially indexed deployment.
- backend/database.py and backend/supabase_client.py indicate the repo can switch between local database access and cloud-backed Supabase access.
5. Spatial and map-serving path
- backend/routes/mvt_routes.py serves Mapbox Vector Tiles for map rendering, which is the high-performance path for dense geospatial visualization.
- PostGIS is used for viewport and radius queries, while geom plus a GIST index support spatial lookups.
- The frontend map is designed to request lightweight tiles first and fetch full entity details only on demand through the detail endpoints.
6. Entity search, filtering, and detail retrieval
- backend/routes/entities.py and related utilities handle entity listing, filtering, search, and lazy detail loading.
- This is the main read path for the app: search bar and map interactions drive entity queries, while detail panels request richer profile data only after user selection.
- The readme’s emphasis on “viewport summary” and “lazy entity detail” matches the repo’s route split between list/summary endpoints and detail endpoints.
7. Real-time data flow
- Supabase Realtime is the cloud event source for live inserts/updates; the frontend can subscribe to these changes, and the backend can also consume them for notifications or refresh logic.
- backend/chat and the SSE chat endpoint provide a second real-time stream: token-by-token responses sent over HTTP with buffering disabled at the proxy layer.
- This repo therefore has two real-time channels: database change events for ecosystem updates and SSE for conversational AI output.
8. Chat and AI subsystem
- backend/chat/engine.py, models.py, config.py, and routes.py define the chat stack, with both blocking completions and streaming responses.
- The readme indicates the streaming path uses SSE and token streaming, while search/web context may be injected into responses.
- The backend also includes backend/search_agent_routes.py, suggesting a separate agentic/search-assisted query surface distinct from plain chat.
9. ML and insights subsystem
- backend/ml/classifier.py, predictor.py, and serving.py form the model-serving boundary for keyword-based sector prediction and related ML features.
- backend/routes/ml_routes.py and backend/routes/mlops_routes.py expose inference and operational metrics, while backend/analytics.py provides ecosystem-level aggregations.
- On the UI side, MLInsightsPanel and AnalyticsPanel consume these endpoints to show derived insights rather than raw entity data.
10. ETL and data ingestion
- backend/etl/extract.py, transform.py, load.py, pipeline.py, and live_extract.py define the ingestion pipeline for startup ecosystem data.
- This pipeline is the path for bringing in government, commercial, and live source data, then normalizing it into the entity schema.
- backend/seed.py and backend/seed_entities.py support initial dataset bootstrapping, while scripts/scheduler.py suggests periodic refresh and cache-warming runs.
11. Normalization, enrichment, and geocoding
- backend/geocoding.py, backend/enrichment.py, backend/matching.py, and backend/directory.py are the post-ingestion services that improve data quality and usefulness.
- Geocoding converts place names into coordinates; enrichment adds missing or derived metadata; matching powers mentor/startup or investor/startup recommendations.
- scripts/dedupe.py is a separate maintenance tool for entity consolidation and duplicate resolution.
12. Caching and performance controls
- backend/cache.py provides a pluggable cache layer with local TTL/LRU behavior and optional Redis backing.
- This cache is important because the app serves high-volume spatial queries, analytics aggregations, and repeated detail lookups.
- The reverse proxy and backend are both tuned for performance, especially for tile delivery and streaming chat responses.
13. Security and operational boundaries
- backend/security.py centralizes request validation, rate limits, and other protections called out in the readme.
- The deployment stack includes nginx for SSL, compression, buffering control, SPA fallback, and headers; buffering is especially relevant for SSE chat streaming.
- Supabase Row-Level Security is part of the data boundary: public reads for active entities, service-role-only writes.
14. Deployment and runtime topology
- docker-compose.yml ties together the app, Redis, nginx, and a scheduler service.
- Dockerfile, systemd/bharat-tech-atlas.service, and scripts/deploy.sh show the project supports containerized deployment as well as a hardened service-managed deployment.
- requirements.txt and the frontend package files imply a classic Python backend plus Node-based frontend build pipeline, with tests in tests/ validating the API surface.
15. How to draw the diagram
- The most important arrows are: browser → nginx → FastAPI, FastAPI ↔ Supabase/Postgres/PostGIS, FastAPI ↔ Redis cache, Supabase Realtime → frontend/backend live updates, and frontend → MVT tiles/detail endpoints.
- The architecture is not one monolith; it is a set of specialized backend services behind one API, optimized for geospatial map rendering, live ecosystem updates, and AI-assisted interaction.
Main entry points: App.jsx App shell · main.py API app · entities.py Entities API · database.py Database layer
Components 23
Frontend
React/Vite UI for map exploration, search, details, analytics, and chat.
- App shell · Assembles the main dashboard layout and coordinates UI regions.
frontend/src/App.jsx - Startup map · Primary geospatial exploration surface for viewport-based browsing.
frontend/src/components/StartupMap.jsx - Search bar · Search and filter entry point driving entity lookup.
frontend/src/components/SearchBar.jsx - Sidebar · Holds browsable entity lists and supporting controls.
frontend/src/components/Sidebar.jsx - Entity detail · Lazy-loaded rich profile view for a selected entity.
frontend/src/components/EntityDetail.jsx - Analytics · Shows ecosystem-level aggregates and derived metrics.
frontend/src/components/AnalyticsPanel.jsx - ML insights · Displays prediction and model-derived insights.
frontend/src/components/MLInsightsPanel.jsx - Chat widget · Streaming conversational interface for assistant interactions.
frontend/src/chat/ChatWidget.jsx - Data hooks · Fetches map data, entity detail, filters, and live updates.
Backend
FastAPI API boundary plus domain services for data, tiles, chat, ML, ETL, and enrichment.
- API app · Wires middleware, security, routes, and health endpoints.
backend/main.py - Entities API · Main read path for listing, filtering, and detail retrieval.
backend/routes/entities.py - MVT tiles · Serves Mapbox Vector Tiles for fast geospatial rendering.
backend/routes/mvt_routes.py - Chat API · Chat routes and engine for blocking and SSE token streaming.
backend/chat/routes.py - ML API · Exposes prediction and model-serving endpoints.
backend/routes/ml_routes.py - ETL pipeline · Extracts, transforms, and loads ecosystem data.
backend/etl/pipeline.py - Enrichment · Geocoding, matching, directory, and enrichment services.
backend/enrichment.py - Cache · TTL/LRU and optional Redis-backed acceleration for hot reads.
backend/cache.py - Database layer · Local SQLite and Supabase/Postgres/PostGIS access boundary.
backend/database.py - Supabase client · Realtime and cloud database integration for live updates.
backend/supabase_client.py - Security · Request validation, rate limiting, and API protection.
backend/security.py
Runtime
Deployment, proxy, and operational pieces around the app.
- nginx · Handles SSL, compression, buffering, and SPA fallback.
nginx/nginx.conf - Compose stack · Runs the app with Redis, nginx, and scheduler services.
docker-compose.yml - Scheduler · Periodic refresh, cache-warming, and maintenance runs.
scripts/scheduler.py
Connections 36
- App shell → Startup map: renders · The app shell composes the main map workspace.
- App shell → Search bar: includes · Search is part of the primary interaction surface.
- App shell → Sidebar: includes · Sidebar supports entity browsing and navigation.
- App shell → Entity detail: includes · Selected entities open in the detail view.
- App shell → Analytics: includes · Analytics is embedded in the dashboard layout.
- App shell → ML insights: includes · ML insights are surfaced in the UI.
- App shell → Chat widget: includes · Chat is available as an assistant surface.
- Data hooks → Startup map: feeds · Hooks provide data for map rendering and viewport updates.
- Data hooks → Entity detail: loads · Hooks fetch lazy entity detail data.
- Data hooks → Entities API: queries · Frontend data hooks call entity list/detail endpoints.
- Data hooks → Supabase client: subscribes · Live update hooks integrate with Supabase realtime.
- Startup map → MVT tiles: requests · Map UI prefers vector tiles for dense spatial rendering.
- Search bar → Entities API: searches · Search drives filtered entity lookup.
- Sidebar → Entities API: browses · Sidebar content is populated from entity summary endpoints.
- Entity detail → Entities API: fetches · Detail panel loads richer records on demand.
- Analytics → ML API: reads · Analytics panel consumes derived backend metrics and predictions.
- ML insights → ML API: reads · ML insights panel is backed by model-serving endpoints.
- Chat widget → Chat API: streams · Chat UI receives streaming responses from the chat API.
- API app → Security: applies · Security controls are applied at the application boundary.
- API app → Entities API: mounts · Main app wires the public routes.
- API app → MVT tiles: mounts · Main app exposes tile-serving routes.
- API app → Chat API: mounts · Main app exposes chat routes.
- API app → ML API: mounts · Main app exposes ML routes.
- Entities API → Database layer: reads/writes · Entity queries rely on the core data layer.
- Entities API → Cache: accelerates · Hot entity reads and summaries are cached.
- MVT tiles → Database layer: spatial queries · Tile generation reads PostGIS-backed geospatial data.
- ETL pipeline → Database layer: loads · ETL writes normalized records into the data store.
- Enrichment → Database layer: updates · Enrichment services improve stored entity data.
- Enrichment → ETL pipeline: extends · Enrichment is a post-ingestion step in the data flow.
- Supabase client → Database layer: connects · Supabase client provides the cloud data/realtime path.
- Supabase client → Data hooks: events · Realtime updates can be consumed by the frontend.
- nginx → API app: proxies · Reverse proxy forwards requests to the API.
- Compose stack → nginx: orchestrates · Compose brings up the proxy alongside the application stack.
- Compose stack → API app: orchestrates · Compose runs the backend service as part of the stack.
- Compose stack → Scheduler: orchestrates · Scheduler is part of the deployment topology.
- Scheduler → ETL pipeline: refreshes · Scheduled jobs trigger ingestion and maintenance flows.