How jeffgbanziali/urban_data_architecture works
Urban Immo combines a Paris property portal with a data-exploration product. Open datasets enter Bronze, are cleaned into Silver, and aggregated into Gold; a separate real-time feed supplies air-quality and Vélib data. Airflow schedules these stages, while the FastAPI service exposes property, analytics, account, and live-data capabilities to role-based React experiences. The graph reflects documented flows and indexed imports; unsampled modules are represented at their supported paths, so some internal links are omitted.
Main entry points: main.py FastAPI application · db.py Gold database · App.tsx React application · dags Airflow DAGs
Components 30
Data platform
- Bronze ingestion
pipeline/bronze/download_sources.py - Bronze object storage
pipeline/bronze/minio_client.py - Silver cleaning
pipeline/silver/clean_silver.py - Gold aggregation
pipeline/gold/aggregate_gold.py - Gold database
api/app/db.py - Geospatial utilities
streaming/geo_utils.py - Realtime fetcher
pipeline/bronze/realtime_fetcher.py - Pipeline reporting
pipeline/bronze/pipeline_reporter.py
API and storage
- FastAPI application
api/app/main.py - Property and favorites API
api/app/routers/biens.py - Analytics API
api/app/routers/marts.py - Price and comparison API
api/app/routers/prix.py - GeoJSON API
api/app/routers/geo.py - Authentication and roles
api/app/auth.py - Admin pipeline reports
api/app/routers/admin_pipeline.py - MongoDB client
api/app/mongo_client.py - MinIO client
api/app/minio_client.py - Realtime websocket
api/app/routers/realtime_ws.py
Web experience
- React application
frontend/src/App.tsx - Auth context
frontend/src/context/AuthContext.tsx - API client
frontend/src/lib/api.ts - Data explorer
frontend/src/pages/Explorateur.tsx - Maps and charts
frontend/src/components/ParisMaps.tsx - Client dashboard
frontend/src/pages/dashboard/DashboardClient.tsx - Admin dashboard
frontend/src/pages/dashboard/DashboardAdmin.tsx - Property pages
frontend/src/pages/Listings.tsx
Orchestration
- Airflow DAGs
airflow/dags
Other components
- Open data sources
- Site visitor
- WAQI and Vélib
Connections 41
- Open data sources → Bronze ingestion: provides data
- Bronze ingestion → Bronze object storage: writes snapshots
- Bronze object storage → Silver cleaning: supplies files
- Silver cleaning → Gold aggregation: supplies cleaned data
- Gold aggregation → Gold database: writes aggregates
- Gold aggregation → Bronze object storage: writes GeoJSON
- Airflow DAGs → Bronze ingestion: schedules
- Airflow DAGs → Silver cleaning: schedules
- Airflow DAGs → Gold aggregation: schedules
- WAQI and Vélib → Realtime fetcher: provides updates
- Airflow DAGs → Realtime fetcher: schedules
airflow/dags/dag_realtime_stream.py - Realtime fetcher → Gold database: publishes live data
- Bronze ingestion → Pipeline reporting: records run metrics
pipeline/bronze/download_sources.py - Pipeline reporting → Gold database: inserts reports
pipeline/bronze/pipeline_reporter.py - FastAPI application → Property and favorites API: registers routes
api/app/main.py - FastAPI application → Analytics API: registers routes
api/app/main.py - FastAPI application → Price and comparison API: registers routes
api/app/main.py - FastAPI application → GeoJSON API: registers routes
api/app/main.py - FastAPI application → Authentication and roles: registers routes
api/app/main.py - FastAPI application → Admin pipeline reports: registers routes
api/app/main.py - FastAPI application → Realtime websocket: registers routes
api/app/main.py - FastAPI application → MongoDB client: opens connection
api/app/main.py - Property and favorites API → MongoDB client: reads attributes
api/app/routers/biens.py - Property and favorites API → Gold database: queries properties
api/app/routers/biens.py - Analytics API → Gold database: queries marts
api/app/routers/marts.py - Price and comparison API → Gold database: queries prices
api/app/routers/prix.py - GeoJSON API → MinIO client: reads GeoJSON
api/app/routers/geo.py - Authentication and roles → Gold database: checks users
api/app/auth.py - Admin pipeline reports → Gold database: reads reports
api/app/routers/admin_pipeline.py - Site visitor → React application: uses
- React application → Data explorer: routes to
frontend/src/App.tsx - React application → Property pages: routes to
frontend/src/App.tsx - React application → Client dashboard: routes to
frontend/src/App.tsx - React application → Admin dashboard: routes to
frontend/src/App.tsx - React application → Auth context: provides auth
frontend/src/App.tsx - Auth context → API client: calls
frontend/src/context/AuthContext.tsx - Data explorer → Maps and charts: renders
frontend/src/pages/Explorateur.tsx - Data explorer → API client: requests data
frontend/src/pages/Explorateur.tsx - Client dashboard → API client: requests favorites and prices
frontend/src/pages/dashboard/DashboardClient.tsx - API client → FastAPI application: sends HTTP requests
- Realtime websocket → Gold database: pushes updates