# jeffgbanziali/urban_data_architecture architecture

> Architecture diagram of the GitHub repository jeffgbanziali/urban_data_architecture, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jeffgbanziali/urban_data_architecture
- Repository: https://github.com/jeffgbanziali/urban_data_architecture

## Overview

Urban Immo combines a Paris property portal with a data-exploration product. Open datasets enter Bronze, are cleaned into Silver, and aggregated into Gold; a separate real-time feed supplies air-quality and Vélib data. Airflow schedules these stages, while the FastAPI service exposes property, analytics, account, and live-data capabilities to role-based React experiences. The graph reflects documented flows and indexed imports; unsampled modules are represented at their supported paths, so some internal links are omitted.

Main entry points: [`api/app/main.py`](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/main.py) (FastAPI application), [`api/app/db.py`](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/db.py) (Gold database), [`frontend/src/App.tsx`](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/App.tsx) (React application), [`airflow/dags`](https://github.com/jeffgbanziali/urban_data_architecture/tree/master/airflow/dags) (Airflow DAGs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data platform"]
  node_bronze["Bronze ingestion"]
  node_bronze_store[("Bronze object storage<br/>[minio_client.py]")]
  node_silver["Silver cleaning<br/>[clean_silver.py]"]
  node_gold["Gold aggregation<br/>[aggregate_gold.py]"]
  node_gold_store[("Gold database<br/>[db.py]")]
  node_geo["Geospatial utilities<br/>[geo_utils.py]"]
  node_realtime_fetch["Realtime fetcher"]
  node_reports["Pipeline reporting"]
end

subgraph group_api["API and storage"]
  node_api_main["FastAPI application<br/>[main.py]"]
  node_property_api["Property and favorites API<br/>[biens.py]"]
  node_analytics_api["Analytics API<br/>[marts.py]"]
  node_prices_api["Price and comparison API<br/>[prix.py]"]
  node_geo_api["GeoJSON API<br/>[geo.py]"]
  node_auth_api["Authentication and roles<br/>[auth.py]"]
  node_admin_api["Admin pipeline reports<br/>[admin_pipeline.py]"]
  node_mongo[("MongoDB client<br/>[mongo_client.py]")]
  node_minio_api[("MinIO client<br/>[minio_client.py]")]
  node_realtime_api["Realtime websocket<br/>[realtime_ws.py]"]
end

subgraph group_web["Web experience"]
  node_frontend["React application<br/>[App.tsx]"]
  node_auth_context["Auth context<br/>[AuthContext.tsx]"]
  node_api_client["API client<br/>[api.ts]"]
  node_explorer["Data explorer<br/>[Explorateur.tsx]"]
  node_explorer_views["Maps and charts<br/>[ParisMaps.tsx]"]
  node_client_dashboard["Client dashboard"]
  node_admin_dashboard["Admin dashboard<br/>[DashboardAdmin.tsx]"]
  node_property_pages["Property pages<br/>[Listings.tsx]"]
end

subgraph group_ops["Orchestration"]
  node_airflow["Airflow DAGs"]
end

node_open(("Open data sources"))
node_visitor(("Site visitor"))
node_live_sources(("WAQI and Vélib"))

node_open -->|"provides data"| node_bronze
node_bronze -->|"writes snapshots"| node_bronze_store
node_bronze_store -->|"supplies files"| node_silver
node_silver -->|"supplies cleaned data"| node_gold
node_gold -->|"writes aggregates"| node_gold_store
node_gold -->|"writes GeoJSON"| node_bronze_store
node_airflow -->|"schedules"| node_bronze
node_airflow -->|"schedules"| node_silver
node_airflow -->|"schedules"| node_gold
node_live_sources -->|"provides updates"| node_realtime_fetch
node_airflow -->|"schedules"| node_realtime_fetch
node_realtime_fetch -->|"publishes live data"| node_gold_store
node_bronze -->|"records run metrics"| node_reports
node_reports -->|"inserts reports"| node_gold_store
node_api_main -->|"registers routes"| node_property_api
node_api_main -->|"registers routes"| node_analytics_api
node_api_main -->|"registers routes"| node_prices_api
node_api_main -->|"registers routes"| node_geo_api
node_api_main -->|"registers routes"| node_auth_api
node_api_main -->|"registers routes"| node_admin_api
node_api_main -->|"registers routes"| node_realtime_api
node_api_main -->|"opens connection"| node_mongo
node_property_api -->|"reads attributes"| node_mongo
node_property_api -->|"queries properties"| node_gold_store
node_analytics_api -->|"queries marts"| node_gold_store
node_prices_api -->|"queries prices"| node_gold_store
node_geo_api -->|"reads GeoJSON"| node_minio_api
node_auth_api -->|"checks users"| node_gold_store
node_admin_api -->|"reads reports"| node_gold_store
node_visitor -->|"uses"| node_frontend
node_frontend -->|"routes to"| node_explorer
node_frontend -->|"routes to"| node_property_pages
node_frontend -->|"routes to"| node_client_dashboard
node_frontend -->|"routes to"| node_admin_dashboard
node_frontend -->|"provides auth"| node_auth_context
node_auth_context -->|"calls"| node_api_client
node_explorer -->|"renders"| node_explorer_views
node_explorer -->|"requests data"| node_api_client
node_client_dashboard -->|"requests favorites and prices"| node_api_client
node_api_client -->|"sends HTTP requests"| node_api_main
node_realtime_api -.->|"pushes updates"| node_gold_store

click node_bronze "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/download_sources.py"
click node_bronze_store "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/minio_client.py"
click node_silver "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/silver/clean_silver.py"
click node_gold "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/gold/aggregate_gold.py"
click node_gold_store "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/db.py"
click node_geo "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/streaming/geo_utils.py"
click node_realtime_fetch "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/realtime_fetcher.py"
click node_reports "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/pipeline_reporter.py"
click node_airflow "https://github.com/jeffgbanziali/urban_data_architecture/tree/master/airflow/dags"
click node_api_main "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/main.py"
click node_property_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/biens.py"
click node_analytics_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/marts.py"
click node_prices_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/prix.py"
click node_geo_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/geo.py"
click node_auth_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/auth.py"
click node_admin_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/admin_pipeline.py"
click node_mongo "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/mongo_client.py"
click node_minio_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/minio_client.py"
click node_realtime_api "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/realtime_ws.py"
click node_frontend "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/App.tsx"
click node_auth_context "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/context/AuthContext.tsx"
click node_api_client "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/lib/api.ts"
click node_explorer "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/Explorateur.tsx"
click node_explorer_views "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/components/ParisMaps.tsx"
click node_client_dashboard "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/dashboard/DashboardClient.tsx"
click node_admin_dashboard "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/dashboard/DashboardAdmin.tsx"
click node_property_pages "https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/Listings.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_bronze,node_bronze_store,node_silver,node_gold,node_gold_store,node_geo,node_realtime_fetch,node_reports toneBlue
class node_api_main,node_property_api,node_analytics_api,node_prices_api,node_geo_api,node_auth_api,node_admin_api,node_mongo,node_minio_api,node_realtime_api toneAmber
class node_frontend,node_auth_context,node_api_client,node_explorer,node_explorer_views,node_client_dashboard,node_admin_dashboard,node_property_pages toneMint
class node_airflow toneRose
class node_open,node_visitor,node_live_sources toneIndigo
```

## Components

### Data platform

- [Bronze ingestion](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/download_sources.py): `pipeline/bronze/download_sources.py`
- [Bronze object storage](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/minio_client.py): `pipeline/bronze/minio_client.py`
- [Silver cleaning](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/silver/clean_silver.py): `pipeline/silver/clean_silver.py`
- [Gold aggregation](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/gold/aggregate_gold.py): `pipeline/gold/aggregate_gold.py`
- [Gold database](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/db.py): `api/app/db.py`
- [Geospatial utilities](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/streaming/geo_utils.py): `streaming/geo_utils.py`
- [Realtime fetcher](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/realtime_fetcher.py): `pipeline/bronze/realtime_fetcher.py`
- [Pipeline reporting](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/pipeline/bronze/pipeline_reporter.py): `pipeline/bronze/pipeline_reporter.py`

### API and storage

- [FastAPI application](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/main.py): `api/app/main.py`
- [Property and favorites API](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/biens.py): `api/app/routers/biens.py`
- [Analytics API](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/marts.py): `api/app/routers/marts.py`
- [Price and comparison API](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/prix.py): `api/app/routers/prix.py`
- [GeoJSON API](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/geo.py): `api/app/routers/geo.py`
- [Authentication and roles](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/auth.py): `api/app/auth.py`
- [Admin pipeline reports](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/admin_pipeline.py): `api/app/routers/admin_pipeline.py`
- [MongoDB client](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/mongo_client.py): `api/app/mongo_client.py`
- [MinIO client](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/minio_client.py): `api/app/minio_client.py`
- [Realtime websocket](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/api/app/routers/realtime_ws.py): `api/app/routers/realtime_ws.py`

### Web experience

- [React application](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Auth context](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/context/AuthContext.tsx): `frontend/src/context/AuthContext.tsx`
- [API client](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/lib/api.ts): `frontend/src/lib/api.ts`
- [Data explorer](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/Explorateur.tsx): `frontend/src/pages/Explorateur.tsx`
- [Maps and charts](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/components/ParisMaps.tsx): `frontend/src/components/ParisMaps.tsx`
- [Client dashboard](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/dashboard/DashboardClient.tsx): `frontend/src/pages/dashboard/DashboardClient.tsx`
- [Admin dashboard](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/dashboard/DashboardAdmin.tsx): `frontend/src/pages/dashboard/DashboardAdmin.tsx`
- [Property pages](https://github.com/jeffgbanziali/urban_data_architecture/blob/master/frontend/src/pages/Listings.tsx): `frontend/src/pages/Listings.tsx`

### Orchestration

- [Airflow DAGs](https://github.com/jeffgbanziali/urban_data_architecture/tree/master/airflow/dags): `airflow/dags`

### Other components

- **Open data sources**
- **Site visitor**
- **WAQI and Vélib**

## Connections

- Open data sources → Bronze ingestion: provides data
- Bronze ingestion → Bronze object storage: writes snapshots
- Bronze object storage → Silver cleaning: supplies files
- Silver cleaning → Gold aggregation: supplies cleaned data
- Gold aggregation → Gold database: writes aggregates
- Gold aggregation → Bronze object storage: writes GeoJSON
- Airflow DAGs → Bronze ingestion: schedules
- Airflow DAGs → Silver cleaning: schedules
- Airflow DAGs → Gold aggregation: schedules
- WAQI and Vélib → Realtime fetcher: provides updates
- Airflow DAGs → Realtime fetcher: schedules (evidence: `airflow/dags/dag_realtime_stream.py`)
- Realtime fetcher → Gold database: publishes live data
- Bronze ingestion → Pipeline reporting: records run metrics (evidence: `pipeline/bronze/download_sources.py`)
- Pipeline reporting → Gold database: inserts reports (evidence: `pipeline/bronze/pipeline_reporter.py`)
- FastAPI application → Property and favorites API: registers routes (evidence: `api/app/main.py`)
- FastAPI application → Analytics API: registers routes (evidence: `api/app/main.py`)
- FastAPI application → Price and comparison API: registers routes (evidence: `api/app/main.py`)
- FastAPI application → GeoJSON API: registers routes (evidence: `api/app/main.py`)
- FastAPI application → Authentication and roles: registers routes (evidence: `api/app/main.py`)
- FastAPI application → Admin pipeline reports: registers routes (evidence: `api/app/main.py`)
- FastAPI application → Realtime websocket: registers routes (evidence: `api/app/main.py`)
- FastAPI application → MongoDB client: opens connection (evidence: `api/app/main.py`)
- Property and favorites API → MongoDB client: reads attributes (evidence: `api/app/routers/biens.py`)
- Property and favorites API → Gold database: queries properties (evidence: `api/app/routers/biens.py`)
- Analytics API → Gold database: queries marts (evidence: `api/app/routers/marts.py`)
- Price and comparison API → Gold database: queries prices (evidence: `api/app/routers/prix.py`)
- GeoJSON API → MinIO client: reads GeoJSON (evidence: `api/app/routers/geo.py`)
- Authentication and roles → Gold database: checks users (evidence: `api/app/auth.py`)
- Admin pipeline reports → Gold database: reads reports (evidence: `api/app/routers/admin_pipeline.py`)
- Site visitor → React application: uses
- React application → Data explorer: routes to (evidence: `frontend/src/App.tsx`)
- React application → Property pages: routes to (evidence: `frontend/src/App.tsx`)
- React application → Client dashboard: routes to (evidence: `frontend/src/App.tsx`)
- React application → Admin dashboard: routes to (evidence: `frontend/src/App.tsx`)
- React application → Auth context: provides auth (evidence: `frontend/src/App.tsx`)
- Auth context → API client: calls (evidence: `frontend/src/context/AuthContext.tsx`)
- Data explorer → Maps and charts: renders (evidence: `frontend/src/pages/Explorateur.tsx`)
- Data explorer → API client: requests data (evidence: `frontend/src/pages/Explorateur.tsx`)
- Client dashboard → API client: requests favorites and prices (evidence: `frontend/src/pages/dashboard/DashboardClient.tsx`)
- API client → FastAPI application: sends HTTP requests
- Realtime websocket → Gold database: pushes updates
