# hossamtabana/terrametria architecture

> Architecture diagram of the GitHub repository hossamtabana/terrametria, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hossamtabana/terrametria
- Repository: https://github.com/hossamtabana/terrametria

## Overview

##### Project type and purpose
- Terrametria is a full-stack, data-driven web application that renders an interactive 3D population density map in the browser.
- Core responsibilities:
- Data ingestion & processing (Databricks workflows)
- Data storage & serving (Databricks SQL Warehouse)
- Backend API (FastAPI as a Databricks App)
- Frontend SPA (React + Deck.gl)

##### File structure for architectural clues
- Top-level folders:
- src/frontend → React/Vite single-page application
- src/terrametria → Python backend modules (entrypoint, loader, API)
- databricks.yml, hooks/yarn_build.py → CI/CD & Databricks bundle config
- Tests target the API layer (tests/test_api.py).
- Configuration files (.env.example, pyproject.toml, package.json, vite.config.ts) hint at two distinct build systems.

##### README for additional context
- Workflow:
1. Databricks bundle deploy & run → executes Python loader + workflows.
2. Data lands in a SQL Warehouse (catalog + schema).
3. Backend and frontend run locally (hatch dev-backend / yarn dev-frontend) or deployed as a Databricks App.
- Tech stack summary (React, TypeScript, FastAPI, Databricks Apps, Databricks SQL).

a. Main components
- External Data Source (HDX/Meta)
- Databricks Compute (Bundles & Jobs)
- Databricks SQL Warehouse (persistent storage)
- Backend Service (FastAPI running as a Databricks App)
- Frontend App (React + Deck.gl on Vite)
- Client Browser

b. Relationships & data-flow
1. Loader module in Databricks Compute fetches HDX data → transforms → writes to SQL tables.
2. Backend Service connects to SQL Warehouse using Client ID/Secret → exposes HTTP endpoints (API).
3. Frontend App makes HTTP requests to Backend (GET density‐map data) → processes JSON → displays via Deck.gl.
4. Developer workflows:
- Local: hatch run dev-backend ↔ local FastAPI
- yarn dev-frontend ↔ local Vite server
- Deployed: Databricks App hosts both backend and frontend (static assets).

c. Important patterns & principles
- Clear separation of concerns: Data ingestion, storage, API layer, UI layer
- Environment-driven configuration (DATABRICKS_HOST, CLIENT_ID/SECRET, CATALOG, SCHEMA)
- Infrastructure as code via databricks.yml and bundle commands
- React componentization (components/ui, pages, lib)

d. Key technologies to annotate
- Deck.gl layer in the browser
- FastAPI endpoints (e.g., /density)
- Databricks CLI & Apps lifecycle
- SQL Warehouse (catalog.schema.table)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Source
    subgraph "External"
        HDX["HDX Data Source"]:::external
    end

    %% Databricks Compute
    subgraph "Databricks Compute"
        Loader["Loader Job"]:::compute
        Entrypoint["Job Entrypoint"]:::compute
        DBBundleConfig["Databricks Bundle Config"]:::compute
    end

    %% Data Layer
    SQLWarehouse["Databricks SQL Warehouse"]:::data

    %% Backend Service
    subgraph "Backend Service (Databricks App)"
        API["FastAPI App"]:::backend
        Tests["API Tests"]:::backend
    end

    %% Frontend SPA
    subgraph "Frontend SPA"
        ReactApp["React + Deck.gl SPA"]:::frontend
        DensityPage["DensityMap.tsx"]:::frontend
        APIlib["api.ts"]:::frontend
        Main["main.tsx"]:::frontend
        UIComp["MapControls & UI"]:::frontend
        BuildConfig["Build Config"]:::frontend
    end

    %% Environment Config
    Env["Environment Config"]:::infra

    %% Client
    Browser["Client Browser"]:::external

    %% Data Flow
    HDX -->|"fetch & transform"| Loader
    Loader -->|"write to"| SQLWarehouse
    Entrypoint -->|"triggers loader"| Loader
    DBBundleConfig -->|"defines jobs"| Entrypoint

    SQLWarehouse -->|"query"| API
    Env -->|"credentials| database"| API
    Tests -->|"test endpoints"| API

    API -->|"HTTP GET /density"| ReactApp
    ReactApp -->|"renders map"| Browser

    %% Click Events
    click Loader "https://github.com/hossamtabana/terrametria/blob/main/src/terrametria/loader.py"
    click Entrypoint "https://github.com/hossamtabana/terrametria/blob/main/src/terrametria/entrypoint.py"
    click DBBundleConfig "https://github.com/hossamtabana/terrametria/blob/main/databricks.yml"
    click DBBundleConfig "https://github.com/hossamtabana/terrametria/blob/main/hooks/yarn_build.py"
    click SQLWarehouse "https://github.com/hossamtabana/terrametria/blob/main/src/terrametria/config.py"
    click API "https://github.com/hossamtabana/terrametria/blob/main/src/terrametria/backend.py"
    click API "https://github.com/hossamtabana/terrametria/blob/main/src/terrametria/logger.py"
    click Tests "https://github.com/hossamtabana/terrametria/blob/main/tests/test_api.py"
    click Env "https://github.com/hossamtabana/terrametria/blob/main/.env.example"
    click ReactApp "https://github.com/hossamtabana/terrametria/tree/main/src/frontend/"
    click DensityPage "https://github.com/hossamtabana/terrametria/blob/main/src/frontend/src/pages/DensityMap.tsx"
    click APIlib "https://github.com/hossamtabana/terrametria/blob/main/src/frontend/src/lib/api.ts"
    click Main "https://github.com/hossamtabana/terrametria/blob/main/src/frontend/src/main.tsx"
    click UIComp "https://github.com/hossamtabana/terrametria/blob/main/src/frontend/src/components/MapControls.tsx"
    click UIComp "https://github.com/hossamtabana/terrametria/tree/main/src/frontend/src/components/ui/*"
    click BuildConfig "https://github.com/hossamtabana/terrametria/blob/main/src/frontend/package.json"
    click BuildConfig "https://github.com/hossamtabana/terrametria/blob/main/src/frontend/vite.config.ts"

    %% Styles
    classDef external fill:#cccccc,stroke:#333,stroke-width:1px
    classDef compute fill:#b3e6b3,stroke:#333,stroke-width:1px
    classDef data fill:#cce0ff,stroke:#333,stroke-width:1px
    classDef backend fill:#ffd699,stroke:#333,stroke-width:1px
    classDef frontend fill:#fff2b3,stroke:#333,stroke-width:1px
    classDef infra fill:#e6ccff,stroke:#333,stroke-width:1px
```
