# urban-toolkit/utk architecture

> Architecture diagram of the GitHub repository urban-toolkit/utk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/urban-toolkit/utk
- Repository: https://github.com/urban-toolkit/utk

## Overview

##### Project type and purpose
- This is a full-stack, cross-language visualization framework (“The Urban Toolkit” or UTK).
- It consists of:
a) A Python library (src/utk) for data loading, spatial computations, grammar parsing, map/network processing, and shadow/thematic calculations.
b) A Python-based HTTP server (src/utk_server.py) that wraps the core UTK library and exposes REST endpoints.
c) A TypeScript library (src/utk-ts) implementing the high-level grammar interpreter, data-API client, WebGL map/visualization pipeline, layer/shader management, and React helper components.
d) A React-based Single Page Application (src/utk-frontend) that ties everything together in the browser (with Jupyter and VR support).

##### File structure
- Top-level files: build scripts (build.sh), packaging (setup.py, requirements.txt), docs.
- Directories:
- docs/: user guides, API and grammar specifications, examples.
- src/utk/: Python package implementing data, grammar, and visualization core.
- src/utk_server.py: minimal Flask/FastAPI server exposing UTK functions.
- src/utk-ts/: TypeScript library packaged via Rollup for the browser.
- src/utk-frontend/: React app, Webpack/Gulp config, scripts, public assets, environment files, plus cave (Unity/VR) and Jupyter integrations.

##### README for additional insights
- The README emphasizes a “grammar-based framework for urban visual analytics,” support for 2D/3D maps, physical/thematic layers, data transformations, and OpenStreetMap integration.

##### Key components and their interactions
a. Presentation Layer (Browser / React SPA)
- Components: React pages (App.tsx, VR.tsx, Jupyter.tsx), UI widgets (GrammarPanel, MapRenderer, VisWidget, etc.), Jupyter bridge.
- Technology: React, TypeScript, Webpack, Babel, Gulp.
b. Visualization Engine (TypeScript Library)
- grammar-manager & grammar-interpreter: ingest high-level JSON grammar → produce visualization instructions.
- data-api & data-loader: fetch spatial/thematic data from Python server REST API or load from local files.
- mapview, layer-manager, shaders, WebGL context: assemble and render 2D/3D map layers (buildings, heatmaps, lines, points).
- Interaction channels & mouse events: handle user input.
c. Backend Application (Python Server)
- Exposes REST endpoints that wrap UTK’s core functions: data extraction (OSMnx, Osmium), building footprints, shadows, thematic layers.
- May use Flask or FastAPI (based on utk_server.py).
d. Core Data & Computation Library (Python UTK)
- Modules: data.py, map.py, osm.py, shadow_accumulator.py, urban_component.py, etc.
- Dependencies: Geopandas, OSMnx, Osmium, Shapely.
e. External Services / Data Sources
- OpenStreetMap servers, GeoJSON or other file inputs.

##### Architectural patterns and design principles
- Layered / Tiered Architecture:
1) Presentation (React SPA)
2) Application (TS library + Python server)
3) Data & Computation (Python UTK with spatial libs)
- Grammar-Based Design: High-level declarative grammar → interpreter pipeline.
- Modular, Pluggable Components: Separate packages for core library, server wrapper, TS engine, and SPA.
- Client-Server (RESTful) for dynamic data loading.
- WebGL rendering pipeline for high-performance visualization.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Browser / React SPA
    subgraph "Browser / React SPA"
        direction TB
        SPA_App["App.tsx"]:::frontend
        SPA_VR["VR.tsx"]:::frontend
        SPA_Jupyter["Jupyter.tsx"]:::frontend
        SPA_CAVE["caveSupport/"]:::frontend
        SPA_PyBridge["jupyterSupport/"]:::frontend
    end

    %% TypeScript Library
    subgraph "UTK-TS Visualization Engine"
        direction TB
        TS_GrammarMgr["grammar-manager.ts"]:::tslib
        TS_GrammarInt["grammar-interpreter.ts"]:::tslib
        TS_DataAPI["data-api.ts"]:::tslib
        TS_DataLoader["data-loader.ts"]:::tslib
        TS_MapView["mapview.ts"]:::tslib
        TS_LayerMgr["layer-manager.ts"]:::tslib
        TS_LayerBld["layer-buildings.ts"]:::tslib
        TS_LayerHeat["layer-heatmap.ts"]:::tslib
        TS_LayerLines["layer-lines.ts"]:::tslib
        TS_LayerPoints["layer-points.ts"]:::tslib
        TS_LayerTri["layer-triangles.ts"]:::tslib
        TS_Shaders["shaders/"]:::tslib
        TS_Interaction["interaction-channel.ts"]:::tslib
        TS_Mouse["mouse-events.ts"]:::tslib
        TS_ReactComp["reactComponents/"]:::tslib
    end

    %% Python Server
    subgraph "Python Server (UTK Server)" 
        direction TB
        PyServer["utk_server.py"]:::server
    end

    %% Python Core
    subgraph "Python UTK Core"
        direction TB
        Core_Data["data.py"]:::core
        Core_Files["files_interface.py"]:::core
        Core_Map["map.py"]:::core
        Core_OSM["osm.py"]:::core
        Core_Shadow["shadow_accumulator.py"]:::core
        Core_Urban["urban_component.py"]:::core
        Core_Utils["utils.py"]:::core
        Core_Cache["cache.py"]:::core
        Core_Errors["errors.py"]:::core
    end

    %% External
    OSM["OpenStreetMap API"]:::external

    %% Connections
    SPA_App -->|uses| TS_GrammarMgr
    SPA_VR -->|uses| TS_GrammarMgr
    SPA_Jupyter -->|uses| TS_GrammarMgr
    SPA_CAVE -->|integrates| TS_ReactComp
    SPA_PyBridge -->|integrates| TS_ReactComp

    TS_GrammarMgr -->|parses grammar| TS_GrammarInt
    TS_GrammarInt -->|generates instructions| TS_LayerMgr
    TS_LayerMgr -->|manages layers| TS_MapView
    TS_MapView -->|renders with| TS_Shaders
    TS_Interaction -->|handles input| TS_MapView
    TS_Mouse -->|captures events| TS_Interaction

    TS_LayerMgr --> TS_LayerBld
    TS_LayerMgr --> TS_LayerHeat
    TS_LayerMgr --> TS_LayerLines
    TS_LayerMgr --> TS_LayerPoints
    TS_LayerMgr --> TS_LayerTri

    TS_DataAPI -->|REST calls| PyServer
    TS_DataLoader -->|local load| Core_Data

    PyServer -->|invokes| Core_Data
    PyServer -->|invokes| Core_Map
    PyServer -->|invokes| Core_OSM
    PyServer -->|invokes| Core_Shadow
    PyServer -->|invokes| Core_Urban

    Core_OSM -->|HTTP fetch| OSM

    %% Click Events
    click SPA_App "https://github.com/urban-toolkit/utk/blob/master/src/utk-frontend/src/App.tsx"
    click SPA_VR "https://github.com/urban-toolkit/utk/blob/master/src/utk-frontend/src/VR.tsx"
    click SPA_Jupyter "https://github.com/urban-toolkit/utk/blob/master/src/utk-frontend/src/Jupyter.tsx"
    click SPA_CAVE "https://github.com/urban-toolkit/utk/tree/master/src/utk-frontend/src/caveSupport/"
    click SPA_PyBridge "https://github.com/urban-toolkit/utk/tree/master/src/utk-frontend/src/pythonComponents/jupyterSupport/"

    click TS_GrammarMgr "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/grammar-manager.ts"
    click TS_GrammarInt "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/grammar-interpreter.ts"
    click TS_DataAPI "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/data-api.ts"
    click TS_DataLoader "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/data-loader.ts"
    click TS_MapView "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/mapview.ts"
    click TS_LayerMgr "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/layer-manager.ts"
    click TS_LayerBld "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/layer-buildings.ts"
    click TS_LayerHeat "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/layer-heatmap.ts"
    click TS_LayerLines "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/layer-lines.ts"
    click TS_LayerPoints "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/layer-points.ts"
    click TS_LayerTri "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/layer-triangles.ts"
    click TS_Shaders "https://github.com/urban-toolkit/utk/tree/master/src/utk-ts/src/shaders/"
    click TS_Interaction "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/interaction-channel.ts"
    click TS_Mouse "https://github.com/urban-toolkit/utk/blob/master/src/utk-ts/src/mouse-events.ts"
    click TS_ReactComp "https://github.com/urban-toolkit/utk/tree/master/src/utk-ts/src/reactComponents/"

    click PyServer "https://github.com/urban-toolkit/utk/blob/master/src/utk_server.py"

    click Core_Data "https://github.com/urban-toolkit/utk/blob/master/src/utk/data.py"
    click Core_Files "https://github.com/urban-toolkit/utk/blob/master/src/utk/files_interface.py"
    click Core_Map "https://github.com/urban-toolkit/utk/blob/master/src/utk/map.py"
    click Core_OSM "https://github.com/urban-toolkit/utk/blob/master/src/utk/osm.py"
    click Core_Shadow "https://github.com/urban-toolkit/utk/blob/master/src/utk/shadow_accumulator.py"
    click Core_Urban "https://github.com/urban-toolkit/utk/blob/master/src/utk/urban_component.py"
    click Core_Utils "https://github.com/urban-toolkit/utk/blob/master/src/utk/utils.py"
    click Core_Cache "https://github.com/urban-toolkit/utk/blob/master/src/utk/cache.py"
    click Core_Errors "https://github.com/urban-toolkit/utk/blob/master/src/utk/errors.py"

    %% Styles
    classDef frontend fill:#e0f7e9,stroke:#2e7d32,stroke-width:1px
    classDef tslib fill:#e3f2fd,stroke:#1565c0,stroke-width:1px
    classDef server fill:#fff3e0,stroke:#ef6c00,stroke-width:1px
    classDef core fill:#f3e5f5,stroke:#6a1b9a,stroke-width:1px
    classDef external fill:#eeeeee,stroke:#888888,stroke-width:1px,stroke-dasharray: 2 2
```
