# hkuds/urbangpt architecture

> Architecture diagram of the GitHub repository hkuds/urbangpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hkuds/urbangpt
- Repository: https://github.com/hkuds/urbangpt

## Overview

##### Main Components Identification

The system should be represented with these key components:

a) Core Model Components:
- Base LLM (Vicuna-7B)
- Spatio-Temporal Encoder
- Model Adapter Layer
- Instruction Tuning Module

b) Data Processing Pipeline:
- Spatio-Temporal Data Processors
- Instruction Data Handlers
- POI (Points of Interest) Data Manager

c) Training Infrastructure:
- Distributed Training System
- Checkpoint Management
- Model Evaluation Pipeline

##### Component Relationships

a) Data Flow:
- Raw Data → ST Data Processors → ST Encoder
- Instructions → Instruction Handler → Training Pipeline
- POI Data → Data Enrichment → Model Input

b) Model Architecture:
- ST Encoder → Model Adapter → Base LLM
- Training Pipeline → Model Weights → Checkpoint Storage

##### Specific Implementation Details

Technical elements:

a) Model Architecture:
- ST-Encoder (TCNs-based) integration
- Instruction tuning pathway
- Multi-GPU training setup

b) Serving Infrastructure:
- API Endpoints
- Model Workers
- Controller
- Web Interface

##### Input Layer
- Spatio-temporal data
- POI information
- Instructions
- Training configurations

##### Processing Layer
- Data preprocessing
- ST encoding
- Instruction handling

##### Model Layer
- Base LLM
- ST-Encoder
- Adapter mechanisms

##### Output Layer
- Prediction results
- Evaluation metrics
- API responses

##### The serving infrastructure

Keep the abstraction level high enough to show the overall architecture while including critical implementation details that make UrbanGPT unique, particularly the spatio-temporal aspects and instruction tuning components.

This system design should reflect UrbanGPT's unique capability to handle complex urban data while maintaining the power of large language models for natural language understanding and generation.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        ST[Spatio-temporal Data]
        POI[POI Information]:::poi
        INST[Instructions]:::inst
        CONF[Training Configurations]
    end

    subgraph Processing
        STP[ST Data Processors]
        IH[Instruction Handler]:::handler
        POIM[POI Data Manager]:::poi
    end

    subgraph "Model Core"
        direction TB
        STE[ST-Encoder]:::encoder
        MAL[Model Adapter]:::adapter
        BLLM[Base LLM - Vicuna-7B]:::llm
        MB[Model Builder]:::builder
    end

    subgraph Training
        DTS[Distributed Training System]:::train
        CKP[Checkpoint Management]:::ckpt
        MEV[Model Evaluation]:::eval
        MC[Metrics Calculator]:::metrics
    end

    subgraph "Serving Infrastructure"
        API[API Server]:::api
        MW[Model Worker]:::worker
        CTRL[Controller]:::ctrl
        WUI[Web Interface]:::web
    end

    %% Data Flow Connections
    ST --> STP
    POI --> POIM
    INST --> IH
    CONF --> DTS

    STP --> STE
    POIM --> MAL
    IH --> DTS

    STE --> MAL
    MAL --> BLLM
    MB --> BLLM

    DTS --> CKP
    CKP --> MEV
    MEV --> MC

    BLLM --> API
    API --> MW
    MW --> CTRL
    CTRL --> WUI

    %% Styling
    classDef default fill:#ddd,stroke:#333,stroke-width:1px
    classDef encoder fill:#90EE90,stroke:#333
    classDef adapter fill:#90EE90,stroke:#333
    classDef llm fill:#90EE90,stroke:#333
    classDef handler fill:#ADD8E6,stroke:#333
    classDef poi fill:#ADD8E6,stroke:#333
    classDef train fill:#FFB6C1,stroke:#333
    classDef ckpt fill:#FFB6C1,stroke:#333
    classDef eval fill:#DDA0DD,stroke:#333
    classDef metrics fill:#DDA0DD,stroke:#333
    classDef api fill:#FFD700,stroke:#333
    classDef worker fill:#FFD700,stroke:#333
    classDef ctrl fill:#FFD700,stroke:#333
    classDef web fill:#FFD700,stroke:#333
    classDef builder fill:#90EE90,stroke:#333
    classDef inst fill:#ADD8E6,stroke:#333

    %% Click Events
    click STE "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/model/st_layers/ST_Encoder.py"
    click MAL "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/model/model_adapter.py"
    click BLLM "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/model/ST_Llama.py"
    click IH "https://github.com/HKUDS/UrbanGPT/blob/main/instruction_generate/instruction_generate.py"
    click POIM "https://github.com/HKUDS/UrbanGPT/tree/main/instruction_generate/poi/"
    click DTS "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/train/stchat_trainer.py"
    click CKP "https://github.com/HKUDS/UrbanGPT/tree/main/checkpoints/"
    click MEV "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/eval/run_urbangpt.py"
    click API "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/serve/openai_api_server.py"
    click MW "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/serve/model_worker.py"
    click CTRL "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/serve/controller.py"
    click WUI "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/serve/gradio_web_server.py"
    click MC "https://github.com/HKUDS/UrbanGPT/blob/main/metric_calculation/metrics.py"
    click MB "https://github.com/HKUDS/UrbanGPT/blob/main/urbangpt/model/builder.py"
```
