# emi420/underpass architecture

> Architecture diagram of the GitHub repository emi420/underpass, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/emi420/underpass
- Repository: https://github.com/emi420/underpass

## Overview

##### Main Components

a) Data Flow Components:
- OSM Planet Data Source
- Bootstrap System
- Replication Engine
- PostgreSQL/PostGIS Database
- Configuration System

b) Core Processing Components:
- Change File Processor
- Geographic Data Handler
- Raw Data Processor
- Database Connection Manager

##### Architecture Pattern
The system follows a pipeline architecture pattern with these key aspects:

- Input Processing -> Data Transformation -> Database Storage
- Parallel processing capabilities (evident from the threads.cc implementation)
- Event-driven updates for real-time synchronization

**[External Sources]**
- OSM Planet Data -----> [Bootstrap System]
- OSM Change Files ---> [Replication Engine]

- v
**[Core Processing Layer]**
- Geographic Utils <---> Change Processor
- Raw Data Handler <---> OSM Objects Parser

- v
**[Storage Layer]**
- PostgreSQL/PostGIS Database

##### Component Details to Highlight

a) Bootstrap System:
- Connection to initial PBF file processing
- Area filter capabilities
- Link to database initialization

b) Replication Engine:
- Multi-threaded nature
- Connection to OSM change files
- Configuration management

c) Processing Components:
- Geographic utilities (src/utils/geo.*)
- OSM object handling (src/osm/*)
- Raw data processing (src/raw/*)

##### Important Relationships

- Bidirectional flow between processors and database
- Configuration influence on all components
- Data transformation pipeline
- Error handling and logging paths

##### Technical Details

- Boost library dependencies
- PostgreSQL/PostGIS integration
- XML processing capabilities
- Geographic data handling

##### Specific Implementation Notes

- Near real-time update capability
- High-performance processing
- Modular component structure
- Geographic data filtering
- Database optimization

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph External["External Data Sources"]
        Planet["OSM Planet Data"]:::external
        Changes["OSM Change Files"]:::external
    end

    subgraph Bootstrap["Bootstrap Processing"]
        Boot["Bootstrap System"]:::bootstrap
        PBF["PBF File Processor"]:::bootstrap
    end

    subgraph Replication["Replication Layer"]
        RepEng["Replication Engine"]:::replication
        Threads["Thread Manager"]:::replication
        PlanetRep["Planet Replicator"]:::replication
    end

    subgraph Processing["Core Processing"]
        Change["Change File Processor"]:::processing
        GeoHandler["Geographic Data Handler"]:::processing
        RawProc["Raw Data Processor"]:::processing
        OSMObj["OSM Objects Handler"]:::processing
    end

    subgraph Storage["Storage Layer"]
        DB["PostgreSQL/PostGIS"]:::storage
        DBSetup["Database Setup"]:::storage
        DBIndexes["Database Indexes"]:::storage
    end

    subgraph Support["Support Systems"]
        Config["Configuration System"]:::support
        Logger["Logging System"]:::support
    end

    %% Relationships
    Planet --> Boot
    Changes --> RepEng
    Boot --> PBF
    PBF --> DB
    RepEng --> Threads
    RepEng --> PlanetRep
    Threads --> Change
    PlanetRep --> Change
    Change --> GeoHandler
    Change --> OSMObj
    GeoHandler --> RawProc
    OSMObj --> RawProc
    RawProc --> DB
    Config --> RepEng
    Config --> Boot
    Logger --> RepEng
    Logger --> Boot
    DBSetup --> DB
    DBIndexes --> DB

    %% Styling
    classDef external fill:#6C8EBF,stroke:#2F5597
    classDef bootstrap fill:#70AD47,stroke:#507E32
    classDef replication fill:#ED7D31,stroke:#B45F24
    classDef processing fill:#FFD966,stroke:#BF9F4C
    classDef storage fill:#A5A5A5,stroke:#7A7A7A
    classDef support fill:#9673A6,stroke:#6F567B

    %% Click Events
    click Boot "https://github.com/emi420/underpass/blob/master/src/bootstrap/bootstrap.cc"
    click RepEng "https://github.com/emi420/underpass/blob/master/src/replicator/replication.cc"
    click Threads "https://github.com/emi420/underpass/blob/master/src/replicator/threads.cc"
    click PlanetRep "https://github.com/emi420/underpass/blob/master/src/replicator/planetreplicator.cc"
    click GeoHandler "https://github.com/emi420/underpass/blob/master/src/utils/geo.cc"
    click RawProc "https://github.com/emi420/underpass/blob/master/src/raw/queryraw.cc"
    click DB "https://github.com/emi420/underpass/blob/master/src/data/pq.cc"
    click DBIndexes "https://github.com/emi420/underpass/blob/master/setup/db/indexes.sql"
    click DBSetup "https://github.com/emi420/underpass/blob/master/setup/db/setupdb.sh"
    click OSMObj "https://github.com/emi420/underpass/blob/master/src/osm/osmobjects.cc"
    click Change "https://github.com/emi420/underpass/blob/master/src/osm/osmchange.cc"
    click Config "https://github.com/emi420/underpass/blob/master/src/underpassconfig.hh"
    click Logger "https://github.com/emi420/underpass/blob/master/src/utils/log.cc"
```
