# felt/tippecanoe architecture

> Architecture diagram of the GitHub repository felt/tippecanoe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/felt/tippecanoe
- Repository: https://github.com/felt/tippecanoe

## Overview

##### Project Type and Purpose
- Tippecanoe is an open-source, command-line C++ tool for generating and manipulating vector tile sets (MBTiles/PMTiles) from geospatial data (GeoJSON, FlatGeobuf, CSV).
- It provides multiple subcommands:
- tippecanoe – build tilesets with tiling, simplification, clustering, filtering
- tile-join – merge and join attributes into existing tilesets
- tippecanoe-json-tool – extract and join GeoJSON features
- tippecanoe-overzoom – generate higher zoom tiles by overzooming.

##### File-Structure Analysis
- Top-Level:
- Build & CI: Dockerfile(.centos7), Makefile, Travis/ GitHub workflows, codecov.yml
- Core sources: attribute.cpp/hpp, geometry.cpp/hpp, evaluator.cpp/hpp, main.cpp/hpp, tile.cpp/hpp, thread.cpp/hpp, etc.
- Third-party libs vendored under directories: mapbox/ (geometry, variant), flatbuffers/, protozero/, milo/, catch/ (tests), clipper2/, jsonpull/, pmtiles/, filters/
- Tests: tests/ (hundreds of test cases organized by feature) and lambda/ (Python wrapper to run tippecanoe as AWS Lambda)
- Subtools: jsontool.cpp, serial.cpp, tile-join.cpp, version.hpp, vector_tile.proto.
- Patterns:
- Modular core library – geometry processing, tile generation, attribute handling
- CLI front-end dispatching into algorithmic pipeline
- Multi-threaded processing (thread.cpp)
- Pluggable filters (filter scripts under filters/)
- Use of SQLite for MBTiles and custom PMTiles backend.

##### README Insights
- Describes the overall workflow: parse input → transform/simplify → tile/pyramid → write MBTiles/PMTiles.
- Enumerates a rich set of options mapping to pipeline stages (simplification, clustering, coalescing, attribute filtering, overzoom).
- Mentions Docker image and usage patterns.

- Main components:
A. CLI Parser
B. Input Handlers
– GeoJSON Reader (jsonpull + homemade parser)
– FlatGeobuf Reader (flatbuffers)
– CSV Parser
C. Core Processing Pipeline
1. Feature Unification & Attribute Module (attribute.cpp/filters, evaluator)
2. Geometry Processing (mapbox geometry, clipper2)
3. Simplification & Clustering (visvalingam.cpp, gamma, tile-cache)
4. Tiling Engine (tile.cpp, dirtiles.cpp, overzoom)
D. Output Writers
– MBTiles Writer (SQLite wrapper)
– PMTiles Writer (pmtiles/)
– Directory Output
E. Subcommands
– tile-join (merging/CSV join engine)
– tippecanoe-json-tool (extraction & join via sorted streams)
– tippecanoe-overzoom (parent‐child tile processing)
F. Third-Party Libraries
– mapbox geometry, flatbuffers, protozero, milo, catch, clipper2, jsonpull

- Relationships:
- CLI Parser feeds options to instantiate the Core Pipeline
- Input Handlers stream Features into Pipeline
- Pipeline stages hand off to next via in‐memory queues or temporary files
- Multi‐thread Manager schedules Input Parsing and Geometry Processing in parallel
- Output Writers consume tile buffers from Tiling Engine
- tile-join reads existing tiles via Output Reader, merges/join attributes, writes via Writers
- Lambda wrapper invokes the CLI binary, handles S3 or local I/O.

- Architectural Patterns & Principles:
- Pipe-and-filter architecture for data processing
- Modular layering: Input → Processing → Output
- Vendor‐in-place for critical libraries for portability
- Thread-pool for parallelism
- Single‐responsibility subcommands built on shared core.

- Key Technologies:
- C++17 standard library, multithreading, filesystem
- SQLite3 for MBTiles
- Flatbuffers & Protozero for reading/writing PBF and .fgb
- Mapbox geometry library for geometric operations
- Clipper2 for polygon clipping
- Milo for double→string conversion
- Catch2 for unit testing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI
    CLI["CLI Parser"]:::io
    subgraph "Input Handlers"
        direction TB
        GeoJSON["GeoJSON Reader"]:::io
        FlatGeobuf["FlatGeobuf Reader"]:::io
        CSV["CSV Parser"]:::io
    end

    subgraph "Core Processing Pipeline"
        direction TB
        Attr["Feature Unification & Attribute Module"]:::core
        Geometry["Geometry Processing"]:::core
        Simplify["Simplification & Clustering"]:::core
        Tiling["Tiling Engine"]:::core
    end

    subgraph "Output Writers"
        direction TB
        MBTiles["MBTiles Writer"]:::io
        PMTiles["PMTiles Writer"]:::io
        DirOut["Directory Output Writer"]:::io
    end

    subgraph "Subcommands & Wrappers"
        direction TB
        TileJoin["tile-join"]:::sub
        JSONTool["tippecanoe-json-tool"]:::sub
        OverzoomSC["tippecanoe-overzoom"]:::sub
        Lambda["AWS Lambda Wrapper"]:::sub
    end

    subgraph "Thread Pool Manager"
        ThreadPool["Thread Pool"]:::core
    end

    subgraph "Vector Tile Schema"
        VTS["vector_tile.proto"]:::core
    end

    subgraph "Core Utilities & I/O"
        direction TB
        Utils["Memfile & Compression"]:::core
    end

    subgraph "Third-Party Libraries"
        direction TB
        MBGeom["mapbox geometry"]:::lib
        FlatBuf["flatbuffers"]:::lib
        ProtoZero["protozero"]:::lib
        Milo["milo"]:::lib
        Catch2["catch2"]:::lib
        Clipper2["clipper2"]:::lib
        JSONPull["jsonpull"]:::lib
        PMTilesLib["pmtiles"]:::lib
    end

    %% Connections
    CLI -->|"options"| ThreadPool
    CLI -->|"dispatch"| GeoJSON
    CLI -->|"dispatch"| FlatGeobuf
    CLI -->|"dispatch"| CSV

    GeoJSON -->|"Feature stream"| Attr
    FlatGeobuf -->|"Feature stream"| Attr
    CSV -->|"Feature stream"| Attr

    ThreadPool -->|"schedule parsing"| GeoJSON
    ThreadPool -->|"schedule geometry"| Geometry

    Attr -->|"Unified features"| Geometry
    Geometry -->|"Processed geometry"| Simplify
    Simplify -->|"Tile buffers"| Tiling
    Tiling -->|"tile buffers"| MBTiles
    Tiling -->|"tile buffers"| PMTiles
    Tiling -->|"tile files"| DirOut

    TileJoin -->|"read tiles"| MBTiles
    TileJoin -->|"merge attributes"| Attr
    TileJoin -->|"write tiles"| MBTiles

    JSONTool -->|"extract features"| Attr
    OverzoomSC -->|"generate overzoomed tiles"| Tiling

    Lambda -->|"invoke CLI"| CLI

    Utils -->|"mem I/O"| Tiling
    VTS -->|"schema"| MBTiles
    VTS -->|"schema"| PMTiles

    %% Legend
    subgraph Legend
        direction LR
        C1["Core Modules"]:::core
        C2["I/O Components"]:::io
        C3["Third-Party Libs"]:::lib
        C4["Subcommands"]:::sub
    end

    %% Click Events
    click CLI "https://github.com/felt/tippecanoe/blob/main/src/main.cpp"
    click GeoJSON "https://github.com/felt/tippecanoe/blob/main/src/geojson.cpp"
    click FlatGeobuf "https://github.com/felt/tippecanoe/blob/main/src/flatgeobuf.cpp"
    click CSV "https://github.com/felt/tippecanoe/blob/main/src/csv.cpp"
    click Attr "https://github.com/felt/tippecanoe/blob/main/src/attribute.cpp"
    click Geometry "https://github.com/felt/tippecanoe/blob/main/src/geometry.cpp"
    click Simplify "https://github.com/felt/tippecanoe/blob/main/src/visvalingam.cpp"
    click Tiling "https://github.com/felt/tippecanoe/blob/main/src/tile.cpp"
    click MBTiles "https://github.com/felt/tippecanoe/blob/main/src/mbtiles.cpp"
    click PMTiles "https://github.com/felt/tippecanoe/blob/main/src/pmtiles/pmtiles.hpp"
    click DirOut "https://github.com/felt/tippecanoe/blob/main/src/dirtiles.cpp"
    click TileJoin "https://github.com/felt/tippecanoe/blob/main/src/tile-join.cpp"
    click JSONTool "https://github.com/felt/tippecanoe/blob/main/src/jsontool.cpp"
    click OverzoomSC "https://github.com/felt/tippecanoe/blob/main/src/overzoom.cpp"
    click ThreadPool "https://github.com/felt/tippecanoe/blob/main/src/thread.cpp"
    click VTS "https://github.com/felt/tippecanoe/blob/main/src/vector_tile.proto"
    click Utils "https://github.com/felt/tippecanoe/blob/main/src/memfile.cpp"
    click MBGeom "https://github.com/felt/tippecanoe/tree/main/mapbox/"
    click FlatBuf "https://github.com/felt/tippecanoe/tree/main/flatbuffers/"
    click ProtoZero "https://github.com/felt/tippecanoe/tree/main/protozero/"
    click Milo "https://github.com/felt/tippecanoe/tree/main/milo/"
    click Catch2 "https://github.com/felt/tippecanoe/tree/main/catch/"
    click Clipper2 "https://github.com/felt/tippecanoe/tree/main/clipper2/"
    click JSONPull "https://github.com/felt/tippecanoe/tree/main/jsonpull/"
    click PMTilesLib "https://github.com/felt/tippecanoe/tree/main/pmtiles/"

    %% Styles
    classDef core fill:#e0ffe0,stroke:#2d8f2d;
    classDef io fill:#e0f0ff,stroke:#1f5fa8;
    classDef lib fill:#f0f0f0,stroke:#666666,stroke-dasharray: 2 2;
    classDef sub fill:#ffece0,stroke:#d86b17;
```
