How felt/tippecanoe works
Project Type and Purpose
• Tippecanoe is an open-source, command-line C++ tool for generating and manipulating vector tile sets (MBTiles/PMTiles) from geospatial data (GeoJSON, FlatGeobuf, CSV).
• It provides multiple subcommands:
• tippecanoe – build tilesets with tiling, simplification, clustering, filtering
• tile-join – merge and join attributes into existing tilesets
• tippecanoe-json-tool – extract and join GeoJSON features
• tippecanoe-overzoom – generate higher zoom tiles by overzooming.
File-Structure Analysis
• Top-Level:
• Build & CI: Dockerfile(.centos7), Makefile, Travis/ GitHub workflows, codecov.yml
• Core sources: attribute.cpp/hpp, geometry.cpp/hpp, evaluator.cpp/hpp, main.cpp/hpp, tile.cpp/hpp, thread.cpp/hpp, etc.
• Third-party libs vendored under directories: mapbox/ (geometry, variant), flatbuffers/, protozero/, milo/, catch/ (tests), clipper2/, jsonpull/, pmtiles/, filters/
• Tests: tests/ (hundreds of test cases organized by feature) and lambda/ (Python wrapper to run tippecanoe as AWS Lambda)
• Subtools: jsontool.cpp, serial.cpp, tile-join.cpp, version.hpp, vector_tile.proto.
• Patterns:
• Modular core library – geometry processing, tile generation, attribute handling
• CLI front-end dispatching into algorithmic pipeline
• Multi-threaded processing (thread.cpp)
• Pluggable filters (filter scripts under filters/)
• Use of SQLite for MBTiles and custom PMTiles backend.
README Insights
• Describes the overall workflow: parse input → transform/simplify → tile/pyramid → write MBTiles/PMTiles.
• Enumerates a rich set of options mapping to pipeline stages (simplification, clustering, coalescing, attribute filtering, overzoom).
• Mentions Docker image and usage patterns.
• Main components:
A. CLI Parser
B. Input Handlers
– GeoJSON Reader (jsonpull + homemade parser)
– FlatGeobuf Reader (flatbuffers)
– CSV Parser
C. Core Processing Pipeline
1. Feature Unification & Attribute Module (attribute.cpp/filters, evaluator)
2. Geometry Processing (mapbox geometry, clipper2)
3. Simplification & Clustering (visvalingam.cpp, gamma, tile-cache)
4. Tiling Engine (tile.cpp, dirtiles.cpp, overzoom)
D. Output Writers
– MBTiles Writer (SQLite wrapper)
– PMTiles Writer (pmtiles/)
– Directory Output
E. Subcommands
– tile-join (merging/CSV join engine)
– tippecanoe-json-tool (extraction & join via sorted streams)
– tippecanoe-overzoom (parent‐child tile processing)
F. Third-Party Libraries
– mapbox geometry, flatbuffers, protozero, milo, catch, clipper2, jsonpull
• Relationships:
• CLI Parser feeds options to instantiate the Core Pipeline
• Input Handlers stream Features into Pipeline
• Pipeline stages hand off to next via in‐memory queues or temporary files
• Multi‐thread Manager schedules Input Parsing and Geometry Processing in parallel
• Output Writers consume tile buffers from Tiling Engine
• tile-join reads existing tiles via Output Reader, merges/join attributes, writes via Writers
• Lambda wrapper invokes the CLI binary, handles S3 or local I/O.
• Architectural Patterns & Principles:
• Pipe-and-filter architecture for data processing
• Modular layering: Input → Processing → Output
• Vendor‐in-place for critical libraries for portability
• Thread-pool for parallelism
• Single‐responsibility subcommands built on shared core.
• Key Technologies:
• C++17 standard library, multithreading, filesystem
• SQLite3 for MBTiles
• Flatbuffers & Protozero for reading/writing PBF and .fgb
• Mapbox geometry library for geometric operations
• Clipper2 for polygon clipping
• Milo for double→string conversion
• Catch2 for unit testing.