# kesai-labs/py123d architecture

> Architecture diagram of the GitHub repository kesai-labs/py123d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kesai-labs/py123d
- Repository: https://github.com/kesai-labs/py123d

## Overview

1. **Top-level shape**  
   The repository is a Python package centered on one goal: normalize autonomous-driving datasets into a shared internal model, then expose that model through a single read/visualization API. The architecture is split into clear layers: raw dataset ingestion, canonical datatypes/geometry, Arrow-backed storage APIs, command-line orchestration, and visualization.

2. **Primary data flow**  
   The main flow is: dataset-specific downloader/parser reads raw files from external dataset sources, maps them into the package’s canonical datatypes, and writes converted logs/maps into an Apache Arrow-based format. After conversion, consumers use the scene/map APIs to read data with zero-copy, memory-mapped access. The converted log often keeps relative references to original camera/lidar assets rather than duplicating them.

3. **Canonical domain model**  
   `src/py123d/datatypes` and `src/py123d/geometry` define the common language of the system. Datatypes cover sensors, vehicle state, detections, metadata, map objects, and time. Geometry provides points, vectors, rotations, transforms, bounding boxes, polylines, poses, and occupancy maps. This is the main boundary that all dataset-specific code must target.

4. **Scene and map APIs**  
   `src/py123d/api/scene` and `src/py123d/api/map` are the public runtime interfaces. They sit on top of the stored Arrow representation and provide dataset-agnostic access to scenes, modalities, metadata, and map layers. The scene API is the main read path for time-indexed multi-modal data; the map API is the read path for HD-map content.

5. **Dataset-specific parser layer**  
   `src/py123d/parser` contains the ingestion/adaptation layer, with one subpackage per supported dataset: AV2, nuScenes, nuPlan, KITTI-360, PandaSet, WOD, CARLA/OpenDRIVE, Physical AI AV, NCore, and L3AD/Lead123D. Each parser package has dataset constants, helper utilities, sensor I/O, and often a separate map parser. This layer is where dataset quirks, schema differences, labels, timestamps, and coordinate conventions are normalized.

6. **Map ingestion strategy**  
   Map support is not monolithic: some datasets have native map parsers, while CARLA/OpenDRIVE is handled through a dedicated XODR parser and helper utilities. The code includes specialized map-object reconstruction, lane/road-edge/signals handling, and embedded CARLA map assets. The map pipeline is a distinct branch from sensor/scene ingestion but converges into the same map-object datatypes.

7. **Arrow writers and modality builders**  
   `src/py123d/api/scene/arrow` and `src/py123d/api/map/arrow` implement the write-side architecture. These modules build Arrow-compatible records for scenes, logs, and maps, with modality-specific handlers for cameras, lidar, ego state, box detections, traffic lights, and custom modalities. The write path is responsible for synchronizing modalities, caching scene state, and enforcing schema consistency before persistence.

8. **Modality boundaries**  
   Modalities are modeled explicitly rather than as generic blobs. Camera, lidar, ego state, detections, sync data, and custom fields each have dedicated Arrow modality implementations. This separation matters because the repository supports both synchronized and native-rate/asynchronous capture streams, so the system must preserve timing semantics per modality.

9. **Common runtime utilities**  
   `src/py123d/common` holds cross-cutting infrastructure: dataset path management, file-format IO for camera and lidar codecs, dependency gating, UUID/timer helpers, and execution backends. The IO subpackages support multiple encodings like JPEG/PNG/MP4 for cameras and LAZ/Draco/Arrow IPC for lidar. This layer is reused by parsers, writers, and visualization code rather than tied to any dataset.

10. **Execution and scalability**  
   The repository supports multiple execution strategies under `src/py123d/common/execution`: sequential, thread pool, process pool, and Ray. That indicates conversion and preprocessing can be parallelized, but the framework abstracts the executor choice so the rest of the pipeline is scheduler-agnostic. This is important for large dataset conversion jobs and for keeping the parsing logic independent of compute deployment.

11. **CLI and configuration system**  
   `src/py123d/script` is the orchestration layer. It wires together dataset download, conversion, and viewer startup through Hydra-based YAML configuration. The configs are split by concern: dataset presets, download presets, conversion presets, scene filters, execution backends, and viewer settings. This makes the runtime behavior highly declarative and allows the same core code to serve multiple dataset pipelines.

12. **Visualization subsystem**  
   `src/py123d/visualization` provides two consumer-facing render paths: Matplotlib for static plotting and Viser for interactive 3D inspection. The Viser stack has controllers, an element manager, and renderable elements for cameras, lidar, ego state, boxes, and maps. It consumes the public scene API, so visualization depends on the canonical data model rather than directly on raw dataset parsers.

13. **External dependencies and dataset boundaries**  
   The repo interacts with external dataset ecosystems and their storage formats: official dataset downloads, dataset-specific metadata and annotations, and in some cases vendored protobufs or map assets. The architecture deliberately isolates these external dependencies inside parser/download modules so the public API remains uniform even when backend formats differ substantially.

14. **Quality and release pipeline**  
   Tests are broad and mirror the package structure, with unit coverage for datatypes, geometry, parsers, executors, IO, and scene/map roundtrips. Docs are built from a dedicated Sphinx tree and tutorials demonstrate the conversion-to-viewer workflow. GitHub Actions handle docs deployment, PyPI release, linting/pre-commit, and pytest, which reinforces that the package is meant to be both a library and a reproducible data-processing toolchain.

Main entry points: [`src/py123d/datatypes`](https://github.com/kesai-labs/py123d/tree/main/src/py123d/datatypes) (Datatypes), [`src/py123d/script`](https://github.com/kesai-labs/py123d/tree/main/src/py123d/script) (CLI Scripts), [`src/py123d/parser/registry.py`](https://github.com/kesai-labs/py123d/blob/main/src/py123d/parser/registry.py) (Parser Registry), [`src/py123d/api/scene/arrow`](https://github.com/kesai-labs/py123d/tree/main/src/py123d/api/scene/arrow) (Scene Arrow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core_model["Core model"]
  node_datatypes["Datatypes<br/>canonical schema"]
  node_geometry["Geometry<br/>spatial primitives"]
end

subgraph group_runtime_api["Runtime API"]
  node_scene_api["Scene API<br/>public read API<br/>[scene_api.py]"]
  node_scene_arrow["Scene Arrow<br/>arrow reader/writer"]
  node_map_api["Map API<br/>[map_api.py]"]
  node_map_arrow["Map Arrow<br/>arrow map I/O"]
end

subgraph group_ingestion["Ingestion"]
  node_parser_registry(("Parser Registry<br/>dataset routing<br/>[registry.py]"))
  node_av2_parser["AV2<br/>dataset parser"]
  node_nuplan_parser["nuPlan<br/>dataset parser"]
  node_nuscenes_parser["nuScenes<br/>dataset parser"]
  node_kitti360_parser["KITTI-360<br/>dataset parser"]
  node_pandaset_parser["PandaSet<br/>dataset parser"]
  node_wod_parser["WOD<br/>dataset parser"]
  node_opendrive_parser["OpenDRIVE<br/>map parser"]
  node_physical_ai_av_parser["Physical AI AV<br/>dataset parser"]
  node_ncore_parser["NCore<br/>dataset parser"]
  node_lead123d_download["Lead123D<br/>downloader"]
end

subgraph group_platform["Platform"]
  node_common_io["Common IO<br/>codec layer"]
  node_execution["Execution<br/>scheduler backends"]
  node_common_utils["Common Utils<br/>shared helpers"]
end

subgraph group_orchestration["Orchestration"]
  node_script_cli["CLI Scripts<br/>entrypoints"]
  node_script_configs["Configs<br/>hydra config"]
end

subgraph group_viz["Visualization"]
  node_visualization["Visualization<br/>rendering"]
end

node_av2_parser -->|"normalize into"| node_datatypes
node_nuplan_parser -->|"normalize into"| node_datatypes
node_nuscenes_parser -->|"normalize into"| node_datatypes
node_kitti360_parser -->|"normalize into"| node_datatypes
node_pandaset_parser -->|"normalize into"| node_datatypes
node_wod_parser -->|"normalize into"| node_datatypes
node_opendrive_parser -->|"build map geometry"| node_geometry
node_opendrive_parser -->|"emit map objects"| node_datatypes
node_parser_registry -->|"dispatches"| node_av2_parser
node_parser_registry -->|"dispatches"| node_nuplan_parser
node_parser_registry -->|"dispatches"| node_nuscenes_parser
node_parser_registry -->|"dispatches"| node_opendrive_parser
node_scene_arrow -->|"serializes"| node_datatypes
node_scene_api -->|"reads"| node_scene_arrow
node_map_arrow -->|"serializes"| node_datatypes
node_map_api -->|"reads"| node_map_arrow
node_common_io -->|"feeds media"| node_scene_arrow
node_common_io -->|"supports ingestion"| node_parser_registry
node_execution -->|"backends for"| node_script_cli
node_script_cli -->|"loads"| node_script_configs
node_script_cli -->|"orchestrates"| node_parser_registry
node_script_cli -->|"writes"| node_scene_arrow
node_script_cli -->|"writes"| node_map_arrow
node_visualization -->|"consumes"| node_scene_api
node_visualization -->|"consumes"| node_map_api
node_scene_arrow -->|"uses"| node_common_utils
node_map_arrow -->|"uses"| node_common_utils
node_lead123d_download -.->|"invoked by"| node_script_cli

click node_datatypes "https://github.com/kesai-labs/py123d/tree/main/src/py123d/datatypes"
click node_geometry "https://github.com/kesai-labs/py123d/tree/main/src/py123d/geometry"
click node_scene_api "https://github.com/kesai-labs/py123d/blob/main/src/py123d/api/scene/scene_api.py"
click node_scene_arrow "https://github.com/kesai-labs/py123d/tree/main/src/py123d/api/scene/arrow"
click node_map_api "https://github.com/kesai-labs/py123d/blob/main/src/py123d/api/map/map_api.py"
click node_map_arrow "https://github.com/kesai-labs/py123d/tree/main/src/py123d/api/map/arrow"
click node_parser_registry "https://github.com/kesai-labs/py123d/blob/main/src/py123d/parser/registry.py"
click node_av2_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/av2"
click node_nuplan_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/nuplan"
click node_nuscenes_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/nuscenes"
click node_kitti360_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/kitti360"
click node_pandaset_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/pandaset"
click node_wod_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/wod"
click node_opendrive_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/opendrive"
click node_physical_ai_av_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/physical_ai_av"
click node_ncore_parser "https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/ncore"
click node_lead123d_download "https://github.com/kesai-labs/py123d/blob/main/src/py123d/parser/lead123d/lead123d_download.py"
click node_common_io "https://github.com/kesai-labs/py123d/tree/main/src/py123d/common/io"
click node_execution "https://github.com/kesai-labs/py123d/tree/main/src/py123d/common/execution"
click node_common_utils "https://github.com/kesai-labs/py123d/tree/main/src/py123d/common/utils"
click node_script_cli "https://github.com/kesai-labs/py123d/tree/main/src/py123d/script"
click node_script_configs "https://github.com/kesai-labs/py123d/tree/main/src/py123d/script/config"
click node_visualization "https://github.com/kesai-labs/py123d/tree/main/src/py123d/visualization"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datatypes,node_geometry toneBlue
class node_scene_api,node_scene_arrow,node_map_api,node_map_arrow toneAmber
class node_parser_registry,node_av2_parser,node_nuplan_parser,node_nuscenes_parser,node_kitti360_parser,node_pandaset_parser,node_wod_parser,node_opendrive_parser,node_physical_ai_av_parser,node_ncore_parser,node_lead123d_download toneMint
class node_common_io,node_execution,node_common_utils toneRose
class node_script_cli,node_script_configs toneIndigo
class node_visualization toneTeal
```

## Components

### Core model

Canonical datatypes and geometry used across ingestion, storage, and visualization.

- [Datatypes](https://github.com/kesai-labs/py123d/tree/main/src/py123d/datatypes): canonical schema · Shared domain objects for sensors, detections, metadata, time, and vehicle state. · `src/py123d/datatypes`
- [Geometry](https://github.com/kesai-labs/py123d/tree/main/src/py123d/geometry): spatial primitives · Points, vectors, rotations, transforms, boxes, polylines, poses, and occupancy maps. · `src/py123d/geometry`

### Runtime API

Public read/write APIs for scenes and maps, backed by Arrow storage.

- [Scene API](https://github.com/kesai-labs/py123d/blob/main/src/py123d/api/scene/scene_api.py): public read API · Dataset-agnostic access to logs, scenes, modalities, and scene metadata. · `src/py123d/api/scene/scene_api.py`
- [Scene Arrow](https://github.com/kesai-labs/py123d/tree/main/src/py123d/api/scene/arrow): arrow reader/writer · Arrow-backed scene builders, log writers, and modality handlers. · `src/py123d/api/scene/arrow`
- [Map API](https://github.com/kesai-labs/py123d/blob/main/src/py123d/api/map/map_api.py): public map API · Dataset-agnostic access to map content and layers. · `src/py123d/api/map/map_api.py`
- [Map Arrow](https://github.com/kesai-labs/py123d/tree/main/src/py123d/api/map/arrow): arrow map I/O · Arrow-backed map writer and reader for persisted map assets. · `src/py123d/api/map/arrow`

### Ingestion

Dataset adapters and map parsers that normalize external sources into the internal model.

- [Parser Registry](https://github.com/kesai-labs/py123d/blob/main/src/py123d/parser/registry.py): dataset routing · Registers and dispatches dataset-specific parsers and downloaders. · `src/py123d/parser/registry.py`
- [AV2](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/av2): dataset parser · Argoverse 2 ingestion, sensor I/O, and map parsing. · `src/py123d/parser/av2`
- [nuPlan](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/nuplan): dataset parser · nuPlan conversion, sensor I/O, and map parsing. · `src/py123d/parser/nuplan`
- [nuScenes](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/nuscenes): dataset parser · nuScenes conversion, sensor I/O, and map parsing. · `src/py123d/parser/nuscenes`
- [KITTI-360](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/kitti360): dataset parser · KITTI-360 ingestion and map parsing. · `src/py123d/parser/kitti360`
- [PandaSet](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/pandaset): dataset parser · PandaSet ingestion and sensor I/O. · `src/py123d/parser/pandaset`
- [WOD](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/wod): dataset parser · Waymo Open Dataset perception and motion ingestion. · `src/py123d/parser/wod`
- [OpenDRIVE](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/opendrive): map parser · CARLA/OpenDRIVE map ingestion and XODR reconstruction. · `src/py123d/parser/opendrive`
- [Physical AI AV](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/physical_ai_av): dataset parser · Physical AI AV ingestion and sensor I/O. · `src/py123d/parser/physical_ai_av`
- [NCore](https://github.com/kesai-labs/py123d/tree/main/src/py123d/parser/ncore): dataset parser · NCore ingestion and sensor I/O. · `src/py123d/parser/ncore`
- [Lead123D](https://github.com/kesai-labs/py123d/blob/main/src/py123d/parser/lead123d/lead123d_download.py): downloader · Dataset download entry for Lead123D. · `src/py123d/parser/lead123d/lead123d_download.py`

### Platform

Cross-cutting utilities for IO, execution, and shared helpers.

- [Common IO](https://github.com/kesai-labs/py123d/tree/main/src/py123d/common/io): codec layer · Shared camera and lidar codecs plus dataset path handling and utilities. · `src/py123d/common/io`
- [Execution](https://github.com/kesai-labs/py123d/tree/main/src/py123d/common/execution): scheduler backends · Sequential, threaded, process, and Ray execution backends. · `src/py123d/common/execution`
- [Common Utils](https://github.com/kesai-labs/py123d/tree/main/src/py123d/common/utils): shared helpers · Dependency gating, UUID/timing helpers, mixins, and messaging utilities. · `src/py123d/common/utils`

### Orchestration

CLI entrypoints and Hydra-based configuration for download, conversion, and viewing.

- [CLI Scripts](https://github.com/kesai-labs/py123d/tree/main/src/py123d/script): entrypoints · Download, conversion, and viewer command entrypoints. · `src/py123d/script`
- [Configs](https://github.com/kesai-labs/py123d/tree/main/src/py123d/script/config): hydra config · Hydra presets for datasets, conversion, execution, scene filters, and viewer settings. · `src/py123d/script/config`

### Visualization

Static and interactive consumers of the public scene/map APIs.

- [Visualization](https://github.com/kesai-labs/py123d/tree/main/src/py123d/visualization): rendering · Matplotlib plots and interactive Viser viewer components. · `src/py123d/visualization`

## Connections

- AV2 → Datatypes: normalize into · Maps AV2-specific records into canonical domain objects.
- nuPlan → Datatypes: normalize into · Maps nuPlan records into canonical domain objects.
- nuScenes → Datatypes: normalize into · Maps nuScenes records into canonical domain objects.
- KITTI-360 → Datatypes: normalize into · Maps KITTI-360 records into canonical domain objects.
- PandaSet → Datatypes: normalize into · Maps PandaSet records into canonical domain objects.
- WOD → Datatypes: normalize into · Maps Waymo Open Dataset records into canonical domain objects.
- OpenDRIVE → Geometry: build map geometry · Reconstructs lane and road geometry from OpenDRIVE assets.
- OpenDRIVE → Datatypes: emit map objects · Converts parsed map features into canonical map objects.
- Parser Registry → AV2: dispatches · Routes AV2 dataset jobs to the AV2 adapter.
- Parser Registry → nuPlan: dispatches · Routes nuPlan dataset jobs to the nuPlan adapter.
- Parser Registry → nuScenes: dispatches · Routes nuScenes dataset jobs to the nuScenes adapter.
- Parser Registry → OpenDRIVE: dispatches · Routes CARLA/OpenDRIVE map jobs to the dedicated parser.
- Scene Arrow → Datatypes: serializes · Persists canonical scene data into Arrow-backed storage.
- Scene API → Scene Arrow: reads · Public scene access is served from Arrow scene storage.
- Map Arrow → Datatypes: serializes · Persists canonical map data into Arrow-backed storage.
- Map API → Map Arrow: reads · Public map access is served from Arrow map storage.
- Common IO → Scene Arrow: feeds media · Camera and lidar codecs support the scene write/read path.
- Common IO → Parser Registry: supports ingestion · Shared file and asset IO is reused during dataset parsing.
- Execution → CLI Scripts: backends for · Provides selectable executors for conversion and preprocessing.
- CLI Scripts → Configs: loads · CLI entrypoints are configured through Hydra presets.
- CLI Scripts → Parser Registry: orchestrates · Download and conversion commands invoke dataset parsers.
- CLI Scripts → Scene Arrow: writes · Conversion commands build Arrow scene logs.
- CLI Scripts → Map Arrow: writes · Conversion commands persist map artifacts.
- Visualization → Scene API: consumes · Static and interactive viewers consume the public scene API.
- Visualization → Map API: consumes · Map rendering uses the public map API.
- Scene Arrow → Common Utils: uses · Scene builders share timing, UUID, and dependency helpers.
- Map Arrow → Common Utils: uses · Map writers share common helper utilities.
- Lead123D → CLI Scripts: invoked by · Dataset download flow is surfaced through the CLI layer.
