How waymo-research/waymo-open-dataset works
Project type and purpose
• This is an open-source evaluation library and toolkit for the Waymo Open Dataset. It defines the dataset format (via Protobuf), provides data-conversion utilities, implements core math and common routines, exposes evaluation metrics (in C++ with Python bindings), a latency-evaluation workflow, pip packaging scripts, and end-user tutorials/examples.
File structure
• Top-level:
• docs/ → design, labeling and metric specifications, Docker usage
• src/ → main codebase, Bazel workspace, build configuration
• tutorial/ → Jupyter notebooks and Dockerfile for hands-on demos
• Under src/waymo_open_dataset/:
• protos/ → all .proto definitions for data and submission formats
• common/, math/ → low-level C++ helpers and utilities
• data_conversion/→ convert raw scenario to internal TFRecord format
• metrics/ →
– core C++ metric implementations and tests
– ops/ → custom TensorFlow/PyTorch ops in C++ + Python wrappers
– python/ → high-level Python metric APIs and tests
– tools/ → CLI tools for batch metric computation and submission generation
• latency/ → scripts and examples (TF/PyTorch, Docker) to measure model inference time
• v2/ → a pandas-style DataFrame API for dataset access and “compat_v1” adapters
• wdl_limited/ → licensed submodules for camera, segmentation, sim_agent metrics
• pip_pkg_scripts/→ Dockerized build and packaging for PyPI distribution
Insights from README
• Confirms three dataset families (Perception, Motion, End-to-End)
• Evaluation code and helper functions in TensorFlow and Python
• Licensing constraints on wdl_limited subfolders