How j-vanetten/openpilot works
Project type and purpose
• This is an embedded robotics operating system (“openpilot”) for advanced driver-assistance and semi-autonomous control in production vehicles.
• It runs as a collection of cooperating daemons on a comma AI device (comma 3X, etc.) inside a car, interfacing with sensors, vehicle CAN bus, and providing real-time control, vision, and UI.
File structure
• Top-level modules:
• system/ – low-level device daemons (hardwared, sensord, camerad, loggerd, locationd, modeld, controlsd, etc.)
• panda/ – the safety-certified board firmware and host driver (“Panda”) that sits on the CAN bus
• openpilot/ – high-level application code (selfdrive, UI, tools)
• cereal/ and msgq/ – Cap’n Proto definitions and messaging bridge libraries
• opendbc/ – automotive CAN database definitions and code generators
• common/ – shared utility libraries (logging, parameters, math, I/O)
• Build and deployment:
• SConstruct, Dockerfiles, Jenkinsfile, pyproject.toml
a. Hardware layer
– Vehicle sensors (camera, IMU, radar via DCAN/FD-CAN)
– Panda board – safety MCU + CAN interface
– Host SoC running Linux
b. System daemons (processes)
– sensord (IMU/GPS), camerad (camera capture), locationd (pose & calibration), modeld (neural network inference), controlsd/plannerd (trajectory planning & control), loggerd (data logging), manager (process supervision), ui (Qt-based user interface), pandad (host-side comms with Panda)
c. Middleware/message bus
– Cap’n Proto pub-sub via cereal/messaging bridging to ZeroMQ or local sockets
d. Data stores & config
– On-disk logs, parameter store (common/params), DBC definitions (opendbc)
e. External interfaces
– Wi-Fi/ethernet OTA updates, cloud telemetry (Athena), developer tools (tools/)
Relationships and data flow
• Sensors → sensord / camerad → publish CAN and video frames → modeld
• modeld → outputs detection, lane, lead car → plannerd (trajectory)
• plannerd → controlsd (lateral/longitudinal) → actuator commands → pandad → Panda → vehicle CAN
• manager → starts/stops all daemons, health checks
• loggerd subscribes to all topics to persist logs
• ui subscribes to key topics (live video, controls state, alerts) to render HMI
• locationd provides localization services to modeld and controlsd
• external Athena updater and registration modules run alongside for cloud sync
Architectural patterns and principles
• Microservices-style: many single-responsibility daemons communicating over a message bus
• Pub-sub messaging: decoupled producers/consumers via Cap’n Proto + ZeroMQ
• Safety domains: Panda enforces ISO 26262 safety rules in C, independent of host code
• Configuration & extensibility: DBC code generation (opendbc), Cap’n Proto schema (cereal), plugin-style “car ports” under selfdrive/car
• Build automation: SCons and Docker ensure reproducible cross-compilation
Key technologies, frameworks, and libraries
• Cap’n Proto schemas (cereal/*.capnp) and C++/Python codegen
• Python and C++ daemons, QML/Qt for UI
• SCons build system, Docker for containerized builds
• OpenCV, Eigen and custom CUDA/OpenCL runners for model inference
• ZeroMQ bridging layer (msgq)
• Flask or internal HTTP for developer tools (tools/)