How feiyang2025/jetson-cuda-overlay works
This repository is a Jetson CUDA/V4L2 overlay for openpilot-family forks. Its primary runtime path captures GMSL camera frames, converts them to NV12, and feeds model inference; model selection supports split engines and the merged BigCombo engine, with TensorRT support and documented fallback behavior. Separate capabilities cover camera calibration, optional CH347 IMU publishing, UI overlays, and cross-fork Panda protocol/firmware maintenance. Some subsystem links are documented only in the README; unsampled implementation details are not inferred.
Main entry points: env.py Fork discovery · camerad.py Camera daemon · modeld.py Model dispatcher · v4l2_dmabuf_camera.py V4L2 capture
Components 21
Camera pipeline
- V4L2 capture
openpilot/system/camerad/webcam/v4l2_dmabuf_camera.py - Camera daemon
openpilot/system/camerad/webcam/camerad.py - CUDA conversion
openpilot/system/camerad/webcam/camera_cuda.py
Model inference
- Model dispatcher
kits/modeld/modeld.py - BigCombo inference
kits/modeld/modeld_bigcombo.py - TensorRT runner
openpilot/selfdrive/modeld/runners/tensorrt_runner.py - GPU backend
openpilot/sunnypilot/modeld_v2/gpu_backend/backend.py - Model profiles
openpilot/sunnypilot/modeld_v2/gpu_backend/models/registry.py - GPU model state
openpilot/sunnypilot/modeld_v2/gpu_model_state.py
Calibration tools
- Camera calibration
openpilot/tools/calib - UI overlays
kits/ui/calibration_panel.py - Speed-limit UI
kits/ui/speed_limit.py
Sensor integration
- CH347 IMU daemon
openpilot/system/sensord/ch347t.cc - IMU consumers
Panda maintenance
- Fork discovery
panda/env.py - Protocol repair
panda/repair_protocol.py - Firmware sync
panda/sync_firmware.py - Protocol verification
panda/verify_protocol.py
Other components
- Camera hardware
- Calibration parameters
- Openpilot forks
Connections 15
- Camera hardware → V4L2 capture: supplies frames
- V4L2 capture → Camera daemon: provides frames
openpilot/system/camerad/webcam/camerad.py - Camera daemon → CUDA conversion: converts frames
- Camera daemon → Model dispatcher: sends vision frames
- Model dispatcher → BigCombo inference: dispatches BigCombo
kits/modeld/modeld.py - Model dispatcher → TensorRT runner: loads TensorRT
kits/modeld/modeld.py - BigCombo inference → TensorRT runner: loads engine
kits/modeld/modeld_bigcombo.py - GPU backend → Model profiles: resolves models
- GPU model state → GPU backend: uses backend
- Camera calibration → Calibration parameters: reads and writes
- CH347 IMU daemon → IMU consumers: publishes samples
- Protocol repair → Fork discovery: discovers forks
panda/repair_protocol.py - Firmware sync → Fork discovery: discovers forks
panda/sync_firmware.py - Protocol verification → Fork discovery: discovers forks
panda/verify_protocol.py - Fork discovery → Openpilot forks: scans forks
panda/env.py