How zesaen/voxdrive works
VoxDrive is a distributed in-vehicle system: RK3588 handles recording and edge voice recognition, while Jetson provides intent routing, vehicle knowledge and tool services, language/vision answers, speech output, and the dashboard. In the voice workflow, RK recognition sends text to Jetson’s router, which classifies it and invokes the relevant RAG, LLM, vehicle-tool, or VLM service; responses can be spoken through TTS and surfaced in the dashboard. The recorder independently captures, encodes, stores, and streams video, exposing status, events, controls, and snapshots across boards. Some subsystem wiring is documented in the README but has no implementation file in the supplied tree; those nodes use only supported paths or null.
Main entry points: intent_router/main.cpp Intent Router · dashboard_ui.py Dashboard · recorder_service.cpp Recorder Service · tool_bus/main.cpp Tool Bus
Components 24
RK Edge
- Edge ASR
rk/voice/voice_service.py - Recorder Service
rk/service/recorder_service.cpp - Capture Encode
rk/service/recorder_service.cpp - Segment Storage
rk/service/recorder_service.cpp - Edge TTS
Query Routing
- Intent Router
jetson/services/intent_router/main.cpp - Query Classifier
jetson/services/intent_router/query_classifier.cpp
Answer Services
- Manual RAG
jetson/services/rag/rag_server.py - Manual Vectors
jetson/services/rag/python/vehicle_vector_search.py - LLM Service
jetson/services/llm/llm_server.py - Visual Q&A
jetson/services/vlm/vlm_service.py - TTS Gateway
jetson/services/tts/tts_server/src/main.cpp
Vehicle Operations
- Tool Bus
jetson/services/tool_bus/main.cpp - Vehicle State
jetson/services/tool_bus/StateManager.h
Cabin Interface
- Dashboard
jetson/dashboard/dashboard_ui.py - Dashcam View
jetson/dashboard/plugins/dashcam.py - Vehicle View
jetson/dashboard/plugins/vehicle.py - Status View
jetson/dashboard/plugins/status.py
Other components
- Driver
- Microphone
- Camera Input
- RTMP Stream
- Media Server
- Speaker
Connections 29
- Driver → Microphone: speaks
- Microphone → Edge ASR: audio
- Edge ASR → Intent Router: recognized text
- Intent Router → Query Classifier: classifies query
- Intent Router → Manual RAG: manual query
- Manual RAG → Manual Vectors: searches
- Intent Router → LLM Service: query and context
- LLM Service → Intent Router: answer or tool call
- Intent Router → Tool Bus: dispatches tool
- Tool Bus → Vehicle State: reads and writes
- Tool Bus → Recorder Service: forwards dashcam commands
- Camera Input → Capture Encode: video frames
- Capture Encode → Segment Storage: writes segments
- Capture Encode → RTMP Stream: sends encoded video
- RTMP Stream → Media Server: publishes stream
- Recorder Service → Intent Router: status and events
- Intent Router → Visual Q&A: visual question
- Visual Q&A → Recorder Service: requests snapshot
- Visual Q&A → LLM Service: manages model swap
- Intent Router → TTS Gateway: sends answer text
- TTS Gateway → Edge TTS: synthesis request
- Edge TTS → Speaker: plays speech
- Intent Router → Dashboard: publishes speech and query status
- Dashboard → Dashcam View: hosts
- Dashboard → Vehicle View: hosts
- Dashboard → Status View: hosts
- Recorder Service → Dashboard: status and alerts
- Tool Bus → Dashboard: state updates
- Dashboard → Recorder Service: dashcam controls