How zftto/banana_autopilot works
Banana AutoPilot is an educational ESP32-S3 driving project. Its implemented workflow takes manual keyboard or Switch input, streams camera frames, sends drive commands to the vehicle, and optionally records images with matching command labels; sessions can then be replayed locally. The README also describes a planned Colab behavior-cloning workflow and local autonomous inference, but marks the BC modules and autonomous workflow as placeholders. Those documented stages are included without implying they work. Relationships for recording are supported by excerpts; learning and replay links rely on README descriptions.
Main entry points: recording.py Recording Workflow · BananaTrain_0.ipynb Teaching Notebooks · controller.py Controller Input · datasets Recorded Sessions
Components 20
Manual Capture
- Recording Workflow
workflows/recording.py - Connection Settings
configs/esp32.py - Controller Input
core/controller.py - Drive Command
core/command.py - Camera Stream
core/camera_stream.py - JPEG Decoder
core/camera_stream.py - Vehicle Control Client
core/websocket_client.py - Live Display
core/display.py
Session Review
- Session Writer
core/dataset_writer.py - Recorded Sessions
datasets - Session Replay
workflows/replay.py
Learning And Autonomy
- Teaching Notebooks
colab_notebooks/BananaTrain_0.ipynb - BC Data Loading
bc/dataset.py - BC Model
bc/model.py - BC Training
bc/train.py - BC Evaluation
bc/evaluate.py - Inference Settings
configs/inference.py - Autonomous Workflow
workflows/autonomous_driving.py
Other components
- Driver
- ESP32-S3
Connections 21
- Driver → Controller Input: provides input
README.md - Recording Workflow → Controller Input: reads state
workflows/recording.py - Controller Input → Drive Command: creates command
core/controller.py - Recording Workflow → Connection Settings: loads settings
workflows/recording.py - Recording Workflow → Camera Stream: starts and reads
workflows/recording.py - Recording Workflow → JPEG Decoder: decodes frames
workflows/recording.py - Recording Workflow → Vehicle Control Client: sends commands
workflows/recording.py - Recording Workflow → Session Writer: records samples
workflows/recording.py - Recording Workflow → Live Display: draws status
workflows/recording.py - Camera Stream → ESP32-S3: receives stream
README.md - Vehicle Control Client → ESP32-S3: sends control
README.md - Session Writer → Recorded Sessions: writes images and labels
core/dataset_writer.py - Session Replay → Recorded Sessions: reads session
README.md - Teaching Notebooks → BC Data Loading: teaches data loading
README.md - BC Data Loading → Recorded Sessions: loads transferred data
README.md - Teaching Notebooks → BC Model: teaches model design
README.md - Teaching Notebooks → BC Training: teaches training
README.md - Teaching Notebooks → BC Evaluation: teaches evaluation
README.md - Teaching Notebooks → Inference Settings: exports settings
README.md - Autonomous Workflow → BC Model: uses trained model
- Autonomous Workflow → Inference Settings: uses settings