# zftto/banana_autopilot architecture

> Architecture diagram of the GitHub repository zftto/banana_autopilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zftto/banana_autopilot
- Repository: https://github.com/zftto/banana_autopilot

## Overview

Banana AutoPilot is an educational ESP32-S3 driving project. Its implemented workflow takes manual keyboard or Switch input, streams camera frames, sends drive commands to the vehicle, and optionally records images with matching command labels; sessions can then be replayed locally. The README also describes a planned Colab behavior-cloning workflow and local autonomous inference, but marks the BC modules and autonomous workflow as placeholders. Those documented stages are included without implying they work. Relationships for recording are supported by excerpts; learning and replay links rely on README descriptions.

Main entry points: [`workflows/recording.py`](https://github.com/zftto/banana_autopilot/blob/main/workflows/recording.py) (Recording Workflow), [`colab_notebooks/BananaTrain_0.ipynb`](https://github.com/zftto/banana_autopilot/blob/main/colab_notebooks/BananaTrain_0.ipynb) (Teaching Notebooks), [`core/controller.py`](https://github.com/zftto/banana_autopilot/blob/main/core/controller.py) (Controller Input), [`datasets`](https://github.com/zftto/banana_autopilot/tree/main/datasets) (Recorded Sessions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Manual Capture"]
  node_recording["Recording Workflow<br/>[recording.py]"]
  node_settings["Connection Settings<br/>[esp32.py]"]
  node_input["Controller Input<br/>[controller.py]"]
  node_drivecommand["Drive Command<br/>[command.py]"]
  node_camera["Camera Stream<br/>[camera_stream.py]"]
  node_decoder["JPEG Decoder<br/>[camera_stream.py]"]
  node_controltransport["Vehicle Control Client"]
  node_display["Live Display<br/>[display.py]"]
end

subgraph group_dataset["Session Review"]
  node_writer["Session Writer<br/>[dataset_writer.py]"]
  node_sessions[("Recorded Sessions")]
  node_replay["Session Replay<br/>[replay.py]"]
end

subgraph group_learning["Learning And Autonomy"]
  node_colab["Teaching Notebooks"]
  node_loader["BC Data Loading<br/>[dataset.py]"]
  node_model["BC Model<br/>[model.py]"]
  node_training["BC Training<br/>[train.py]"]
  node_evaluation["BC Evaluation<br/>[evaluate.py]"]
  node_inferenceconfig["Inference Settings<br/>[inference.py]"]
  node_autonomy["Autonomous Workflow"]
end

node_driver(("Driver"))
node_vehicle["ESP32-S3"]

node_driver -->|"provides input"| node_input
node_recording -->|"reads state"| node_input
node_input -->|"creates command"| node_drivecommand
node_recording -->|"loads settings"| node_settings
node_recording -->|"starts and reads"| node_camera
node_recording -->|"decodes frames"| node_decoder
node_recording -->|"sends commands"| node_controltransport
node_recording -->|"records samples"| node_writer
node_recording -->|"draws status"| node_display
node_camera -->|"receives stream"| node_vehicle
node_controltransport -->|"sends control"| node_vehicle
node_writer -->|"writes images and labels"| node_sessions
node_replay -->|"reads session"| node_sessions
node_colab -.->|"teaches data loading"| node_loader
node_loader -.->|"loads transferred data"| node_sessions
node_colab -.->|"teaches model design"| node_model
node_colab -.->|"teaches training"| node_training
node_colab -.->|"teaches evaluation"| node_evaluation
node_colab -.->|"exports settings"| node_inferenceconfig
node_autonomy -.->|"uses trained model"| node_model
node_autonomy -.->|"uses settings"| node_inferenceconfig

click node_recording "https://github.com/zftto/banana_autopilot/blob/main/workflows/recording.py"
click node_settings "https://github.com/zftto/banana_autopilot/blob/main/configs/esp32.py"
click node_input "https://github.com/zftto/banana_autopilot/blob/main/core/controller.py"
click node_drivecommand "https://github.com/zftto/banana_autopilot/blob/main/core/command.py"
click node_camera "https://github.com/zftto/banana_autopilot/blob/main/core/camera_stream.py"
click node_decoder "https://github.com/zftto/banana_autopilot/blob/main/core/camera_stream.py"
click node_controltransport "https://github.com/zftto/banana_autopilot/blob/main/core/websocket_client.py"
click node_display "https://github.com/zftto/banana_autopilot/blob/main/core/display.py"
click node_writer "https://github.com/zftto/banana_autopilot/blob/main/core/dataset_writer.py"
click node_sessions "https://github.com/zftto/banana_autopilot/tree/main/datasets"
click node_replay "https://github.com/zftto/banana_autopilot/blob/main/workflows/replay.py"
click node_colab "https://github.com/zftto/banana_autopilot/blob/main/colab_notebooks/BananaTrain_0.ipynb"
click node_loader "https://github.com/zftto/banana_autopilot/blob/main/bc/dataset.py"
click node_model "https://github.com/zftto/banana_autopilot/blob/main/bc/model.py"
click node_training "https://github.com/zftto/banana_autopilot/blob/main/bc/train.py"
click node_evaluation "https://github.com/zftto/banana_autopilot/blob/main/bc/evaluate.py"
click node_inferenceconfig "https://github.com/zftto/banana_autopilot/blob/main/configs/inference.py"
click node_autonomy "https://github.com/zftto/banana_autopilot/blob/main/workflows/autonomous_driving.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_recording,node_settings,node_input,node_drivecommand,node_camera,node_decoder,node_controltransport,node_display toneBlue
class node_writer,node_sessions,node_replay toneAmber
class node_colab,node_loader,node_model,node_training,node_evaluation,node_inferenceconfig,node_autonomy toneMint
class node_driver,node_vehicle toneIndigo
```

## Components

### Manual Capture

- [Recording Workflow](https://github.com/zftto/banana_autopilot/blob/main/workflows/recording.py): `workflows/recording.py`
- [Connection Settings](https://github.com/zftto/banana_autopilot/blob/main/configs/esp32.py): `configs/esp32.py`
- [Controller Input](https://github.com/zftto/banana_autopilot/blob/main/core/controller.py): `core/controller.py`
- [Drive Command](https://github.com/zftto/banana_autopilot/blob/main/core/command.py): `core/command.py`
- [Camera Stream](https://github.com/zftto/banana_autopilot/blob/main/core/camera_stream.py): `core/camera_stream.py`
- [JPEG Decoder](https://github.com/zftto/banana_autopilot/blob/main/core/camera_stream.py): `core/camera_stream.py`
- [Vehicle Control Client](https://github.com/zftto/banana_autopilot/blob/main/core/websocket_client.py): `core/websocket_client.py`
- [Live Display](https://github.com/zftto/banana_autopilot/blob/main/core/display.py): `core/display.py`

### Session Review

- [Session Writer](https://github.com/zftto/banana_autopilot/blob/main/core/dataset_writer.py): `core/dataset_writer.py`
- [Recorded Sessions](https://github.com/zftto/banana_autopilot/tree/main/datasets): `datasets`
- [Session Replay](https://github.com/zftto/banana_autopilot/blob/main/workflows/replay.py): `workflows/replay.py`

### Learning And Autonomy

- [Teaching Notebooks](https://github.com/zftto/banana_autopilot/blob/main/colab_notebooks/BananaTrain_0.ipynb): `colab_notebooks/BananaTrain_0.ipynb`
- [BC Data Loading](https://github.com/zftto/banana_autopilot/blob/main/bc/dataset.py): `bc/dataset.py`
- [BC Model](https://github.com/zftto/banana_autopilot/blob/main/bc/model.py): `bc/model.py`
- [BC Training](https://github.com/zftto/banana_autopilot/blob/main/bc/train.py): `bc/train.py`
- [BC Evaluation](https://github.com/zftto/banana_autopilot/blob/main/bc/evaluate.py): `bc/evaluate.py`
- [Inference Settings](https://github.com/zftto/banana_autopilot/blob/main/configs/inference.py): `configs/inference.py`
- [Autonomous Workflow](https://github.com/zftto/banana_autopilot/blob/main/workflows/autonomous_driving.py): `workflows/autonomous_driving.py`

### Other components

- **Driver**
- **ESP32-S3**

## Connections

- Driver → Controller Input: provides input (evidence: `README.md`)
- Recording Workflow → Controller Input: reads state (evidence: `workflows/recording.py`)
- Controller Input → Drive Command: creates command (evidence: `core/controller.py`)
- Recording Workflow → Connection Settings: loads settings (evidence: `workflows/recording.py`)
- Recording Workflow → Camera Stream: starts and reads (evidence: `workflows/recording.py`)
- Recording Workflow → JPEG Decoder: decodes frames (evidence: `workflows/recording.py`)
- Recording Workflow → Vehicle Control Client: sends commands (evidence: `workflows/recording.py`)
- Recording Workflow → Session Writer: records samples (evidence: `workflows/recording.py`)
- Recording Workflow → Live Display: draws status (evidence: `workflows/recording.py`)
- Camera Stream → ESP32-S3: receives stream (evidence: `README.md`)
- Vehicle Control Client → ESP32-S3: sends control (evidence: `README.md`)
- Session Writer → Recorded Sessions: writes images and labels (evidence: `core/dataset_writer.py`)
- Session Replay → Recorded Sessions: reads session (evidence: `README.md`)
- Teaching Notebooks → BC Data Loading: teaches data loading (evidence: `README.md`)
- BC Data Loading → Recorded Sessions: loads transferred data (evidence: `README.md`)
- Teaching Notebooks → BC Model: teaches model design (evidence: `README.md`)
- Teaching Notebooks → BC Training: teaches training (evidence: `README.md`)
- Teaching Notebooks → BC Evaluation: teaches evaluation (evidence: `README.md`)
- Teaching Notebooks → Inference Settings: exports settings (evidence: `README.md`)
- Autonomous Workflow → BC Model: uses trained model
- Autonomous Workflow → Inference Settings: uses settings
