# jugpanda/pickuppal architecture

> Architecture diagram of the GitHub repository jugpanda/pickuppal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jugpanda/pickuppal
- Repository: https://github.com/jugpanda/pickuppal

## Overview

1. Project shape  
PickupPal is a robotics capstone repo centered on an autonomous “pickup” system. The repository is split into software for perception and decision-making, firmware for the embedded controller, and large image datasets/model artifacts for training and deployment.

2. Main runtime boundary: Python vs. Arduino  
The clearest architectural boundary is between Python scripts under `src/object-detection` / `src/yolo` and the Arduino firmware under `src/pickup-arduino`. Python appears to run computer vision and higher-level autonomy, while the Arduino handles low-level actuation and sensor I/O for the robot.

3. Perception subsystem  
Object detection is the core perception layer. It uses Ultralytics YOLO models and associated assets such as `best.pt`, `yolo11n.pt`, `yolov8*.pt`, and a TorchScript/NCNN export. The scripts `yolo_detect.py`, `yolo.py`, and `pickup-learn.py` suggest inference, experiment/training, and possibly robot interaction during learning or testing.

4. Autonomous control subsystem  
`src/object-detection/autonomous.py` is the likely orchestration entry point for autonomous behavior. In architecture terms, it sits above perception and below operator interaction, turning detections into pickup/navigation decisions, likely by sending commands to the embedded controller.

5. Embedded control subsystem  
`src/pickup-arduino/autonomous.ino` is the firmware boundary. It likely owns motors, servos, and sensors at real-time timescales. The Python layer should be treated as an upstream controller, with Arduino responsible for deterministic hardware actions and feedback.

6. Hardware validation and integration tests  
`src/tests/camera_test.py`, `motor_test.py`, and `sensor_test.py` indicate separate hardware test paths rather than a single unified test harness. That implies the system was developed and validated component-by-component: camera input, motor output, and sensor readings.

7. Training data pipeline  
There are two dataset areas: `src/object-detection` with `data.yaml` plus `data_original.zip`, and `src/yolo/Lego` with train/valid/test splits and labels. The presence of `README.dataset.txt` and `README.roboflow.txt` files suggests the datasets were prepared/imported via Roboflow and then organized for YOLO training.

8. Model lifecycle and conversion  
The repo contains multiple model forms: raw PyTorch weights (`.pt`), a TorchScript export (`yolo11n.torchscript`), and an NCNN converted model directory (`yolo11n_ncnn_model`). That shows a deployment pipeline from training weights to optimized inference formats for edge or embedded-friendly execution.

9. Generated outputs and experiment artifacts  
`src/yolo/runs/detect/predict` looks like a generated inference output folder rather than source code. It indicates the team ran local prediction experiments and saved results separately from the main code paths.

10. Likely end-to-end data flow  
A reasonable architecture diagram would show: camera frames enter Python inference, YOLO produces detections, autonomous logic decides pickup behavior, and commands are sent to the Arduino controller, which drives motors and reads sensors. Sensor feedback then closes the loop back to the autonomy layer for continued operation.

11. What is intentionally not present  
There is no sign of a web backend, database, or cloud service. This is primarily an offline/edge robotics system with local datasets, local model files, and hardware-attached control rather than a networked application.

12. Documentation and project context  
The README is minimal, but the repo layout and notes point to a capstone build with split responsibilities: programming/circuitry, 3D modeling, and documentation. That matters architecturally because the software stack is only one part of a broader electromechanical system.

Main entry points: [`src/yolo/yolo_detect.py`](https://github.com/jugpanda/pickuppal/blob/main/src/yolo/yolo_detect.py) (Detect), [`src/pickup-arduino/autonomous.ino`](https://github.com/jugpanda/pickuppal/blob/main/src/pickup-arduino/autonomous.ino) (Arduino), [`src/object-detection/best.pt`](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/best.pt) (Best Weights), [`src/object-detection/autonomous.py`](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/autonomous.py) (Autonomous)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_python["Python stack"]
  node_node_autonomous_py["Autonomous<br/>orchestrator<br/>[autonomous.py]"]
  node_node_yolo_detect["Detect<br/>inference script<br/>[yolo_detect.py]"]
  node_node_yolo_py["YOLO Script<br/>experiment script<br/>[yolo.py]"]
  node_node_pickup_learn["Learn<br/>training script<br/>[pickup-learn.py]"]
  node_node_ncnn_convert["Convert<br/>export pipeline<br/>[ncnn_conversion.py]"]
  node_node_camera_test["Camera Test<br/>hardware test<br/>[camera_test.py]"]
  node_node_motor_test["Motor Test<br/>hardware test<br/>[motor_test.py]"]
  node_node_sensor_test["Sensor Test<br/>hardware test<br/>[sensor_test.py]"]
end

subgraph group_group_firmware["Firmware"]
  node_node_arduino["Arduino<br/>firmware<br/>[autonomous.ino]"]
end

subgraph group_group_data["Data & models"]
  node_node_torchscript["TorchScript<br/>exported model"]
  node_node_ncnn_model["NCNN Model<br/>optimized model"]
  node_node_best_pt["Best Weights<br/>trained weights<br/>[best.pt]"]
  node_node_yolo11n_pt_obj["Base Weights<br/>model weights<br/>[yolo11n.pt]"]
  node_node_yolo11n_pt_root["YOLO Weights<br/>model weights<br/>[yolo11n.pt]"]
  node_node_world_weights["World Weights<br/>model weights<br/>[yolov8l-world.pt]"]
  node_node_world_weights_2["More Weights<br/>model weights<br/>[yolov8m-world.pt]"]
  node_node_yolov8n["YOLOv8n<br/>model weights<br/>[yolov8n.pt]"]
  node_node_yolov8s_world["YOLOv8s World<br/>model weights<br/>[yolov8s-world.pt]"]
  node_node_yolov8s_worldv2["World v2<br/>model weights<br/>[yolov8s-worldv2.pt]"]
  node_node_dataset_obj["Obj Dataset<br/>training data<br/>[data.yaml]"]
  node_node_dataset_lego["Lego Dataset<br/>training data<br/>[data.yaml]"]
  node_node_runs["Predict Runs<br/>generated output"]
end

node_node_camera_test -->|"feeds"| node_node_yolo_detect
node_node_yolo_detect -->|"detections"| node_node_autonomous_py
node_node_yolo_py -->|"supports"| node_node_yolo_detect
node_node_pickup_learn -->|"experiments"| node_node_autonomous_py
node_node_autonomous_py -->|"commands"| node_node_arduino
node_node_arduino -->|"feedback"| node_node_sensor_test
node_node_arduino -->|"actuation"| node_node_motor_test
node_node_arduino -->|"sensing"| node_node_sensor_test
node_node_dataset_obj -->|"trains"| node_node_best_pt
node_node_dataset_lego -->|"trains"| node_node_yolo11n_pt_root
node_node_best_pt -->|"exports"| node_node_torchscript
node_node_best_pt -->|"exports"| node_node_ncnn_model
node_node_ncnn_convert -->|"produces"| node_node_ncnn_model
node_node_yolo11n_pt_obj -->|"basis"| node_node_torchscript
node_node_yolov8n -->|"inference"| node_node_yolo_detect
node_node_world_weights -->|"inference"| node_node_yolo_detect
node_node_world_weights_2 -->|"inference"| node_node_yolo_detect
node_node_yolov8s_world -->|"inference"| node_node_yolo_detect
node_node_yolov8s_worldv2 -->|"inference"| node_node_yolo_detect
node_node_torchscript -->|"evaluated as"| node_node_runs

click node_node_autonomous_py "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/autonomous.py"
click node_node_yolo_detect "https://github.com/jugpanda/pickuppal/blob/main/src/yolo/yolo_detect.py"
click node_node_yolo_py "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolo.py"
click node_node_pickup_learn "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/pickup-learn.py"
click node_node_ncnn_convert "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/ncnn_conversion.py"
click node_node_torchscript "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolo11n.torchscript"
click node_node_ncnn_model "https://github.com/jugpanda/pickuppal/tree/main/src/yolo/yolo11n_ncnn_model"
click node_node_best_pt "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/best.pt"
click node_node_yolo11n_pt_obj "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolo11n.pt"
click node_node_yolo11n_pt_root "https://github.com/jugpanda/pickuppal/blob/main/src/yolo/yolo11n.pt"
click node_node_world_weights "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8l-world.pt"
click node_node_world_weights_2 "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8m-world.pt"
click node_node_yolov8n "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8n.pt"
click node_node_yolov8s_world "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8s-world.pt"
click node_node_yolov8s_worldv2 "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8s-worldv2.pt"
click node_node_arduino "https://github.com/jugpanda/pickuppal/blob/main/src/pickup-arduino/autonomous.ino"
click node_node_camera_test "https://github.com/jugpanda/pickuppal/blob/main/src/tests/camera_test.py"
click node_node_motor_test "https://github.com/jugpanda/pickuppal/blob/main/src/tests/motor_test.py"
click node_node_sensor_test "https://github.com/jugpanda/pickuppal/blob/main/src/tests/sensor_test.py"
click node_node_dataset_obj "https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/data.yaml"
click node_node_dataset_lego "https://github.com/jugpanda/pickuppal/blob/main/src/yolo/Lego/data.yaml"
click node_node_runs "https://github.com/jugpanda/pickuppal/tree/main/src/yolo/runs/detect/predict"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_autonomous_py,node_node_yolo_detect,node_node_yolo_py,node_node_pickup_learn,node_node_ncnn_convert,node_node_camera_test,node_node_motor_test,node_node_sensor_test toneBlue
class node_node_arduino toneAmber
class node_node_torchscript,node_node_ncnn_model,node_node_best_pt,node_node_yolo11n_pt_obj,node_node_yolo11n_pt_root,node_node_world_weights,node_node_world_weights_2,node_node_yolov8n,node_node_yolov8s_world,node_node_yolov8s_worldv2,node_node_dataset_obj,node_node_dataset_lego,node_node_runs toneMint
```

## Components

### Python stack

Perception, autonomy, model conversion, and local experiment scripts.

- [Autonomous](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/autonomous.py): orchestrator · High-level robot behavior that turns detections into pickup decisions. · `src/object-detection/autonomous.py`
- [Detect](https://github.com/jugpanda/pickuppal/blob/main/src/yolo/yolo_detect.py): inference script · Primary YOLO inference entry for camera-based object detection. · `src/yolo/yolo_detect.py`
- [YOLO Script](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolo.py): experiment script · Supporting YOLO workflow script for running or testing detection logic. · `src/object-detection/yolo.py`
- [Learn](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/pickup-learn.py): training script · Robot learning or experimentation script tied to object detection workflows. · `src/object-detection/pickup-learn.py`
- [Convert](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/ncnn_conversion.py): export pipeline · Converts trained YOLO models into an optimized NCNN deployment format. · `src/object-detection/ncnn_conversion.py`
- [Camera Test](https://github.com/jugpanda/pickuppal/blob/main/src/tests/camera_test.py): hardware test · Standalone camera validation path for vision input debugging. · `src/tests/camera_test.py`
- [Motor Test](https://github.com/jugpanda/pickuppal/blob/main/src/tests/motor_test.py): hardware test · Standalone motor validation path for motion debugging. · `src/tests/motor_test.py`
- [Sensor Test](https://github.com/jugpanda/pickuppal/blob/main/src/tests/sensor_test.py): hardware test · Standalone sensor validation path for hardware feedback debugging. · `src/tests/sensor_test.py`

### Firmware

Embedded controller logic for motors and sensors.

- [Arduino](https://github.com/jugpanda/pickuppal/blob/main/src/pickup-arduino/autonomous.ino): firmware · Embedded controller firmware for actuators and sensors. · `src/pickup-arduino/autonomous.ino`

### Data & models

Training datasets, exported weights, and generated inference artifacts.

- [TorchScript](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolo11n.torchscript): exported model · Serialized model artifact for Python-side optimized inference. · `src/object-detection/yolo11n.torchscript`
- [NCNN Model](https://github.com/jugpanda/pickuppal/tree/main/src/yolo/yolo11n_ncnn_model): optimized model · Edge-friendly NCNN export package used for deployment or validation. · `src/yolo/yolo11n_ncnn_model`
- [Best Weights](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/best.pt): trained weights · Primary trained YOLO checkpoint used for detection performance. · `src/object-detection/best.pt`
- [Base Weights](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolo11n.pt): model weights · Ultralytics YOLO base checkpoint used for inference or fine-tuning. · `src/object-detection/yolo11n.pt`
- [YOLO Weights](https://github.com/jugpanda/pickuppal/blob/main/src/yolo/yolo11n.pt): model weights · Repository-level YOLO checkpoint used by the alternate YOLO workspace. · `src/yolo/yolo11n.pt`
- [World Weights](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8l-world.pt): model weights · Additional YOLO world-model checkpoints used for experimentation. · `src/object-detection/yolov8l-world.pt`
- [More Weights](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8m-world.pt): model weights · Additional YOLO world-model checkpoints used for experimentation. · `src/object-detection/yolov8m-world.pt`
- [YOLOv8n](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8n.pt): model weights · Lightweight YOLO checkpoint available for testing and comparison. · `src/object-detection/yolov8n.pt`
- [YOLOv8s World](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8s-world.pt): model weights · World-model checkpoint variant for experimentation. · `src/object-detection/yolov8s-world.pt`
- [World v2](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/yolov8s-worldv2.pt): model weights · Second world-model variant for experimentation. · `src/object-detection/yolov8s-worldv2.pt`
- [Obj Dataset](https://github.com/jugpanda/pickuppal/blob/main/src/object-detection/data.yaml): training data · Primary object-detection dataset and label configuration. · `src/object-detection/data.yaml`
- [Lego Dataset](https://github.com/jugpanda/pickuppal/blob/main/src/yolo/Lego/data.yaml): training data · Prepared YOLO dataset split with images and labels for training and validation. · `src/yolo/Lego/data.yaml`
- [Predict Runs](https://github.com/jugpanda/pickuppal/tree/main/src/yolo/runs/detect/predict): generated output · Saved local prediction outputs from inference experiments. · `src/yolo/runs/detect/predict`

## Connections

- Camera Test → Detect: feeds · Camera validation supports the perception pipeline.
- Detect → Autonomous: detections · Inference outputs drive higher-level behavior.
- YOLO Script → Detect: supports · Auxiliary YOLO logic informs the main inference path.
- Learn → Autonomous: experiments · Learning/testing work informs autonomy behavior.
- Autonomous → Arduino: commands · Autonomy sends actuation commands to embedded control.
- Arduino → Sensor Test: feedback · Embedded feedback path is validated through sensor testing.
- Arduino → Motor Test: actuation · Motor control is validated against the firmware boundary.
- Arduino → Sensor Test: sensing · Sensor input is owned by the firmware boundary.
- Obj Dataset → Best Weights: trains · Primary dataset produces the main trained checkpoint.
- Lego Dataset → YOLO Weights: trains · Prepared dataset supports alternate YOLO training artifacts.
- Best Weights → TorchScript: exports · Trained weights are converted into TorchScript.
- Best Weights → NCNN Model: exports · Trained weights are converted into NCNN for deployment.
- Convert → NCNN Model: produces · Conversion script generates the NCNN artifact bundle.
- Base Weights → TorchScript: basis · Base checkpoint can be used to build exported artifacts.
- YOLOv8n → Detect: inference · Lightweight model option for detection experiments.
- World Weights → Detect: inference · Alternative model checkpoint for detection experiments.
- More Weights → Detect: inference · Alternative model checkpoint for detection experiments.
- YOLOv8s World → Detect: inference · Alternative model checkpoint for detection experiments.
- World v2 → Detect: inference · Alternative model checkpoint for detection experiments.
- TorchScript → Predict Runs: evaluated as · Exported model is exercised in local prediction runs.
