How pbhatt0022/vision-follow-project works
From the README + file tree, this is a single-machine, real-time vision-to-control pipeline written in Python 3.11. It:
• Captures frames from a webcam (capture/)
• Runs YOLO-based person detection/tracking (perception/)
• Computes horizontal centroid error (target center vs frame center)
• Converts error → control signal via PID (control/)
• Logs telemetry to disk (outputs/logs/centroid_log.csv)
• Runs an offline PID replay/analysis that versions experiments under outputs/experiments/<timestamp>/
It’s not yet an IoT robot system in this repo; it’s the vision + control logic core that is *designed* to integrate into a robot later.
1) Online / real-time loop (webcam → YOLO → error → (future live PID) → actuator integration)
Offline experiment pipeline (centroid_log.csv → PID sim → plots/artifacts)
Use the package boundaries under src/vision_follow/ as your primary component boundaries:
(device I/O)
• src/vision_follow/capture/test_camera.py
• Entry point to validate camera access
• Responsibilities: open webcam, read frames, basic display/verification (likely OpenCV)
Perception (ML inference + tracking + telemetry)
• src/vision_follow/perception/yolo_detect.py
• YOLO inference / detection
• src/vision_follow/perception/yolo_track.py
• Main real-time script: camera loop + detect/track + compute error + optional logging
• Perception: YOLO Detector (PyTorch/Ultralytics)
• Perception: Tracker + Centroid Error Calculator
Control (PID + follow logic)
• src/vision_follow/control/pid_controller.py
• Core PID implementation (Kp/Ki/Kd, dt, clamps, etc.)
• src/vision_follow/control/follow_logic.py
• Higher-level follow behavior (how PID output maps to “turn left/right/stop”, etc.)
• src/vision_follow/control/pid_from_log.py
• Offline replay: reads log, runs PID, generates plots + experiment folder
• Control: PID Controller
• Control: Follow Logic (decision/mapping layer)
• Offline Analysis: PID Replay + Plot Generator
Data/artifacts (filesystem as a first-class “store”)
Even though there’s no database, the filesystem is central to reproducibility:
• outputs/logs/centroid_log.csv
• outputs/experiments/<timestamp>/{plots, run_config, copied log}
Model assets (local weights)
The README mentions models/ (not present in the tree, but conceptually important).
• YOLO weights file(s) loaded by perception
Environment / runtime platform (deployment boundary)
The runtime is the developer machine:
• CPU or optional NVIDIA GPU (CUDA via PyTorch)
• Windows/Linux
• Python venv + requirements*.txt
Webcam → Capture
• Payload: Frame (H×W×3)
Capture → Perception: YOLO Detector
• Payload: Frame
YOLO Detector → Tracker + Centroid Error Calculator
• Payload: Detections/track state (bbox, id, confidence)
Tracker + Error Calc → Control (future live PID)
• Payload: error_x (pixels or normalized)
Tracker + Error Calc → Telemetry Logger → outputs/logs/centroid_log.csv
• Payload: timestamped error_x samples
• Payload: turn_rate / angular_velocity / command
Important nuance for accuracy: today, the repo clearly logs error and does offline PID; the live PID overlay/control is listed as “Next”. So either:
pid_from_log.py → outputs/experiments/<timestamp>/
• Artifacts: error_plot.png, turn_plot.png, run_config.txt, copied centroid_log.csv
This offline pipeline is a distinct “batch job” and should be visually separated from the real-time loop.
This repo is structured as a pipeline with clean separation of concerns:
• I/O boundary: webcam capture isolated under capture/
• Compute/ML boundary: inference + tracking isolated under perception/
• Control boundary: PID + behavior mapping isolated under control/
• Reproducibility boundary: outputs are versioned, logs are persisted, lockfile exists
Purpose: show who/what interacts with the system.
• Actor: User/Developer
• External systems: Webcam Device, optional NVIDIA GPU, optional future Robot Platform
• System: Vision Follow (this repo)
Purpose: show runtime blocks on the host machine.
Inside Host Machine boundary:
• Python App (vision_follow)
• PyTorch/Ultralytics YOLO
• OpenCV
• Local Filesystem (outputs/, models/)
• Webcam → Python App
• Python App ↔ GPU (optional acceleration)
• Python App ↔ Filesystem (logs, experiments, weights)
Purpose: match the file tree 1:1 to components and entrypoints.
Components:
• capture/test_camera.py (entrypoint)
• perception/yolo_track.py (entrypoint)
• perception/yolo_detect.py (module)
• control/pid_from_log.py (entrypoint)
• control/pid_controller.py (module)
• control/follow_logic.py (module)
Show:
• which scripts call which modules
• data artifacts produced/consumed
Labels because they’re core to understanding:
• Frame source: “Webcam (camera-index N)”
• Inference engine: “YOLO (PyTorch; CPU/GPU)”
• Key computed signal: “Centroid horizontal error (px or normalized)”
• Telemetry format: “CSV log: outputs/logs/centroid_log.csv”
• Experiment versioning: “outputs/experiments/<timestamp>/”
• Control output meaning: “turn command / angular velocity (PID output)”
• Don’t imply distributed services; everything runs locally in-process today.
• two entrypoints clearly:
• yolo_track.py for online tracking/logging
• pid_from_log.py for offline replay/plots
• Treat “robot actuation” and “owner face recognition” as external / future unless you find code for them (they’re not in the current tree).
• Perception Runner (yolo_track.py)
• PID Implementation (pid_controller.py)
• Offline Experiment Builder (pid_from_log.py)