How unimakers/espresso works
Project type and purpose
• This is an embedded‐focused robotics toolkit (“Espresso Toolkit”) targeted at ESP32 microcontrollers.
• It provides decoupled C/C++ libraries for common robotics functions (LiDAR, steppers, EQL, vision), vendored TensorFlow Lite Micro for on‐device ML inference, plus Python ML scripts, data assets, documentation, and PlatformIO build configuration.
File structure
• Top level
• platformio.ini – defines the embedded build (ESP32) and dependencies.
• src/main.cpp – the entry point for firmware.
• lib/… – contains subfolders for eql, lidar, stepper, vision, and a vendored TensorFlow Lite Micro tree.
• ml/… – Python scripts and data for training and testing ML models (YOLOv8, pickle).
• dataset/… – raw and processed image data for ML training.
• tools/… – helper Python scripts (eql.py, robovision UI).
• docs/… – MkDocs site for user and developer documentation.
• Patterns
• Each robotics capability is its own library folder (lib/lidar, lib/stepper, lib/vision, lib/eql).
• Vendored third‐party code under lib/vendors/tensorflow includes the full TFLite Micro engine and examples.
• ML pipeline separated into its own directory; scripts generate .pt/.pkl models outside of embedded build.
• PlatformIO orchestrates firmware build and linking of all C/C++ source.
README for additional insights
• Purpose: a lightweight robotics toolkit specialized for ESP32.
• Not yet stable for external package use; modules are decoupled, built into your code via copy/paste.
Main components
a. Development Environment
• Host PC
– IDE/CLI running PlatformIO
– Python environment for ML (training.py, processing.py)
– Documentation builder (MkDocs)
b. Firmware Build System
• PlatformIO
• CMakeLists and header/source files under lib/ and src/
• Vendored TensorFlow Lite Micro
c. Embedded Target
• ESP32 Module (dual‐core microcontroller)
• Sensor Interfaces
– LiDAR (via UART/SPI/I2C to lib/lidar)
• Actuators
– Stepper motors (via lib/stepper)
– Servos (if added in future)
• In‐device ML Inference
– TensorFlow Lite Micro interpreter linking a compiled .tflite or model data blob
• Custom EQL Protocol Module (lib/eql)
d. ML Pipeline (Host‐side)
• Dataset (dataset/raw, dataset/blobs)
• Training Script (ml/training.py) → generates ml/yolov8n.pt, ml/model.pkl
• Processing Script (ml/processing.py) → converts/optimizes model for embedded inference
e. Tools & Utilities
• EQL code generator (tools/eql.py)
• Robovision host UI (tools/robovision/main.py + components)
• Script.eql domain language file
Relationships and data flows
• On the host:
1. Collect images (dataset/) → run ml/training.py → produce model files.
2. Run ml/processing.py → embed converted model into firmware resources.
3. Run PlatformIO build → links lib/* (including TensorFlow, EQL, LIDAR, stepper, vision).
4. Flash firmware onto ESP32.
• On the device:
1. Sensors feed data into library modules (e.g., LiDAR → lib/lidar, camera → lib/vision).
2. Vision/EQL modules process raw sensor data.
3. If ML inference is enabled, data passes into TFLite Micro interpreter.
4. Control commands output to actuators via lib/stepper or future servo library.
5. Optionally, send telemetry back over Wi-Fi/Bluetooth to host or Robovision tool.
Architectural patterns and design principles
• Modular, decoupled libraries: each domain (LiDAR, vision, stepper, EQL) stands alone.
• Single‐binary firmware: all modules and vendored TF‐Lite Micro link into one ESP32 executable.
• Well‐separated host vs. device responsibilities: ML training and model conversion on host; inference on device.
• Use of PlatformIO for cross‐platform embedded build management.
• Python scripts organized under ml/ and tools/ for extensibility and maintainability.
Be detailed but focused
• External/vendor code (TensorFlow Lite) as a separate component.
• ML assets (model files) flowing into the firmware.
Layer 1: Host (ML pipeline, tools, docs)
Layer 2: Build system (PlatformIO, resource embedding)
Layer 3: Embedded firmware modules
Layer 4: Hardware (sensors/actuators)