How orionramos/hpis_app_code works
Project Type and Purpose
• HPIS is a distributed, real-time human-prosthesis interaction system.
• It comprises an embedded device (ESP32), a Python WebSocket server (orchestration layer), several Python clients (control apps + simulations), a Unity application (visualization), and auxiliary modules (HRI audio strategies, YOLO).
Main Components
a. Hardware Layer
• ESP32 device (firmware in ESP/main: main.ino, config.h, wifi_communicator.h)
• (Optional) ESP32 simulation client (ClienteESP32.py)
b. Server Layer
• WebSocketServer (Python/Main/WebsocketServer.py) – central hub
• HRControl client (Python/Main/HRControl.py) – fetches heart-rate via Pulsoid API
• HPISControl client (Python/Main/HPIS_Control.py) – user GUI/CLI for selecting participant/HRI strategy
• launch_manager, client_data_recorder (supporting scripts)
c. Auxiliary Services / Modules
• HRI_Strategies (Python/HRI_Strategies/HRI_Audio_Creation.py + JSON + generated MP3s)
• YOLO object-detection module (Python/YOLO/YOLO.py, seting.py)
d. Client Layer
• Unity App (Unity packages + lecturaJSONDatabase.py, feedback_data_named.json)
• Unity simulation client (Python/Simulation/clientUnitySimulation.py)
e. Deployment / Configuration
• requirements.txt, install_requirements.py
• Arduino library dependencies (ArduinoJson, ESPping)
Relationships and Data Flows
• ESP32 ↔ WebSocketServer
• ESP32 streams EMG pulses to server
• Server sends back “grip type” commands (pulse-train)
• HRControl ↔ Pulsoid API ↔ WebSocketServer
• HRControl polls Pulsoid → WS server
• HPISControl ↔ WebSocketServer
• User selections (participant/strategy/activity) → server
• WebSocketServer ↔ Unity App (or Simulation)
• Server pushes processed data (EMG, HR, strategy cues) → Unity
• HPISControl → HRI_Strategies module
• On strategy selection, invokes audio generation → MP3 files served or played
• (Optional) YOLO ⚙ WebSocketServer
• Object detection results feed into HRI logic
Architectural Patterns / Design Principles
• Event-driven / message-oriented via WebSockets
• Modular separation:
• Hardware clients vs. software clients
• Core orchestration (server) vs. auxiliary processing (audio, vision)
• Simulations decoupled (ESP32 & Unity) for offline testing
• Clear separation of concerns:
• Data ingestion (EMG, HR)
• Control logic (HPISControl)
• Visualization (Unity)
• Extensibility points (HRI_Strategies, YOLO)
• Notes on technology: Python 3.x, WebSocket library, ArduinoJson, Unity.
• Modular grouping to reflect directory structure.