How antbono/hni works
Project Type and Purpose
• This project is a robotics-HRI (Human-Robot Interaction) system that provides capabilities for experimentation with the NAO v6 robot.
• Its primary functions include teaching by demonstration (recording and reproducing joint movements), face/object detection using Yolov8 models, speech recognition/synthesis via Google Cloud APIs, and natural language conversational abilities based on ChatGPT.
Directory Structure
• There are three main packages:
• hni_cpp: Contains C++ code, including ROS nodes implementing actions and services (chat, joints play, head tracking, etc.). It also holds launch files (e.g., experiment_nao_launch.py and experiment_pc_launch.py) to start different robotic experiments.
• hni_interfaces: Defines the ROS actions, messages, and services (action definitions for ChatPlay, JointsPlay, etc.) that standardize communication across the system.
• hni_py: Contains Python code that acts as clients or service implementations, handling additional functionalities like speech services (Google’s TTS/STT) and video tracking using Yolov8 models.
• The .github folder shows CI/CD pipelines testing on multiple ROS distributions (Humble, Iron, Rolling).
• Configuration files (e.g., CMakeLists.txt, package.xml, setup.py) indicate both C++ and Python components are built and managed in the ROS2 framework.
a. Main Components to Include:
• hni_cpp Module: Represent a block labeled “C++ ROS Nodes” featuring components such as Chat Action Client/Server, Joints Play Action Client/Server, Head Track Action Server/Client, and related launch mechanisms.
• hni_interfaces: Represent as “ROS Interface Definitions” encapsulating the action, message, and service definitions. This module connects the C++ and Python nodes.
• hni_py Module: Represent a block called “Python ROS Nodes & Clients” which includes nodes for Chat Service, speech-to-text (STT) / text-to-speech (TTS) services, and video tracking using Yolov8 models.
• External Services: Create blocks for the “Google Cloud APIs” (for TTS and STT) and “ChatGPT API” (for conversation processing).
• Robot Hardware / Simulation: Represent how the NAO robot (or its simulation) interfaces with the nodes via ROS2 topics, services, and actions.
b. Relationships and Interactions:
• Between hni_cpp (C++ nodes) and hni_interfaces, indicating they use predefined message/action protocols.
• Between hni_py and external services (Google APIs and ChatGPT) for sending audio data or processing text.
• Between the two experimental launch setups (experiment_nao_launch.py for the robot and experiment_pc_launch.py for visual feedback/auxiliary processing) indicating that both share the same underlying interface definitions.
• From the sensor inputs (video, audio) into nodes that process these inputs (e.g., speech recognition, object detection) and then route the results to the higher-level conversational or motion control nodes.
c. Architectural Patterns and Design Principles:
• The system follows the ROS2 component-based architecture emphasizing modularity and message-passing through standardized interfaces.
• There is a clear separation between the control logic (action servers and clients in hni_cpp/hni_py) and the communication contracts (hni_interfaces).
• The design embraces microservice-like principles within a distributed robotic framework where multiple nodes (written in different languages) interact over ROS2.
d. Technologies and Frameworks:
• ROS2 middleware (topics, services, actions) as the backbone messaging framework.
• C++ and Python for different modules (with C++ for performance-critical tasks, Python for easier integration with external APIs).
• Google Cloud APIs for speech synthesis/recognition.
• Yolov8 models for face/object detection.
• ChatGPT for conversational processing.
Final Recommendations
• Being very detailed is key: Instead of lumping everything together, break the system into as many well-defined components as the file structure suggests.
• Aim for clarity: Even though the system is modular, the clear labeling of components and explicit data flows will make it easier for someone new (or even for maintenance purposes) to understand the architecture thoroughly.