How cmanjah/tiagogpt--robot-assisted-interventions- works
Main Components
• Application Launcher and Bootstrapping:
• TIAGoGPT.pyproject, TIAGoGPT.pyproject.user, and TIAGoGPTAppLauncher.py files serve as the entry points.
• Controller Layer:
• The “controller” directory (with AppController.py and ThreadSignalController.py) handles high-level orchestration, conversation management, and thread handling.
• Modules Layer:
• “modules” contains key functional units including:
• Audio Recording Module (for capturing spoken input),
• Data Storage Module (managing databases and file storage for chat logs, consent, etc.),
• EndPoint Module (to integrate external APIs for transcription, response generation, and voice synthesis).
• Resources & Assets:
• The “resources” directory holds audio files, system logs, images, templates, and other static files needed for GUIs, CSS skins, and logging.
• External Services & APIs:
• OpenAI’s Whisper (voice transcription) and GPT-3.5 Turbo (response generation) APIs,
• Google Cloud Text-to-Speech API for synthesised voice outputs.
• GUI/User Interaction:
• Evident from release notes mentioning a user-friendly graphical interface that displays responses and handles dual conversation modes (text and voice).
Relationships and Interactions
• The Application Launcher initiates the system, which passes control to the Controller Layer.
• The Controller then coordinates calls between various Modules:
• The Audio Recording module captures user speech.
• The captured audio is sent to the transcription service (via the EndPoint Module) and then forwarded to the GPT response generator.
• Generated responses are passed to the Text Synthesis module, and results are played back or displayed through the GUI.
• The Data Storage Module ensures that chat logs, consent forms, and audit logs are properly managed.
• Thread and concurrency controllers handle simultaneous processes (like recording, transcription, and response generation), ensuring smooth interaction.
• There is clear communication between internal modules and external APIs, with configuration and credential files located in the “modules/includes” folder.
Architectural Patterns and Design Principles
• MVC Pattern / Layered Architecture: Separation of concerns is indicated by splitting the controllers (business logic), modules (functional units), and resources (static assets).
• Modular Design: Individual modules target distinct responsibilities such as audio processing, data storage, API integrations, and UI handling.
• Thread Management: Dedicated components (ThreadSignalController) manage asynchronous operations.
• Security and Data Integrity: Components for consent handling, file access control, audit logging, and data encryption/hashing illustrate responsible AI practices and secure design.
Relevant Technologies, Frameworks, and Libraries
• Python (with virtual environment setups in the README),
• OpenAI’s Whisper API and GPT-3.5 Turbo,
• Google Cloud TTS,
• Possibly a GUI framework (given the mention of skins and interface; look into libraries like PyQt or Tkinter based on code or further documentation),
• Standard Python threading (as seen in the controllers for thread management).
Suggested Elements
• Clear labels for every component (e.g., “Audio Recording Module”, “Transcription Service”, “Response Generation Module”, “Text Synthesis Module”, “GUI Interface”).
• From user input (via microphone/UI) → Audio Recording Module → Controller → Transcription Service → GPT Generator → Text Synthesis → Response to User Interface.
• Another for external APIs/services,
• A distinct style for storage/data repositories.
• Annotations highlighting the MVC structure and threading components.
• If applicable, notes explaining integration with the TIAGo robot hardware.
Putting It All Together
• Frontend/UI,
• Business logic (controllers),
• Functional modules, and
• External integrations and storage.
• Directional data flow is clearly mapped, showing sequential interactions between voice input, processing, response generation, and output.
• Security and data handling components (like audit logs and consent forms).