How mizly/wingman works
Major Components
a. Physical & Hardware Layer:
• Arduino Subsystem:
• Contains arduino_files (e.g., TFTLCDDisplay.ino and motorAndTextController.ino).
• Responsible for controlling display output on the LCD panel and motor functions for movement.
b. Core Python Application (Control & Orchestration):
• Main Controller:
• Modules like main.py and controller.py orchestrate the different functionalities.
• Acts as the central hub, coordinating all subsystems.
c. Vision/Facial Recognition Subsystem:
• Face Recognition:
• Files: facerec.py, cvcont.py, update_face.py, trackinginfo.py.
• Uses the faces/ directory and face_encodings.pkl for recognizing and updating facial data.
d. Natural Language Processing & Conversation:
• NLP Engine:
• Modules: cohere_nlp.py, gemini.py, basic_functionalities.py.
• Likely manages general prompting and conversation handling.
e. Text-to-Speech (TTS) Subsystem:
• TTS Engine:
• Provided by the tiktok_voice package (including tiktok_voice/src/text_to_speech.py, voice.py) and tts.py.
• Converts generated text responses into audio output, which is then possibly played via speakers.
f. Music & External Integrations:
• Music/Spotify Connectivity:
• Handled by music.py (and possibly additional integration in gemini.py) to manage streaming or playing music.
Relationships/Interactions
• The Main Controller (main.py) serves as the coordinator. It initiates commands and feeds input/output between subsystems.
• Facial recognition processes visual input (presumably captured externally) and sends recognition results to the central controller.
• NLP processes convert speech to text (speech-to-text not visible directly, though there is TTS and NLP code) and interprets commands; it might interact with both the music functionalities and display movements.
• The TTS subsystem converts text into audio, completing the interaction loop with the user.
• The Arduino subsystem receives commands (possibly via a serial or similar connection) from the Python controller to move the device and update the LCD display with emotions or text.
• Music integration and weather functionalities are triggered based on conversational inputs processed by NLP.
Architectural Patterns & Design Principles
• Modularity: Each major functionality (vision, NLP, TTS, hardware control) is separated into its own module or group of modules—a clear separation of concerns.
• Layered Structure: The design cleanly separates low-level hardware control (Arduino) from high-level software logic (Python modules).
• Extensibility: The use of external projects (e.g., TikTok Voice TTS and a face recognition library) indicates that the design accommodates integration with external libraries and services.
Technologies & Frameworks
• Arduino IDE/C++ for firmware controlling motors and LCD display.
• Python modules leveraging libraries for NLP and face recognition (referencing GitHub sources provided in the README).