How mocco329/htwm works
Main Components
• App Front End: Built with React Native and TypeScript (see the “front_app” directory). This mobile application handles user interactions, routine management, PT calls, workout logs, and photo management.
• Device Front End: Located under “front_device”, this component is built with Electron, React, and Redux. It runs on the smart mirror device (Raspberry Pi, Jetson NANO) and is responsible for interacting with hardware (vibration sensors, cameras) as well as presenting reference/exercise videos.
• Back End: Found under “back/htwm”, this Spring Boot Java application uses controllers, services, repositories, and DTOs to implement APIs. It integrates features such as WebSocket communication (see WebSocketConfig, CustomHandshakeHandler), S3-based file storage, and various REST endpoints for managing data (users, routines, notices, pictures, records, etc.).
• Embedded Modules for Sensor/AI Functions: The “emb” directory contains Python modules for pose estimation (with OpenCV) and speech processing. These scripts integrate sensor triggers (knocking or voice commands) with device functions.
• Infrastructure: The README specifies that infrastructure is managed with Jenkins, AWS EC2, S3, and NginX, and there’s also a Dockerfile for containerization (in back/htwm). A MySQL database (ERD and SQL dump in “exec/dumpHTWM.sql”) serves as the primary datastore.
Detail the Relationships and Interactions
• Mobile App and Device Front End as Clients:
• The mobile app communicates directly with the backend via API endpoints (notice the organized API calls in “front_app/api”).
• The device front end is used to interact with the smart mirror hardware and integrates sensor inputs from connected devices.
• Backend Server:
• Handles business logic and API requests, serves as the central hub for users, exercises, routines, and media management.
• It further manages WebSocket connections (for real-time PT sessions and streaming) and integrates with external services like AWS S3 for file storage.
• Embedded Sensor/AI Modules:
• The pose and speech modules (in the “emb” directory) provide additional functionality (e.g., posture-based exercise tracking and voice command recognition) that better integrate with the device front end.
• Infrastructure:
• Deployment and CI/CD processes (Jenkins) manage build and deployment.
• Docker and AWS setup ensure scalability and availability.
Architectural Patterns and Technologies
• Modular & Layered Architecture: The separation into distinct modules – mobile app, device application, backend, and embedded services – points to a micro-modular design.
• Client-Server Interaction / RESTful API: The backend exposes REST endpoints and WebSocket channels for real-time notifications and video PT sessions.
• Hardware-Software Integration: The device side is customized to interact with physical sensors (vibration, camera) and leverages speech & pose modules for improved exercise monitoring.
• Technology Stack:
• Frontend: React Native (mobile) and Electron with React (device).
• Backend: Spring Boot with Gradle, Java, MySQL, WebSockets, and AWS integrations.
• Embedded: Python scripts for pose and speech (utilizing OpenCV for computer vision).
Instructing the Principal Software Engineer
• Annotations or labels that identify key frameworks (React Native, Electron, Spring Boot) as well as technologies (Docker, AWS EC2/S3, Jenkins).
• Integration points, like the S3 uploader in the backend configuration and the sensor triggers in the device front end.