How akshitjain16/visionmate works
Major Components
• Frontend Layer (Client):
• Located in the “client” directory.
• Built with React.js, Tailwind CSS, and uses TensorFlow.js alongside the Web Speech API.
• Consists of several UI components for features such as home, login/signup, object detection, text-to-speech, speech-to-text, to-do list, and voice navigation.
• Backend Layer (Server):
• Found in the “server” directory.
• Implements the RESTful API using Node.js/Express.js.
• Contains controllers (e.g., AuthAPI, STT_API), routes (for authentication and speech-to-text), and model definitions for users and task management.
• Integrates with a MongoDB database for persistent storage.
• Object Detection Service (YOLO Flask):
• Hosted in the “YoloWebFlask” directory.
• A separate Python/Flask application that leverages YOLO models (as evidenced by the .pt files such as yolov8n.pt and yolov8s.pt) to perform object detection.
• Likely interfaces with the main system either via RESTful API calls or is triggered by the client for specialized object detection tasks.
• External/Browser-Level Features:
• Use of the browser’s Web Speech API and TensorFlow.js enables real-time object detection and voice interaction, directly in the user’s browser.
• The client application also handles text-to-speech functionality.
Relationships/Interactions
• The React Frontend communicates with the Node/Express backend via REST API calls.
• The backend handles user authentication (e.g., via JWT), speech-to-text processing, and manages data (e.g., user tasks).
• For object detection, the client might leverage two possible approaches:
• Direct integration through TensorFlow.js (as per the README), or
• Service integration with the Python Flask API (YoloWebFlask) which serves YOLO models for advanced object detection.
• Data flows from the backend to the MongoDB database for persistence.
Architectural Patterns and Principles
• Separation of concerns: Clearly partition the frontend, backend, and specialized object detection service.
• RESTful API communication between the client and backend.
• Modular design: Each component (authentication, speech-to-text, object detection) is independently encapsulated.
• Use of industry-standard technologies for each layer (React for UI, Express for API, Flask for ML inference, and MongoDB for persistence).
Relevant Technologies
• Frontend: React.js, Tailwind CSS, Framer Motion, TensorFlow.js, Web Speech API.
• Backend: Node.js, Express.js, MongoDB, JWT Authentication.
• Object Detection: Python, Flask, and YOLO (.pt models).
• You may also indicate additional scripts or build tools present in “client” (such as PostCSS and package management files) to emphasize a robust development workflow.