How hack4innovation-2026/hack4innovation works
DrishtiAI is a voice- and camera-driven visual assistant for visually impaired users. The principal workflow starts with a user request, uses the Flutter voice copilot to route intent to signboard interpretation or person recognition, then delivers concise spoken feedback. Person recognition is also exposed through a FastAPI backend: it detects and distance-filters people, matches face embeddings against a local JSON store, and can optionally back up face images to Google Cloud Storage. README documents the frontend capabilities, but its Dart source is absent from the supplied tree, so those relationships are documented rather than source-verified.
Main entry points: recognition.py Person Analysis & Registration · services Voice Copilot · main.py Recognition API · storage.py Face Image Backup
Components 16
Flutter Assistant
- Flutter App
lib/main.dart - Voice Copilot
lib/services - Intent Routing
lib/services - Speech & Haptics
lib/services
Visual Understanding
- On-device OCR
lib/services - Signboard Interpretation
lib/services - Person Recognition Client
lib/services
Recognition Backend
- Recognition API
backend/app/main.py - Person Analysis & Registration
backend/app/recognition.py - Recognition Settings
backend/app/config.py - API Schemas
backend/app/schemas.py
Recognition Data
- Face Embedding Store
backend/app/database.py - Face Image Backup
backend/app/storage.py
Other components
- Visually Impaired User
- Google Gemini
- Google Cloud Storage
Connections 16
- Visually Impaired User → Voice Copilot: speaks request
README.md - Voice Copilot → Intent Routing: transcribes and routes
README.md - Intent Routing → Signboard Interpretation: dispatches signboard queries
README.md - Intent Routing → Person Recognition Client: dispatches person queries
README.md - Signboard Interpretation → On-device OCR: uses extracted text
README.md - Signboard Interpretation → Google Gemini: interprets image and text
README.md - Person Recognition Client → Recognition API: sends frame requests
README.md - Recognition API → Person Analysis & Registration: calls analysis or registration
backend/app/main.py - Recognition API → API Schemas: validates and returns
backend/app/main.py - Person Analysis & Registration → Recognition Settings: uses thresholds and paths
backend/app/recognition.py - Person Analysis & Registration → Face Embedding Store: matches or adds embeddings
backend/app/recognition.py - Person Analysis & Registration → Face Image Backup: uploads registered face
backend/app/recognition.py - Face Image Backup → Google Cloud Storage: uploads image
backend/app/storage.py - Voice Copilot → Speech & Haptics: speaks prioritized response
README.md - Speech & Haptics → Visually Impaired User: returns spoken and haptic feedback
README.md - Face Image Backup → Google Cloud Storage: optionally backs up
README.md