How khalaimovp/openglasses works
OpenGlasses is a voice-first AI assistant for Meta smart glasses, with iPhone controls and companion surfaces. The main workflow is a spoken request, model selection and response, optional native-tool execution, then an answer delivered by voice or glasses HUD. The map also preserves documented offline voice, accessibility, field guidance, memory, and MCP capabilities. The README and file tree support these product areas; source excerpts were unavailable, so uncertain internal wiring is omitted.
Main entry points: NativeToolRouter.swift Native tool routing · ConversationFlowEngine.swift Conversation flow · LLMService.swift LLM service · GlassesConnectionService.swift Glasses connection
Components 23
User experience
- iPhone app
OpenGlasses/Sources/App/OpenGlassesApp.swift - Voice interface
OpenGlasses/Sources/App/Views/VoiceTab.swift - Siri intents
OpenGlasses/Sources/App/Intents/AskOpenGlassesIntent.swift - Listening widget
GlassesActivityWidget/ListeningControlWidget.swift
Voice and AI
- Conversation flow
OpenGlasses/Sources/Services/Flow/ConversationFlowEngine.swift - Model routing
OpenGlasses/Sources/Services/Flow/ModelRoutingPolicy.swift - LLM service
OpenGlasses/Sources/Services/LLMService.swift - On-device AI
OpenGlasses/Sources/Services/LocalLLMService.swift - Speech recognition
OpenGlasses/Sources/Services/ASR/ASREngine.swift - Speech output
OpenGlasses/Sources/Services/TTS/TTSEngineSelector.swift
Assistant capabilities
- Native tool routing
OpenGlasses/Sources/Services/NativeTools/NativeToolRouter.swift - Accessibility assistance
OpenGlasses/Sources/Services/Accessibility/AssistiveRouter.swift - Memory and recall
OpenGlasses/Sources/Services/Memory/RecallService.swift - Live translation
OpenGlasses/Sources/Services/LiveTranslationService.swift - Capture flows
OpenGlasses/Sources/Services/CaptureFlow/CaptureFlowService.swift
Glasses and guidance
- Glasses connection
OpenGlasses/Sources/Services/GlassesConnectionService.swift - In-lens HUD
OpenGlasses/Sources/Services/Display/HUDRouter.swift - Field guidance
OpenGlasses/Sources/Services/FieldAssist/FieldSessionService.swift
Integrations
- MCP servers
OpenGlasses/Sources/Services/MCPClient.swift - CarPlay companion
OpenGlasses/Sources/App/CarPlaySceneDelegate.swift
Other components
- User
- AI model providers
- External tool services
Connections 22
- User → Voice interface: speaks to
- User → Siri intents: asks via Siri
- User → Listening widget: controls listening
- Voice interface → Conversation flow: submits request
- Siri intents → Conversation flow: submits question
- Conversation flow → Speech recognition: transcribes audio
- Conversation flow → Model routing: selects model
- Model routing → LLM service: routes request
- LLM service → AI model providers: queries model
- On-device AI → AI model providers: runs local model
- LLM service → Native tool routing: invokes tools
- Native tool routing → External tool services: accesses services
- Conversation flow → Speech output: speaks answer
- Conversation flow → Glasses connection: uses glasses
- Glasses connection → In-lens HUD: displays guidance
- Native tool routing → Accessibility assistance: supports access
- Native tool routing → Memory and recall: provides recall
- Native tool routing → Live translation: offers translation
- Native tool routing → Capture flows: runs capture flows
- Field guidance → Glasses connection: guides field work
- Native tool routing → MCP servers: uses MCP tools
- User → CarPlay companion: uses companion