# khalaimovp/openglasses architecture

> Architecture diagram of the GitHub repository khalaimovp/openglasses, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khalaimovp/openglasses
- Repository: https://github.com/khalaimovp/openglasses

## Overview

OpenGlasses is a voice-first AI assistant for Meta smart glasses, with iPhone controls and companion surfaces. The main workflow is a spoken request, model selection and response, optional native-tool execution, then an answer delivered by voice or glasses HUD. The map also preserves documented offline voice, accessibility, field guidance, memory, and MCP capabilities. The README and file tree support these product areas; source excerpts were unavailable, so uncertain internal wiring is omitted.

Main entry points: [`OpenGlasses/Sources/Services/NativeTools/NativeToolRouter.swift`](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/NativeTools/NativeToolRouter.swift) (Native tool routing), [`OpenGlasses/Sources/Services/Flow/ConversationFlowEngine.swift`](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Flow/ConversationFlowEngine.swift) (Conversation flow), [`OpenGlasses/Sources/Services/LLMService.swift`](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LLMService.swift) (LLM service), [`OpenGlasses/Sources/Services/GlassesConnectionService.swift`](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/GlassesConnectionService.swift) (Glasses connection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experience["User experience"]
  node_app["iPhone app"]
  node_voice_ui["Voice interface<br/>[VoiceTab.swift]"]
  node_shortcuts["Siri intents"]
  node_widget["Listening widget"]
end

subgraph group_conversation["Voice and AI"]
  node_flow["Conversation flow"]
  node_routing["Model routing"]
  node_llm["LLM service<br/>[LLMService.swift]"]
  node_local_llm["On-device AI"]
  node_asr["Speech recognition<br/>[ASREngine.swift]"]
  node_tts["Speech output"]
end

subgraph group_capabilities["Assistant capabilities"]
  node_tool_router["Native tool routing"]
  node_accessibility["Accessibility assistance"]
  node_memory["Memory and recall"]
  node_translation["Live translation"]
  node_capture["Capture flows"]
end

subgraph group_device["Glasses and guidance"]
  node_glasses["Glasses connection"]
  node_hud["In-lens HUD<br/>[HUDRouter.swift]"]
  node_field["Field guidance"]
end

subgraph group_ecosystem["Integrations"]
  node_mcp["MCP servers<br/>[MCPClient.swift]"]
  node_carplay["CarPlay companion"]
end

node_user(("User"))
node_model_providers{{"AI model providers"}}
node_tool_services{{"External tool services"}}

node_user -->|"speaks to"| node_voice_ui
node_user -->|"asks via Siri"| node_shortcuts
node_user -->|"controls listening"| node_widget
node_voice_ui -->|"submits request"| node_flow
node_shortcuts -->|"submits question"| node_flow
node_flow -->|"transcribes audio"| node_asr
node_flow -->|"selects model"| node_routing
node_routing -->|"routes request"| node_llm
node_llm -->|"queries model"| node_model_providers
node_local_llm -.->|"runs local model"| node_model_providers
node_llm -->|"invokes tools"| node_tool_router
node_tool_router -.->|"accesses services"| node_tool_services
node_flow -->|"speaks answer"| node_tts
node_flow -->|"uses glasses"| node_glasses
node_glasses -->|"displays guidance"| node_hud
node_tool_router -->|"supports access"| node_accessibility
node_tool_router -->|"provides recall"| node_memory
node_tool_router -->|"offers translation"| node_translation
node_tool_router -->|"runs capture flows"| node_capture
node_field -->|"guides field work"| node_glasses
node_tool_router -.->|"uses MCP tools"| node_mcp
node_user -->|"uses companion"| node_carplay

click node_app "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/OpenGlassesApp.swift"
click node_voice_ui "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/Views/VoiceTab.swift"
click node_shortcuts "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/Intents/AskOpenGlassesIntent.swift"
click node_widget "https://github.com/khalaimovp/openglasses/blob/main/GlassesActivityWidget/ListeningControlWidget.swift"
click node_flow "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Flow/ConversationFlowEngine.swift"
click node_routing "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Flow/ModelRoutingPolicy.swift"
click node_llm "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LLMService.swift"
click node_local_llm "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LocalLLMService.swift"
click node_asr "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/ASR/ASREngine.swift"
click node_tts "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/TTS/TTSEngineSelector.swift"
click node_tool_router "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/NativeTools/NativeToolRouter.swift"
click node_accessibility "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Accessibility/AssistiveRouter.swift"
click node_memory "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Memory/RecallService.swift"
click node_translation "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LiveTranslationService.swift"
click node_capture "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/CaptureFlow/CaptureFlowService.swift"
click node_glasses "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/GlassesConnectionService.swift"
click node_hud "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Display/HUDRouter.swift"
click node_field "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/FieldAssist/FieldSessionService.swift"
click node_mcp "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/MCPClient.swift"
click node_carplay "https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/CarPlaySceneDelegate.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_voice_ui,node_shortcuts,node_widget,node_user toneBlue
class node_flow,node_routing,node_llm,node_local_llm,node_asr,node_tts toneAmber
class node_tool_router,node_accessibility,node_memory,node_translation,node_capture toneMint
class node_glasses,node_hud,node_field toneRose
class node_mcp,node_carplay,node_model_providers,node_tool_services toneIndigo
```

## Components

### User experience

- [iPhone app](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/OpenGlassesApp.swift): `OpenGlasses/Sources/App/OpenGlassesApp.swift`
- [Voice interface](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/Views/VoiceTab.swift): `OpenGlasses/Sources/App/Views/VoiceTab.swift`
- [Siri intents](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/Intents/AskOpenGlassesIntent.swift): `OpenGlasses/Sources/App/Intents/AskOpenGlassesIntent.swift`
- [Listening widget](https://github.com/khalaimovp/openglasses/blob/main/GlassesActivityWidget/ListeningControlWidget.swift): `GlassesActivityWidget/ListeningControlWidget.swift`

### Voice and AI

- [Conversation flow](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Flow/ConversationFlowEngine.swift): `OpenGlasses/Sources/Services/Flow/ConversationFlowEngine.swift`
- [Model routing](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Flow/ModelRoutingPolicy.swift): `OpenGlasses/Sources/Services/Flow/ModelRoutingPolicy.swift`
- [LLM service](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LLMService.swift): `OpenGlasses/Sources/Services/LLMService.swift`
- [On-device AI](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LocalLLMService.swift): `OpenGlasses/Sources/Services/LocalLLMService.swift`
- [Speech recognition](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/ASR/ASREngine.swift): `OpenGlasses/Sources/Services/ASR/ASREngine.swift`
- [Speech output](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/TTS/TTSEngineSelector.swift): `OpenGlasses/Sources/Services/TTS/TTSEngineSelector.swift`

### Assistant capabilities

- [Native tool routing](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/NativeTools/NativeToolRouter.swift): `OpenGlasses/Sources/Services/NativeTools/NativeToolRouter.swift`
- [Accessibility assistance](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Accessibility/AssistiveRouter.swift): `OpenGlasses/Sources/Services/Accessibility/AssistiveRouter.swift`
- [Memory and recall](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Memory/RecallService.swift): `OpenGlasses/Sources/Services/Memory/RecallService.swift`
- [Live translation](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/LiveTranslationService.swift): `OpenGlasses/Sources/Services/LiveTranslationService.swift`
- [Capture flows](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/CaptureFlow/CaptureFlowService.swift): `OpenGlasses/Sources/Services/CaptureFlow/CaptureFlowService.swift`

### Glasses and guidance

- [Glasses connection](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/GlassesConnectionService.swift): `OpenGlasses/Sources/Services/GlassesConnectionService.swift`
- [In-lens HUD](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/Display/HUDRouter.swift): `OpenGlasses/Sources/Services/Display/HUDRouter.swift`
- [Field guidance](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/FieldAssist/FieldSessionService.swift): `OpenGlasses/Sources/Services/FieldAssist/FieldSessionService.swift`

### Integrations

- [MCP servers](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/Services/MCPClient.swift): `OpenGlasses/Sources/Services/MCPClient.swift`
- [CarPlay companion](https://github.com/khalaimovp/openglasses/blob/main/OpenGlasses/Sources/App/CarPlaySceneDelegate.swift): `OpenGlasses/Sources/App/CarPlaySceneDelegate.swift`

### Other components

- **User**
- **AI model providers**
- **External tool services**

## Connections

- User → Voice interface: speaks to
- User → Siri intents: asks via Siri
- User → Listening widget: controls listening
- Voice interface → Conversation flow: submits request
- Siri intents → Conversation flow: submits question
- Conversation flow → Speech recognition: transcribes audio
- Conversation flow → Model routing: selects model
- Model routing → LLM service: routes request
- LLM service → AI model providers: queries model
- On-device AI → AI model providers: runs local model
- LLM service → Native tool routing: invokes tools
- Native tool routing → External tool services: accesses services
- Conversation flow → Speech output: speaks answer
- Conversation flow → Glasses connection: uses glasses
- Glasses connection → In-lens HUD: displays guidance
- Native tool routing → Accessibility assistance: supports access
- Native tool routing → Memory and recall: provides recall
- Native tool routing → Live translation: offers translation
- Native tool routing → Capture flows: runs capture flows
- Field guidance → Glasses connection: guides field work
- Native tool routing → MCP servers: uses MCP tools
- User → CarPlay companion: uses companion