How basedhardware/openglass works
OpenGlass pairs ESP32 camera glasses with an Expo app: a user connects over Bluetooth, the app receives and displays photos, describes them with a vision model, and answers questions about the accumulated descriptions. OpenAI speech output and configurable Groq, Ollama, and OpenAI integrations are also present. The graph follows relationships visible in excerpts or the source index; firmware-to-app protocol details and some provider wiring were not sampled, so those links are omitted or marked unverified.
Main entry points: DeviceView.tsx Device view · Main.tsx Main screen · Agent.ts Photo and answer agent · imageDescription.ts Image description
Components 20
App experience
- App entry
App.tsx - Main screen
sources/app/Main.tsx - Connect button
sources/app/components/RoundButton.tsx - UI theme
sources/app/components/theme.ts - Device view
sources/app/DeviceView.tsx
Glasses connection
- Bluetooth connection
sources/modules/useDevice.ts - OpenGlass firmware
firmware/firmware.ino - Image rotation
sources/modules/imaging.ts - Image encoding
sources/utils/base64.ts
Image and Q&A
- Photo and answer agent
sources/agent/Agent.ts - Image description
sources/agent/imageDescription.ts - Async lock
sources/utils/lock.ts
Model services
- Ollama inference
sources/modules/ollama.ts - Groq requests
sources/modules/groq-llama3.ts - OpenAI audio and chat
sources/modules/openai.ts - Provider settings
sources/keys.ts
Other components
- Glasses wearer
- Ollama service
- Groq service
- OpenAI service
Connections 22
- Glasses wearer → Main screen: uses
README.md - App entry → Main screen: renders
App.tsx - Main screen → Connect button: shows
sources/app/Main.tsx - Main screen → UI theme: styles
sources/app/Main.tsx - Main screen → Bluetooth connection: connects
sources/app/Main.tsx - Bluetooth connection → Device view: provides device
sources/app/Main.tsx - Device view → OpenGlass firmware: receives BLE photos
- Device view → Image rotation: rotates photos
sources/app/DeviceView.tsx - Device view → Image encoding: encodes display
sources/app/DeviceView.tsx - Device view → Photo and answer agent: submits photos
sources/app/DeviceView.tsx - Device view → OpenAI audio and chat: speaks answers
sources/app/DeviceView.tsx - Photo and answer agent → Async lock: serializes work
sources/agent/Agent.ts - Photo and answer agent → Image description: describes photos
sources/agent/Agent.ts - Image description → Ollama inference: requests vision
sources/agent/imageDescription.ts - Photo and answer agent → Image description: asks question
sources/agent/Agent.ts - Image description → Groq requests: requests answer
sources/agent/imageDescription.ts - Ollama inference → Provider settings: reads endpoint
sources/modules/ollama.ts - Groq requests → Provider settings: reads credentials
sources/modules/groq-llama3.ts - OpenAI audio and chat → Provider settings: reads credentials
sources/modules/openai.ts - Ollama inference → Ollama service: calls inference
- Groq requests → Groq service: calls model
- OpenAI audio and chat → OpenAI service: calls audio API
sources/modules/openai.ts