# basedhardware/openglass architecture

> Architecture diagram of the GitHub repository basedhardware/openglass, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/basedhardware/openglass
- Repository: https://github.com/basedhardware/openglass

## Overview

OpenGlass pairs ESP32 camera glasses with an Expo app: a user connects over Bluetooth, the app receives and displays photos, describes them with a vision model, and answers questions about the accumulated descriptions. OpenAI speech output and configurable Groq, Ollama, and OpenAI integrations are also present. The graph follows relationships visible in excerpts or the source index; firmware-to-app protocol details and some provider wiring were not sampled, so those links are omitted or marked unverified.

Main entry points: [`sources/app/DeviceView.tsx`](https://github.com/basedhardware/openglass/blob/main/sources/app/DeviceView.tsx) (Device view), [`sources/app/Main.tsx`](https://github.com/basedhardware/openglass/blob/main/sources/app/Main.tsx) (Main screen), [`sources/agent/Agent.ts`](https://github.com/basedhardware/openglass/blob/main/sources/agent/Agent.ts) (Photo and answer agent), [`sources/agent/imageDescription.ts`](https://github.com/basedhardware/openglass/blob/main/sources/agent/imageDescription.ts) (Image description)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["App experience"]
  node_entry["App entry<br/>[App.tsx]"]
  node_main["Main screen<br/>[Main.tsx]"]
  node_button["Connect button<br/>[RoundButton.tsx]"]
  node_theme["UI theme<br/>[theme.ts]"]
  node_deviceview["Device view<br/>[DeviceView.tsx]"]
end

subgraph group_device["Glasses connection"]
  node_connection["Bluetooth connection<br/>[useDevice.ts]"]
  node_glasses["OpenGlass firmware<br/>[firmware.ino]"]
  node_imaging["Image rotation<br/>[imaging.ts]"]
  node_base64["Image encoding<br/>[base64.ts]"]
end

subgraph group_agent["Image and Q&amp;A"]
  node_agent["Photo and answer agent<br/>[Agent.ts]"]
  node_description["Image description"]
  node_lock["Async lock<br/>[lock.ts]"]
end

subgraph group_providers["Model services"]
  node_ollama["Ollama inference<br/>[ollama.ts]"]
  node_groq["Groq requests<br/>[groq-llama3.ts]"]
  node_openai["OpenAI audio and chat<br/>[openai.ts]"]
  node_keys["Provider settings<br/>[keys.ts]"]
end

node_user(("Glasses wearer"))
node_ollama_service{{"Ollama service"}}
node_groq_service{{"Groq service"}}
node_openai_service{{"OpenAI service"}}

node_user -->|"uses"| node_main
node_entry -->|"renders"| node_main
node_main -->|"shows"| node_button
node_main -->|"styles"| node_theme
node_main -->|"connects"| node_connection
node_connection -->|"provides device"| node_deviceview
node_deviceview -.->|"receives BLE photos"| node_glasses
node_deviceview -->|"rotates photos"| node_imaging
node_deviceview -->|"encodes display"| node_base64
node_deviceview -->|"submits photos"| node_agent
node_deviceview -->|"speaks answers"| node_openai
node_agent -->|"serializes work"| node_lock
node_agent -->|"describes photos"| node_description
node_description -->|"requests vision"| node_ollama
node_agent -->|"asks question"| node_description
node_description -->|"requests answer"| node_groq
node_ollama -->|"reads endpoint"| node_keys
node_groq -->|"reads credentials"| node_keys
node_openai -->|"reads credentials"| node_keys
node_ollama -.->|"calls inference"| node_ollama_service
node_groq -.->|"calls model"| node_groq_service
node_openai -->|"calls audio API"| node_openai_service

click node_entry "https://github.com/basedhardware/openglass/blob/main/App.tsx"
click node_main "https://github.com/basedhardware/openglass/blob/main/sources/app/Main.tsx"
click node_button "https://github.com/basedhardware/openglass/blob/main/sources/app/components/RoundButton.tsx"
click node_theme "https://github.com/basedhardware/openglass/blob/main/sources/app/components/theme.ts"
click node_connection "https://github.com/basedhardware/openglass/blob/main/sources/modules/useDevice.ts"
click node_glasses "https://github.com/basedhardware/openglass/blob/main/firmware/firmware.ino"
click node_deviceview "https://github.com/basedhardware/openglass/blob/main/sources/app/DeviceView.tsx"
click node_imaging "https://github.com/basedhardware/openglass/blob/main/sources/modules/imaging.ts"
click node_base64 "https://github.com/basedhardware/openglass/blob/main/sources/utils/base64.ts"
click node_agent "https://github.com/basedhardware/openglass/blob/main/sources/agent/Agent.ts"
click node_description "https://github.com/basedhardware/openglass/blob/main/sources/agent/imageDescription.ts"
click node_lock "https://github.com/basedhardware/openglass/blob/main/sources/utils/lock.ts"
click node_ollama "https://github.com/basedhardware/openglass/blob/main/sources/modules/ollama.ts"
click node_groq "https://github.com/basedhardware/openglass/blob/main/sources/modules/groq-llama3.ts"
click node_openai "https://github.com/basedhardware/openglass/blob/main/sources/modules/openai.ts"
click node_keys "https://github.com/basedhardware/openglass/blob/main/sources/keys.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_main,node_button,node_theme,node_deviceview toneBlue
class node_connection,node_glasses,node_imaging,node_base64 toneAmber
class node_agent,node_description,node_lock toneMint
class node_ollama,node_groq,node_openai,node_keys toneRose
class node_user,node_ollama_service,node_groq_service,node_openai_service toneIndigo
```

## Components

### App experience

- [App entry](https://github.com/basedhardware/openglass/blob/main/App.tsx): `App.tsx`
- [Main screen](https://github.com/basedhardware/openglass/blob/main/sources/app/Main.tsx): `sources/app/Main.tsx`
- [Connect button](https://github.com/basedhardware/openglass/blob/main/sources/app/components/RoundButton.tsx): `sources/app/components/RoundButton.tsx`
- [UI theme](https://github.com/basedhardware/openglass/blob/main/sources/app/components/theme.ts): `sources/app/components/theme.ts`
- [Device view](https://github.com/basedhardware/openglass/blob/main/sources/app/DeviceView.tsx): `sources/app/DeviceView.tsx`

### Glasses connection

- [Bluetooth connection](https://github.com/basedhardware/openglass/blob/main/sources/modules/useDevice.ts): `sources/modules/useDevice.ts`
- [OpenGlass firmware](https://github.com/basedhardware/openglass/blob/main/firmware/firmware.ino): `firmware/firmware.ino`
- [Image rotation](https://github.com/basedhardware/openglass/blob/main/sources/modules/imaging.ts): `sources/modules/imaging.ts`
- [Image encoding](https://github.com/basedhardware/openglass/blob/main/sources/utils/base64.ts): `sources/utils/base64.ts`

### Image and Q&A

- [Photo and answer agent](https://github.com/basedhardware/openglass/blob/main/sources/agent/Agent.ts): `sources/agent/Agent.ts`
- [Image description](https://github.com/basedhardware/openglass/blob/main/sources/agent/imageDescription.ts): `sources/agent/imageDescription.ts`
- [Async lock](https://github.com/basedhardware/openglass/blob/main/sources/utils/lock.ts): `sources/utils/lock.ts`

### Model services

- [Ollama inference](https://github.com/basedhardware/openglass/blob/main/sources/modules/ollama.ts): `sources/modules/ollama.ts`
- [Groq requests](https://github.com/basedhardware/openglass/blob/main/sources/modules/groq-llama3.ts): `sources/modules/groq-llama3.ts`
- [OpenAI audio and chat](https://github.com/basedhardware/openglass/blob/main/sources/modules/openai.ts): `sources/modules/openai.ts`
- [Provider settings](https://github.com/basedhardware/openglass/blob/main/sources/keys.ts): `sources/keys.ts`

### Other components

- **Glasses wearer**
- **Ollama service**
- **Groq service**
- **OpenAI service**

## Connections

- Glasses wearer → Main screen: uses (evidence: `README.md`)
- App entry → Main screen: renders (evidence: `App.tsx`)
- Main screen → Connect button: shows (evidence: `sources/app/Main.tsx`)
- Main screen → UI theme: styles (evidence: `sources/app/Main.tsx`)
- Main screen → Bluetooth connection: connects (evidence: `sources/app/Main.tsx`)
- Bluetooth connection → Device view: provides device (evidence: `sources/app/Main.tsx`)
- Device view → OpenGlass firmware: receives BLE photos
- Device view → Image rotation: rotates photos (evidence: `sources/app/DeviceView.tsx`)
- Device view → Image encoding: encodes display (evidence: `sources/app/DeviceView.tsx`)
- Device view → Photo and answer agent: submits photos (evidence: `sources/app/DeviceView.tsx`)
- Device view → OpenAI audio and chat: speaks answers (evidence: `sources/app/DeviceView.tsx`)
- Photo and answer agent → Async lock: serializes work (evidence: `sources/agent/Agent.ts`)
- Photo and answer agent → Image description: describes photos (evidence: `sources/agent/Agent.ts`)
- Image description → Ollama inference: requests vision (evidence: `sources/agent/imageDescription.ts`)
- Photo and answer agent → Image description: asks question (evidence: `sources/agent/Agent.ts`)
- Image description → Groq requests: requests answer (evidence: `sources/agent/imageDescription.ts`)
- Ollama inference → Provider settings: reads endpoint (evidence: `sources/modules/ollama.ts`)
- Groq requests → Provider settings: reads credentials (evidence: `sources/modules/groq-llama3.ts`)
- OpenAI audio and chat → Provider settings: reads credentials (evidence: `sources/modules/openai.ts`)
- Ollama inference → Ollama service: calls inference
- Groq requests → Groq service: calls model
- OpenAI audio and chat → OpenAI service: calls audio API (evidence: `sources/modules/openai.ts`)
