# nsp909/img2brl architecture

> Architecture diagram of the GitHub repository nsp909/img2brl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nsp909/img2brl
- Repository: https://github.com/nsp909/img2brl

## Overview

Braille Pin Visualizer turns camera, microphone, ASL, or typed input into prioritized messages, encodes them as six-dot braille, streams cells for playback, and sends pin masks to the Pi actuator. The browser app also exposes recognition controls and status through the laptop bridge. The README documents the camera/object/text and sound stages; source excerpts confirm UI orchestration and client seams, but not the bridge’s internal detection calls, so those relationships are documented with README evidence. ASL modules are shown as a distinct documented capability; their precise runtime wiring is not expanded beyond supported references.

Main entry points: [`src/App.tsx`](https://github.com/nsp909/img2brl/blob/main/src/App.tsx) (App orchestration), [`bridge/detect_bridge.py`](https://github.com/nsp909/img2brl/blob/main/bridge/detect_bridge.py) (Priority queue and controls), [`src/lib/bridge.ts`](https://github.com/nsp909/img2brl/blob/main/src/lib/bridge.ts) (Bridge API client), [`pi/models`](https://github.com/nsp909/img2brl/tree/main/pi/models) (Object detection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_input["Recognition inputs"]
  node_camera["Pi camera stream"]
  node_mic["Microphone speech"]
end

subgraph group_bridge["Laptop bridge"]
  node_objects["Object detection"]
  node_textgate["Text gate and reader<br/>[detect_bridge.py]"]
  node_speech["Name-matched speech<br/>[sound_mode.py]"]
  node_asl["ASL recognition<br/>[asl_mode.py]"]
  node_queue["Priority queue and controls<br/>[detect_bridge.py]"]
end

subgraph group_app["Braille app"]
  node_bridgeclient["Bridge API client<br/>[bridge.ts]"]
  node_bridgehook["Bridge state hook<br/>[useBridge.ts]"]
  node_app["App orchestration<br/>[App.tsx]"]
  node_detection["Detection model<br/>[detections.ts]"]
  node_encoder["Braille encoder<br/>[braille.ts]"]
  node_stream["Timed cell stream<br/>[useCellStream.ts]"]
  node_panels["Playback and lab UI<br/>[DevView.tsx]"]
  node_hardwarehook["Hardware client hook<br/>[useHardware.ts]"]
  node_hardwareclient["Pin transport<br/>[hardware.ts]"]
end

subgraph group_device["Pi actuator"]
  node_solenoid["Pi solenoid server<br/>[solenoid_server.py]"]
  node_brailleplay["Pi braille playback<br/>[braille_play.py]"]
  node_braillemodel["Pi braille encoding<br/>[braille.py]"]
end

node_wearer(("Wearer"))

node_camera -->|"supplies frames"| node_objects
node_camera -->|"supplies frames"| node_textgate
node_mic -->|"transcribes utterances"| node_speech
node_objects -->|"queues candidates"| node_queue
node_textgate -->|"queues text"| node_queue
node_speech -->|"queues matches"| node_queue
node_wearer -->|"types or controls"| node_app
node_app -->|"reads bridge state"| node_bridgehook
node_bridgehook -->|"calls API"| node_bridgeclient
node_bridgeclient -->|"requests queue actions"| node_queue
node_queue -->|"returns detections"| node_bridgeclient
node_app -->|"creates detections"| node_detection
node_app -->|"encodes label"| node_encoder
node_app -->|"plays encoded cells"| node_stream
node_app -->|"sends current cell"| node_hardwarehook
node_app -->|"renders controls"| node_panels
node_hardwarehook -->|"calls pin API"| node_hardwareclient
node_hardwareclient -->|"sends pin frames"| node_solenoid
node_solenoid -->|"uses braille mapping"| node_braillemodel
node_wearer -.->|"presents scene"| node_camera
node_asl -.->|"provides sign input"| node_queue
node_wearer -.->|"speaks name"| node_mic

click node_objects "https://github.com/nsp909/img2brl/tree/main/pi/models"
click node_textgate "https://github.com/nsp909/img2brl/blob/main/bridge/detect_bridge.py"
click node_speech "https://github.com/nsp909/img2brl/blob/main/bridge/sound_mode.py"
click node_asl "https://github.com/nsp909/img2brl/blob/main/bridge/asl_mode.py"
click node_queue "https://github.com/nsp909/img2brl/blob/main/bridge/detect_bridge.py"
click node_bridgeclient "https://github.com/nsp909/img2brl/blob/main/src/lib/bridge.ts"
click node_bridgehook "https://github.com/nsp909/img2brl/blob/main/src/hooks/useBridge.ts"
click node_app "https://github.com/nsp909/img2brl/blob/main/src/App.tsx"
click node_detection "https://github.com/nsp909/img2brl/blob/main/src/lib/detections.ts"
click node_encoder "https://github.com/nsp909/img2brl/blob/main/src/lib/braille.ts"
click node_stream "https://github.com/nsp909/img2brl/blob/main/src/hooks/useCellStream.ts"
click node_panels "https://github.com/nsp909/img2brl/blob/main/src/components/DevView.tsx"
click node_hardwarehook "https://github.com/nsp909/img2brl/blob/main/src/hooks/useHardware.ts"
click node_hardwareclient "https://github.com/nsp909/img2brl/blob/main/src/lib/hardware.ts"
click node_solenoid "https://github.com/nsp909/img2brl/blob/main/pi/solenoid_server.py"
click node_brailleplay "https://github.com/nsp909/img2brl/blob/main/pi/braille_play.py"
click node_braillemodel "https://github.com/nsp909/img2brl/blob/main/pi/braille.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_camera,node_mic toneBlue
class node_objects,node_textgate,node_speech,node_asl,node_queue toneAmber
class node_bridgeclient,node_bridgehook,node_app,node_detection,node_encoder,node_stream,node_panels,node_hardwarehook,node_hardwareclient toneMint
class node_solenoid,node_brailleplay,node_braillemodel toneRose
class node_wearer toneIndigo
```

## Components

### Recognition inputs

- **Pi camera stream**
- **Microphone speech**

### Laptop bridge

- [Object detection](https://github.com/nsp909/img2brl/tree/main/pi/models): `pi/models`
- [Text gate and reader](https://github.com/nsp909/img2brl/blob/main/bridge/detect_bridge.py): `bridge/detect_bridge.py`
- [Name-matched speech](https://github.com/nsp909/img2brl/blob/main/bridge/sound_mode.py): `bridge/sound_mode.py`
- [ASL recognition](https://github.com/nsp909/img2brl/blob/main/bridge/asl_mode.py): `bridge/asl_mode.py`
- [Priority queue and controls](https://github.com/nsp909/img2brl/blob/main/bridge/detect_bridge.py): `bridge/detect_bridge.py`

### Braille app

- [Bridge API client](https://github.com/nsp909/img2brl/blob/main/src/lib/bridge.ts): `src/lib/bridge.ts`
- [Bridge state hook](https://github.com/nsp909/img2brl/blob/main/src/hooks/useBridge.ts): `src/hooks/useBridge.ts`
- [App orchestration](https://github.com/nsp909/img2brl/blob/main/src/App.tsx): `src/App.tsx`
- [Detection model](https://github.com/nsp909/img2brl/blob/main/src/lib/detections.ts): `src/lib/detections.ts`
- [Braille encoder](https://github.com/nsp909/img2brl/blob/main/src/lib/braille.ts): `src/lib/braille.ts`
- [Timed cell stream](https://github.com/nsp909/img2brl/blob/main/src/hooks/useCellStream.ts): `src/hooks/useCellStream.ts`
- [Playback and lab UI](https://github.com/nsp909/img2brl/blob/main/src/components/DevView.tsx): `src/components/DevView.tsx`
- [Hardware client hook](https://github.com/nsp909/img2brl/blob/main/src/hooks/useHardware.ts): `src/hooks/useHardware.ts`
- [Pin transport](https://github.com/nsp909/img2brl/blob/main/src/lib/hardware.ts): `src/lib/hardware.ts`

### Pi actuator

- [Pi solenoid server](https://github.com/nsp909/img2brl/blob/main/pi/solenoid_server.py): `pi/solenoid_server.py`
- [Pi braille playback](https://github.com/nsp909/img2brl/blob/main/pi/braille_play.py): `pi/braille_play.py`
- [Pi braille encoding](https://github.com/nsp909/img2brl/blob/main/pi/braille.py): `pi/braille.py`

### Other components

- **Wearer**

## Connections

- Pi camera stream → Object detection: supplies frames (evidence: `README.md`)
- Pi camera stream → Text gate and reader: supplies frames (evidence: `README.md`)
- Microphone speech → Name-matched speech: transcribes utterances (evidence: `README.md`)
- Object detection → Priority queue and controls: queues candidates (evidence: `README.md`)
- Text gate and reader → Priority queue and controls: queues text (evidence: `README.md`)
- Name-matched speech → Priority queue and controls: queues matches (evidence: `README.md`)
- Wearer → App orchestration: types or controls (evidence: `README.md`)
- App orchestration → Bridge state hook: reads bridge state (evidence: `src/App.tsx`)
- Bridge state hook → Bridge API client: calls API (evidence: `src/hooks/useBridge.ts`)
- Bridge API client → Priority queue and controls: requests queue actions (evidence: `src/lib/bridge.ts`)
- Priority queue and controls → Bridge API client: returns detections (evidence: `README.md`)
- App orchestration → Detection model: creates detections (evidence: `src/App.tsx`)
- App orchestration → Braille encoder: encodes label (evidence: `src/App.tsx`)
- App orchestration → Timed cell stream: plays encoded cells (evidence: `src/App.tsx`)
- App orchestration → Hardware client hook: sends current cell (evidence: `src/App.tsx`)
- App orchestration → Playback and lab UI: renders controls (evidence: `src/App.tsx`)
- Hardware client hook → Pin transport: calls pin API (evidence: `src/hooks/useHardware.ts`)
- Pin transport → Pi solenoid server: sends pin frames (evidence: `README.md`)
- Pi solenoid server → Pi braille encoding: uses braille mapping (evidence: `pi/solenoid_server.py`)
- Wearer → Pi camera stream: presents scene
- ASL recognition → Priority queue and controls: provides sign input (evidence: `bridge/detect_bridge.py`)
- Wearer → Microphone speech: speaks name
