# hurrypatter/asl-glasses architecture

> Architecture diagram of the GitHub repository hurrypatter/asl-glasses, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hurrypatter/asl-glasses
- Repository: https://github.com/hurrypatter/asl-glasses

## Overview

ASL Smart Glasses performs on-device sign recognition, turning camera frames into committed text and optional speech. The live path detects one hand, recognizes J/Z from motion or static handshapes with a trained classifier, debounces commits, then corrects fingerspelled runs and expands word-sign tokens. Separate workflows collect signer-labeled landmarks, train the model, and evaluate the complete pipeline under recorded conditions. Some wiring is supported by README documentation rather than sampled source; unsampled details are omitted.

Main entry points: [`main.py`](https://github.com/hurrypatter/asl-glasses/blob/main/main.py) (Live application), [`evaluate.py`](https://github.com/hurrypatter/asl-glasses/blob/main/evaluate.py) (End-to-end evaluation), [`collect_data.py`](https://github.com/hurrypatter/asl-glasses/blob/main/collect_data.py) (Data collection), [`train_classifier.py`](https://github.com/hurrypatter/asl-glasses/blob/main/train_classifier.py) (Model training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_live["Live translation"]
  node_main["Live application<br/>[main.py]"]
  node_display["On-screen output<br/>[main.py]"]
end

subgraph group_recognition["Recognition"]
  node_landmarker["Hand Landmarker"]
  node_task["MediaPipe model"]
  node_motion["Motion detection<br/>[motion.py]"]
  node_classifier["Static classifier"]
  node_labels["Classifier labels"]
  node_debouncer["Commit debouncer<br/>[debouncer.py]"]
end

subgraph group_language["Text output"]
  node_nlp["Text bridge<br/>[nlp_bridge.py]"]
  node_audio["Speech output<br/>[audio.py]"]
end

subgraph group_data["Data and evaluation"]
  node_collector["Data collection<br/>[collect_data.py]"]
  node_dataset["Dataset schema<br/>[dataset.py]"]
  node_landmarks[("Landmark data<br/>[landmark_data.csv]")]
  node_trainer["Model training"]
  node_evaluator["End-to-end evaluation<br/>[evaluate.py]"]
  node_results[("Evaluation results<br/>[eval_results.csv]")]
end

node_signer(("Signer"))
node_camera(("Camera"))
node_symspell["SymSpell"]
node_powershell["PowerShell speech"]

node_signer -->|"signs to"| node_camera
node_camera -->|"provides frames"| node_main
node_main -->|"detects hands"| node_landmarker
node_landmarker -->|"uses model"| node_task
node_main -->|"checks trajectories"| node_motion
node_main -->|"classifies features"| node_classifier
node_main -->|"loads labels"| node_labels
node_main -->|"commits predictions"| node_debouncer
node_main -->|"corrects commits"| node_nlp
node_nlp -->|"segments text"| node_symspell
node_main -->|"speaks text"| node_audio
node_audio -.->|"invokes speech"| node_powershell
node_main -->|"renders status"| node_display
node_signer -->|"selects labels"| node_collector
node_collector -->|"detects hands"| node_landmarker
node_collector -->|"uses schema"| node_dataset
node_collector -->|"appends features"| node_landmarks
node_trainer -->|"reads samples"| node_landmarks
node_trainer -->|"writes model"| node_classifier
node_trainer -->|"writes labels"| node_labels
node_signer -->|"provides targets"| node_evaluator
node_evaluator -->|"detects hands"| node_landmarker
node_evaluator -->|"checks trajectories"| node_motion
node_evaluator -->|"classifies features"| node_classifier
node_evaluator -->|"loads labels"| node_labels
node_evaluator -->|"commits predictions"| node_debouncer
node_evaluator -->|"appends scores"| node_results

click node_main "https://github.com/hurrypatter/asl-glasses/blob/main/main.py"
click node_task "https://github.com/hurrypatter/asl-glasses/blob/main/hand_landmarker.task"
click node_motion "https://github.com/hurrypatter/asl-glasses/blob/main/motion.py"
click node_classifier "https://github.com/hurrypatter/asl-glasses/blob/main/landmark_model.joblib"
click node_labels "https://github.com/hurrypatter/asl-glasses/blob/main/landmark_labels.json"
click node_debouncer "https://github.com/hurrypatter/asl-glasses/blob/main/debouncer.py"
click node_nlp "https://github.com/hurrypatter/asl-glasses/blob/main/nlp_bridge.py"
click node_audio "https://github.com/hurrypatter/asl-glasses/blob/main/audio.py"
click node_collector "https://github.com/hurrypatter/asl-glasses/blob/main/collect_data.py"
click node_dataset "https://github.com/hurrypatter/asl-glasses/blob/main/dataset.py"
click node_landmarks "https://github.com/hurrypatter/asl-glasses/blob/main/landmark_data.csv"
click node_trainer "https://github.com/hurrypatter/asl-glasses/blob/main/train_classifier.py"
click node_evaluator "https://github.com/hurrypatter/asl-glasses/blob/main/evaluate.py"
click node_results "https://github.com/hurrypatter/asl-glasses/blob/main/eval_results.csv"
click node_display "https://github.com/hurrypatter/asl-glasses/blob/main/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_display toneBlue
class node_landmarker,node_task,node_motion,node_classifier,node_labels,node_debouncer toneAmber
class node_nlp,node_audio toneMint
class node_collector,node_dataset,node_landmarks,node_trainer,node_evaluator,node_results toneRose
class node_signer,node_camera,node_symspell,node_powershell toneIndigo
```

## Components

### Live translation

- [Live application](https://github.com/hurrypatter/asl-glasses/blob/main/main.py): `main.py`
- [On-screen output](https://github.com/hurrypatter/asl-glasses/blob/main/main.py): `main.py`

### Recognition

- **Hand Landmarker**
- [MediaPipe model](https://github.com/hurrypatter/asl-glasses/blob/main/hand_landmarker.task): `hand_landmarker.task`
- [Motion detection](https://github.com/hurrypatter/asl-glasses/blob/main/motion.py): `motion.py`
- [Static classifier](https://github.com/hurrypatter/asl-glasses/blob/main/landmark_model.joblib): `landmark_model.joblib`
- [Classifier labels](https://github.com/hurrypatter/asl-glasses/blob/main/landmark_labels.json): `landmark_labels.json`
- [Commit debouncer](https://github.com/hurrypatter/asl-glasses/blob/main/debouncer.py): `debouncer.py`

### Text output

- [Text bridge](https://github.com/hurrypatter/asl-glasses/blob/main/nlp_bridge.py): `nlp_bridge.py`
- [Speech output](https://github.com/hurrypatter/asl-glasses/blob/main/audio.py): `audio.py`

### Data and evaluation

- [Data collection](https://github.com/hurrypatter/asl-glasses/blob/main/collect_data.py): `collect_data.py`
- [Dataset schema](https://github.com/hurrypatter/asl-glasses/blob/main/dataset.py): `dataset.py`
- [Landmark data](https://github.com/hurrypatter/asl-glasses/blob/main/landmark_data.csv): `landmark_data.csv`
- [Model training](https://github.com/hurrypatter/asl-glasses/blob/main/train_classifier.py): `train_classifier.py`
- [End-to-end evaluation](https://github.com/hurrypatter/asl-glasses/blob/main/evaluate.py): `evaluate.py`
- [Evaluation results](https://github.com/hurrypatter/asl-glasses/blob/main/eval_results.csv): `eval_results.csv`

### Other components

- **Signer**
- **Camera**
- **SymSpell**
- **PowerShell speech**

## Connections

- Signer → Camera: signs to
- Camera → Live application: provides frames
- Live application → Hand Landmarker: detects hands
- Hand Landmarker → MediaPipe model: uses model
- Live application → Motion detection: checks trajectories
- Live application → Static classifier: classifies features
- Live application → Classifier labels: loads labels
- Live application → Commit debouncer: commits predictions
- Live application → Text bridge: corrects commits
- Text bridge → SymSpell: segments text
- Live application → Speech output: speaks text
- Speech output → PowerShell speech: invokes speech
- Live application → On-screen output: renders status
- Signer → Data collection: selects labels
- Data collection → Hand Landmarker: detects hands
- Data collection → Dataset schema: uses schema
- Data collection → Landmark data: appends features
- Model training → Landmark data: reads samples
- Model training → Static classifier: writes model
- Model training → Classifier labels: writes labels
- Signer → End-to-end evaluation: provides targets
- End-to-end evaluation → Hand Landmarker: detects hands
- End-to-end evaluation → Motion detection: checks trajectories
- End-to-end evaluation → Static classifier: classifies features
- End-to-end evaluation → Classifier labels: loads labels
- End-to-end evaluation → Commit debouncer: commits predictions
- End-to-end evaluation → Evaluation results: appends scores