# buriburi-zaeemon/unmute architecture

> Architecture diagram of the GitHub repository buriburi-zaeemon/unmute, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/buriburi-zaeemon/unmute
- Repository: https://github.com/buriburi-zaeemon/unmute

## Overview

Unmute is a bilingual ASL/ISL sign-language translation application. Its central workflow sends live camera frames or uploaded video from a browser to a FastAPI service, which extracts hand landmarks, recognizes signs, and returns translation results; the documented pipeline also includes temporal filtering and sentence processing. The repository includes separate custom-gesture, practice/dictionary/3D-guide, and static-model training capabilities. Some runtime wiring and the frontend’s internal feature boundaries are not shown in the sampled source, so those relationships are omitted or marked unverified.

Main entry points: [`backend/main.py`](https://github.com/buriburi-zaeemon/unmute/blob/main/backend/main.py) (FastAPI service), [`sign_engine/video_processor.py`](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/video_processor.py) (Video translation), [`nlp/sentence_processor.py`](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/sentence_processor.py) (Sentence processor), [`static/app.js`](https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js) (Web application)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Browser experience"]
  node_webapp["Web application<br/>[app.js]"]
  node_practice["Practice and dictionary<br/>[app.js]"]
  node_guide["3D sign guide<br/>[app.js]"]
end

subgraph group_service["Translation service"]
  node_api["FastAPI service<br/>[main.py]"]
  node_video["Video translation<br/>[video_processor.py]"]
end

subgraph group_recognition["Sign recognition"]
  node_landmarks["Hand landmarks"]
  node_features["Feature engineering"]
  node_classifier["ASL classifier<br/>[asl_classifier.py]"]
  node_temporal["Temporal gestures"]
  node_custom["Custom gesture trainer<br/>[gesture_trainer.py]"]
end

subgraph group_language["Sentence processing"]
  node_tokens["Token contract<br/>[token_schema.py]"]
  node_normalizer["Token normalization"]
  node_buffer["Sequence buffer<br/>[sequence_buffer.py]"]
  node_sentence["Sentence processor"]
  node_grammar["Grammar rules<br/>[grammar_rules.py]"]
  node_vocabulary["Language vocabulary<br/>[vocabulary.py]"]
end

subgraph group_ml["Model development"]
  node_dataset["Static sign datasets<br/>[dataset.py]"]
  node_labels["Class labels<br/>[labels.py]"]
  node_training["Static model training<br/>[train_static.py]"]
  node_mlp["Static MLP models<br/>[static_mlp.py]"]
end

node_user(("Signer"))
node_video_file["Video file"]

node_user -->|"uses"| node_webapp
node_webapp -->|"sends frames"| node_api
node_api -->|"returns results"| node_webapp
node_user -->|"provides"| node_video_file
node_webapp -->|"uploads video"| node_api
node_api -.->|"processes video"| node_video
node_video_file -->|"is analyzed by"| node_video
node_api -->|"invokes"| node_landmarks
node_api -->|"invokes"| node_features
node_api -->|"invokes"| node_classifier
node_api -->|"invokes"| node_temporal
node_api -->|"invokes"| node_custom
node_video -->|"extracts"| node_landmarks
node_video -->|"computes"| node_features
node_video -->|"classifies"| node_classifier
node_video -->|"tracks gestures"| node_temporal
node_video -->|"uses trainer"| node_custom
node_classifier -->|"extracts features"| node_features
node_classifier -->|"accepts landmarks"| node_landmarks
node_api -.->|"processes recognized text"| node_sentence
node_sentence -->|"normalizes with"| node_normalizer
node_sentence -->|"buffers with"| node_buffer
node_sentence -->|"applies rules"| node_grammar
node_sentence -->|"uses vocabulary"| node_vocabulary
node_normalizer -->|"normalizes"| node_tokens
node_buffer -->|"stores"| node_tokens
node_grammar -->|"transforms"| node_tokens
node_grammar -->|"uses vocabulary"| node_vocabulary
node_training -->|"loads batches"| node_dataset
node_training -->|"trains"| node_mlp
node_dataset -->|"maps classes"| node_labels
node_training -->|"selects classes"| node_labels
node_practice -->|"offers sign guides"| node_guide

click node_webapp "https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js"
click node_practice "https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js"
click node_guide "https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js"
click node_api "https://github.com/buriburi-zaeemon/unmute/blob/main/backend/main.py"
click node_video "https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/video_processor.py"
click node_landmarks "https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/landmark_extractor.py"
click node_features "https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/feature_engineering.py"
click node_classifier "https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/asl_classifier.py"
click node_temporal "https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/temporal_tracker.py"
click node_custom "https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/gesture_trainer.py"
click node_tokens "https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/token_schema.py"
click node_normalizer "https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/token_normalizer.py"
click node_buffer "https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/sequence_buffer.py"
click node_sentence "https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/sentence_processor.py"
click node_grammar "https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/grammar_rules.py"
click node_vocabulary "https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/vocabulary.py"
click node_dataset "https://github.com/buriburi-zaeemon/unmute/blob/main/ml/data/dataset.py"
click node_labels "https://github.com/buriburi-zaeemon/unmute/blob/main/ml/data/labels.py"
click node_training "https://github.com/buriburi-zaeemon/unmute/blob/main/ml/training/train_static.py"
click node_mlp "https://github.com/buriburi-zaeemon/unmute/blob/main/ml/models/static_mlp.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webapp,node_practice,node_guide toneBlue
class node_api,node_video toneAmber
class node_landmarks,node_features,node_classifier,node_temporal,node_custom toneMint
class node_tokens,node_normalizer,node_buffer,node_sentence,node_grammar,node_vocabulary toneRose
class node_dataset,node_labels,node_training,node_mlp,node_user,node_video_file toneIndigo
```

## Components

### Browser experience

- [Web application](https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js): `static/app.js`
- [Practice and dictionary](https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js): `static/app.js`
- [3D sign guide](https://github.com/buriburi-zaeemon/unmute/blob/main/static/app.js): `static/app.js`

### Translation service

- [FastAPI service](https://github.com/buriburi-zaeemon/unmute/blob/main/backend/main.py): `backend/main.py`
- [Video translation](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/video_processor.py): `sign_engine/video_processor.py`

### Sign recognition

- [Hand landmarks](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/landmark_extractor.py): `sign_engine/landmark_extractor.py`
- [Feature engineering](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/feature_engineering.py): `sign_engine/feature_engineering.py`
- [ASL classifier](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/asl_classifier.py): `sign_engine/asl_classifier.py`
- [Temporal gestures](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/temporal_tracker.py): `sign_engine/temporal_tracker.py`
- [Custom gesture trainer](https://github.com/buriburi-zaeemon/unmute/blob/main/sign_engine/gesture_trainer.py): `sign_engine/gesture_trainer.py`

### Sentence processing

- [Token contract](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/token_schema.py): `nlp/token_schema.py`
- [Token normalization](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/token_normalizer.py): `nlp/token_normalizer.py`
- [Sequence buffer](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/sequence_buffer.py): `nlp/sequence_buffer.py`
- [Sentence processor](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/sentence_processor.py): `nlp/sentence_processor.py`
- [Grammar rules](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/grammar_rules.py): `nlp/grammar_rules.py`
- [Language vocabulary](https://github.com/buriburi-zaeemon/unmute/blob/main/nlp/vocabulary.py): `nlp/vocabulary.py`

### Model development

- [Static sign datasets](https://github.com/buriburi-zaeemon/unmute/blob/main/ml/data/dataset.py): `ml/data/dataset.py`
- [Class labels](https://github.com/buriburi-zaeemon/unmute/blob/main/ml/data/labels.py): `ml/data/labels.py`
- [Static model training](https://github.com/buriburi-zaeemon/unmute/blob/main/ml/training/train_static.py): `ml/training/train_static.py`
- [Static MLP models](https://github.com/buriburi-zaeemon/unmute/blob/main/ml/models/static_mlp.py): `ml/models/static_mlp.py`

### Other components

- **Signer**
- **Video file**

## Connections

- Signer → Web application: uses (evidence: `README.md`)
- Web application → FastAPI service: sends frames (evidence: `README.md`)
- FastAPI service → Web application: returns results (evidence: `README.md`)
- Signer → Video file: provides (evidence: `README.md`)
- Web application → FastAPI service: uploads video (evidence: `README.md`)
- FastAPI service → Video translation: processes video (evidence: `backend/main.py`)
- Video file → Video translation: is analyzed by (evidence: `sign_engine/video_processor.py`)
- FastAPI service → Hand landmarks: invokes (evidence: `backend/main.py`)
- FastAPI service → Feature engineering: invokes (evidence: `backend/main.py`)
- FastAPI service → ASL classifier: invokes (evidence: `backend/main.py`)
- FastAPI service → Temporal gestures: invokes (evidence: `backend/main.py`)
- FastAPI service → Custom gesture trainer: invokes (evidence: `backend/main.py`)
- Video translation → Hand landmarks: extracts (evidence: `sign_engine/video_processor.py`)
- Video translation → Feature engineering: computes (evidence: `sign_engine/video_processor.py`)
- Video translation → ASL classifier: classifies (evidence: `sign_engine/video_processor.py`)
- Video translation → Temporal gestures: tracks gestures (evidence: `sign_engine/video_processor.py`)
- Video translation → Custom gesture trainer: uses trainer (evidence: `sign_engine/video_processor.py`)
- ASL classifier → Feature engineering: extracts features (evidence: `sign_engine/asl_classifier.py`)
- ASL classifier → Hand landmarks: accepts landmarks (evidence: `sign_engine/asl_classifier.py`)
- FastAPI service → Sentence processor: processes recognized text
- Sentence processor → Token normalization: normalizes with (evidence: `nlp/sentence_processor.py`)
- Sentence processor → Sequence buffer: buffers with (evidence: `nlp/sentence_processor.py`)
- Sentence processor → Grammar rules: applies rules (evidence: `nlp/sentence_processor.py`)
- Sentence processor → Language vocabulary: uses vocabulary (evidence: `nlp/sentence_processor.py`)
- Token normalization → Token contract: normalizes (evidence: `nlp/token_normalizer.py`)
- Sequence buffer → Token contract: stores (evidence: `nlp/sequence_buffer.py`)
- Grammar rules → Token contract: transforms (evidence: `nlp/grammar_rules.py`)
- Grammar rules → Language vocabulary: uses vocabulary (evidence: `nlp/grammar_rules.py`)
- Static model training → Static sign datasets: loads batches (evidence: `ml/training/train_static.py`)
- Static model training → Static MLP models: trains (evidence: `ml/training/train_static.py`)
- Static sign datasets → Class labels: maps classes (evidence: `ml/data/dataset.py`)
- Static model training → Class labels: selects classes (evidence: `ml/training/train_static.py`)
- Practice and dictionary → 3D sign guide: offers sign guides (evidence: `README.md`)
