# butcher05/american-sign-language-interpretor architecture

> Architecture diagram of the GitHub repository butcher05/american-sign-language-interpretor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/butcher05/american-sign-language-interpretor
- Repository: https://github.com/butcher05/american-sign-language-interpretor

## Overview

This is a real-time ASL interpreter with two recognition modes: fingerspelling letters and word-level signs. Its principal runtime flow sends browser webcam frames to a FastAPI WebSocket backend, which extracts landmarks, runs the selected model, and returns predictions for the dashboard and sentence buffer. Separate preprocessing and training scripts prepare and fit the two models. The README documents WLASL fine-tuning, but no corresponding script appears in the supplied tree; that capability is shown at the enclosing Code directory. Some runtime details are omitted in the excerpts, so only documented or excerpt-supported connections are drawn.

Main entry points: [`Code/Inference pipeline.py`](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py) (WebSocket server), [`Code/Preprocess ASL Citizen.py`](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Preprocess%20ASL%20Citizen.py) (ASL Citizen sequences), [`Code/Preprocess asl alphabet.py`](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Preprocess%20asl%20alphabet.py) (Alphabet landmarks), [`Code/train fingerspelling.py`](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/train%20fingerspelling.py) (MLP training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Live interpretation"]
  node_browser["Browser dashboard"]
  node_server["WebSocket server"]
  node_landmarks["Landmark extraction"]
  node_mode["Mode and sentence buffer"]
end

subgraph group_models["Recognition models"]
  node_fsmodel["Fingerspelling MLP"]
  node_wordmodel["Pose Transformer"]
  node_artifacts["Model weights"]
end

subgraph group_data["Data preparation"]
  node_fsdata["Alphabet landmarks"]
  node_worddata["ASL Citizen sequences"]
end

subgraph group_training["Model training"]
  node_fs_train["MLP training"]
  node_word_train["Transformer pretraining"]
  node_wlasl["WLASL fine-tuning"]
end

node_user(("Signer"))
node_webcam(("Webcam"))
node_datasets["Sign datasets"]

node_user -->|"signs to"| node_browser
node_webcam -->|"captures frames"| node_browser
node_browser -->|"streams frames"| node_server
node_server -->|"extracts landmarks"| node_landmarks
node_server -->|"processes mode"| node_mode
node_server -->|"predicts letters"| node_fsmodel
node_server -->|"predicts words"| node_wordmodel
node_server -->|"returns predictions"| node_browser
node_browser -->|"controls mode"| node_mode
node_fsdata -.->|"provides landmarks"| node_fs_train
node_datasets -.->|"supplies images"| node_fsdata
node_datasets -->|"supplies videos"| node_worddata
node_worddata -->|"provides sequences"| node_word_train
node_fs_train -->|"saves model"| node_artifacts
node_word_train -->|"saves model"| node_artifacts
node_artifacts -->|"loads weights"| node_server
node_worddata -.->|"supports experiments"| node_wlasl
node_wlasl -.->|"fine-tunes model"| node_wordmodel

click node_browser "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py"
click node_server "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py"
click node_landmarks "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py"
click node_mode "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py"
click node_fsmodel "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py"
click node_wordmodel "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py"
click node_fsdata "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Preprocess%20asl%20alphabet.py"
click node_worddata "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Preprocess%20ASL%20Citizen.py"
click node_fs_train "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/train%20fingerspelling.py"
click node_word_train "https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/training%20asl%20citizen.py"
click node_wlasl "https://github.com/butcher05/american-sign-language-interpretor/tree/main/Code"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_server,node_landmarks,node_mode toneBlue
class node_fsmodel,node_wordmodel,node_artifacts toneAmber
class node_fsdata,node_worddata toneMint
class node_fs_train,node_word_train,node_wlasl toneRose
class node_user,node_webcam,node_datasets toneIndigo
```

## Components

### Live interpretation

- [Browser dashboard](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py): `Code/Inference pipeline.py`
- [WebSocket server](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py): `Code/Inference pipeline.py`
- [Landmark extraction](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py): `Code/Inference pipeline.py`
- [Mode and sentence buffer](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py): `Code/Inference pipeline.py`

### Recognition models

- [Fingerspelling MLP](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py): `Code/Inference pipeline.py`
- [Pose Transformer](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Inference%20pipeline.py): `Code/Inference pipeline.py`
- **Model weights**

### Data preparation

- [Alphabet landmarks](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Preprocess%20asl%20alphabet.py): `Code/Preprocess asl alphabet.py`
- [ASL Citizen sequences](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/Preprocess%20ASL%20Citizen.py): `Code/Preprocess ASL Citizen.py`

### Model training

- [MLP training](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/train%20fingerspelling.py): `Code/train fingerspelling.py`
- [Transformer pretraining](https://github.com/butcher05/american-sign-language-interpretor/blob/main/Code/training%20asl%20citizen.py): `Code/training asl citizen.py`
- [WLASL fine-tuning](https://github.com/butcher05/american-sign-language-interpretor/tree/main/Code): `Code`

### Other components

- **Signer**
- **Webcam**
- **Sign datasets**

## Connections

- Signer → Browser dashboard: signs to (evidence: `README.md`)
- Webcam → Browser dashboard: captures frames (evidence: `README.md`)
- Browser dashboard → WebSocket server: streams frames (evidence: `README.md`)
- WebSocket server → Landmark extraction: extracts landmarks (evidence: `Code/Inference pipeline.py`)
- WebSocket server → Mode and sentence buffer: processes mode (evidence: `Code/Inference pipeline.py`)
- WebSocket server → Fingerspelling MLP: predicts letters (evidence: `Code/Inference pipeline.py`)
- WebSocket server → Pose Transformer: predicts words (evidence: `Code/Inference pipeline.py`)
- WebSocket server → Browser dashboard: returns predictions (evidence: `README.md`)
- Browser dashboard → Mode and sentence buffer: controls mode (evidence: `Code/Inference pipeline.py`)
- Alphabet landmarks → MLP training: provides landmarks
- Sign datasets → Alphabet landmarks: supplies images
- Sign datasets → ASL Citizen sequences: supplies videos (evidence: `Code/Preprocess ASL Citizen.py`)
- ASL Citizen sequences → Transformer pretraining: provides sequences (evidence: `Code/training asl citizen.py`)
- MLP training → Model weights: saves model (evidence: `Code/train fingerspelling.py`)
- Transformer pretraining → Model weights: saves model (evidence: `Code/training asl citizen.py`)
- Model weights → WebSocket server: loads weights (evidence: `Code/Inference pipeline.py`)
- ASL Citizen sequences → WLASL fine-tuning: supports experiments
- WLASL fine-tuning → Pose Transformer: fine-tunes model