butcher05/american-sign-language-interpretor

How butcher05/american-sign-language-interpretor works

This is a real-time ASL interpreter with two recognition modes: fingerspelling letters and word-level signs. Its principal runtime flow sends browser webcam frames to a FastAPI WebSocket backend, which extracts landmarks, runs the selected model, and returns predictions for the dashboard and sentence buffer. Separate preprocessing and training scripts prepare and fit the two models. The README documents WLASL fine-tuning, but no corresponding script appears in the supplied tree; that capability is shown at the enclosing Code directory. Some runtime details are omitted in the excerpts, so only documented or excerpt-supported connections are drawn.

Main entry points: Inference pipeline.py WebSocket server · Preprocess ASL Citizen.py ASL Citizen sequences · Preprocess asl alphabet.py Alphabet landmarks · train fingerspelling.py MLP training

Components 15

Live interpretation

Browser dashboard Code/Inference pipeline.py
WebSocket server Code/Inference pipeline.py
Landmark extraction Code/Inference pipeline.py
Mode and sentence buffer Code/Inference pipeline.py

Recognition models

Fingerspelling MLP Code/Inference pipeline.py
Pose Transformer Code/Inference pipeline.py
Model weights

Data preparation

Alphabet landmarks Code/Preprocess asl alphabet.py
ASL Citizen sequences Code/Preprocess ASL Citizen.py

Model training

MLP training Code/train fingerspelling.py
Transformer pretraining Code/training asl citizen.py
WLASL fine-tuning Code

Other components

Signer
Webcam
Sign datasets

Connections 18

Signer → Browser dashboard: signs to README.md
Webcam → Browser dashboard: captures frames README.md
Browser dashboard → WebSocket server: streams frames README.md
WebSocket server → Landmark extraction: extracts landmarks Code/Inference pipeline.py
WebSocket server → Mode and sentence buffer: processes mode Code/Inference pipeline.py
WebSocket server → Fingerspelling MLP: predicts letters Code/Inference pipeline.py
WebSocket server → Pose Transformer: predicts words Code/Inference pipeline.py
WebSocket server → Browser dashboard: returns predictions README.md
Browser dashboard → Mode and sentence buffer: controls mode Code/Inference pipeline.py
Alphabet landmarks → MLP training: provides landmarks
Sign datasets → Alphabet landmarks: supplies images
Sign datasets → ASL Citizen sequences: supplies videos Code/Preprocess ASL Citizen.py
ASL Citizen sequences → Transformer pretraining: provides sequences Code/training asl citizen.py
MLP training → Model weights: saves model Code/train fingerspelling.py
Transformer pretraining → Model weights: saves model Code/training asl citizen.py
Model weights → WebSocket server: loads weights Code/Inference pipeline.py
ASL Citizen sequences → WLASL fine-tuning: supports experiments
WLASL fine-tuning → Pose Transformer: fine-tunes model

More Python diagrams: marcus-alicia/iredadmin-pro-sql · rawr-ai/mcp-graphiti · scaleml/agentspex · scottduncanai/industrial-ai-troubleshooting-agent · sueszli/notionsnapshot

Drawn by GitDiagram on September 30, 2026 from the repository's file tree, README and source files. Read as Markdown ·