# usf132/signify architecture

> Architecture diagram of the GitHub repository usf132/signify, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/usf132/signify
- Repository: https://github.com/usf132/signify

## Overview

Signify translates live-camera or uploaded-video hand gestures into text, then can synthesize that text as speech. The Streamlit interface delegates frame recognition to backend logic, which detects a hand, extracts normalized landmarks, classifies them, and applies confidence filtering; the separate SignPipeline composes stabilization and sentence editing for a fuller inference workflow. The README documents both input modes and text-to-speech. App excerpts are partial, so some UI-to-backend wiring is supported by imports and documentation rather than a complete call-site excerpt.

Main entry points: [`signlens/pipeline.py`](https://github.com/usf132/signify/blob/main/signlens/pipeline.py) (Sign Pipeline), [`app/backend.py`](https://github.com/usf132/signify/blob/main/app/backend.py) (Frame Backend), [`app/main.py`](https://github.com/usf132/signify/blob/main/app/main.py) (Streamlit App), [`signlens/yolo_detector.py`](https://github.com/usf132/signify/blob/main/signlens/yolo_detector.py) (YOLO Detector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User Interface"]
  node_streamlit["Streamlit App<br/>[main.py]"]
  node_backend["Frame Backend<br/>[backend.py]"]
end

subgraph group_recognition["Gesture Recognition"]
  node_pipeline["Sign Pipeline<br/>[pipeline.py]"]
  node_detector["YOLO Detector<br/>[yolo_detector.py]"]
  node_landmarks["Landmark Features<br/>[landmark.py]"]
  node_recognizer["Sign Classifier<br/>[recognizer.py]"]
end

subgraph group_composition["Text Composition"]
  node_stabilizer["Prediction Stabilizer<br/>[stabilizer.py]"]
  node_sentence["Sentence Builder<br/>[sentence.py]"]
  node_text["Composed Text"]
end

subgraph group_speech["Speech Output"]
  node_tts["Speech Conversion<br/>[backend.py]"]
  node_elevenlabs["ElevenLabs"]
  node_audio["Generated Audio"]
end

node_user(("User"))
node_video_input(("Camera or Video"))

node_user -->|"uses"| node_streamlit
node_video_input -->|"provides frames"| node_streamlit
node_streamlit -->|"calls"| node_backend
node_backend -->|"detects hand"| node_detector
node_backend -->|"extracts features"| node_landmarks
node_backend -->|"classifies features"| node_recognizer
node_pipeline -->|"detects hand"| node_detector
node_pipeline -->|"extracts features"| node_landmarks
node_pipeline -->|"classifies features"| node_recognizer
node_pipeline -->|"stabilizes predictions"| node_stabilizer
node_pipeline -->|"edits sentence"| node_sentence
node_backend -->|"returns prediction"| node_text
node_streamlit -->|"requests speech"| node_tts
node_tts -->|"converts text"| node_elevenlabs
node_elevenlabs -->|"returns audio"| node_audio
node_streamlit -->|"plays audio"| node_audio

click node_streamlit "https://github.com/usf132/signify/blob/main/app/main.py"
click node_backend "https://github.com/usf132/signify/blob/main/app/backend.py"
click node_pipeline "https://github.com/usf132/signify/blob/main/signlens/pipeline.py"
click node_detector "https://github.com/usf132/signify/blob/main/signlens/yolo_detector.py"
click node_landmarks "https://github.com/usf132/signify/blob/main/signlens/landmark.py"
click node_recognizer "https://github.com/usf132/signify/blob/main/signlens/recognizer.py"
click node_stabilizer "https://github.com/usf132/signify/blob/main/signlens/stabilizer.py"
click node_sentence "https://github.com/usf132/signify/blob/main/signlens/sentence.py"
click node_tts "https://github.com/usf132/signify/blob/main/app/backend.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_streamlit,node_backend,node_user toneBlue
class node_pipeline,node_detector,node_landmarks,node_recognizer toneAmber
class node_stabilizer,node_sentence,node_text toneMint
class node_tts,node_elevenlabs,node_audio toneRose
class node_video_input toneIndigo
```

## Components

### User Interface

- [Streamlit App](https://github.com/usf132/signify/blob/main/app/main.py): `app/main.py`
- [Frame Backend](https://github.com/usf132/signify/blob/main/app/backend.py): `app/backend.py`

### Gesture Recognition

- [Sign Pipeline](https://github.com/usf132/signify/blob/main/signlens/pipeline.py): `signlens/pipeline.py`
- [YOLO Detector](https://github.com/usf132/signify/blob/main/signlens/yolo_detector.py): `signlens/yolo_detector.py`
- [Landmark Features](https://github.com/usf132/signify/blob/main/signlens/landmark.py): `signlens/landmark.py`
- [Sign Classifier](https://github.com/usf132/signify/blob/main/signlens/recognizer.py): `signlens/recognizer.py`

### Text Composition

- [Prediction Stabilizer](https://github.com/usf132/signify/blob/main/signlens/stabilizer.py): `signlens/stabilizer.py`
- [Sentence Builder](https://github.com/usf132/signify/blob/main/signlens/sentence.py): `signlens/sentence.py`
- **Composed Text**

### Speech Output

- [Speech Conversion](https://github.com/usf132/signify/blob/main/app/backend.py): `app/backend.py`
- **ElevenLabs**
- **Generated Audio**

### Other components

- **User**
- **Camera or Video**

## Connections

- User → Streamlit App: uses (evidence: `README.md`)
- Camera or Video → Streamlit App: provides frames (evidence: `README.md`)
- Streamlit App → Frame Backend: calls (evidence: `app/main.py`)
- Frame Backend → YOLO Detector: detects hand (evidence: `app/backend.py`)
- Frame Backend → Landmark Features: extracts features (evidence: `app/backend.py`)
- Frame Backend → Sign Classifier: classifies features (evidence: `app/backend.py`)
- Sign Pipeline → YOLO Detector: detects hand (evidence: `signlens/pipeline.py`)
- Sign Pipeline → Landmark Features: extracts features (evidence: `signlens/pipeline.py`)
- Sign Pipeline → Sign Classifier: classifies features (evidence: `signlens/pipeline.py`)
- Sign Pipeline → Prediction Stabilizer: stabilizes predictions (evidence: `signlens/pipeline.py`)
- Sign Pipeline → Sentence Builder: edits sentence (evidence: `signlens/pipeline.py`)
- Frame Backend → Composed Text: returns prediction (evidence: `app/backend.py`)
- Streamlit App → Speech Conversion: requests speech (evidence: `app/main.py`)
- Speech Conversion → ElevenLabs: converts text (evidence: `app/backend.py`)
- ElevenLabs → Generated Audio: returns audio
- Streamlit App → Generated Audio: plays audio (evidence: `app/main.py`)