How usf132/signify works
Signify translates live-camera or uploaded-video hand gestures into text, then can synthesize that text as speech. The Streamlit interface delegates frame recognition to backend logic, which detects a hand, extracts normalized landmarks, classifies them, and applies confidence filtering; the separate SignPipeline composes stabilization and sentence editing for a fuller inference workflow. The README documents both input modes and text-to-speech. App excerpts are partial, so some UI-to-backend wiring is supported by imports and documentation rather than a complete call-site excerpt.
Main entry points: pipeline.py Sign Pipeline · backend.py Frame Backend · main.py Streamlit App · yolo_detector.py YOLO Detector
Components 14
User Interface
- Streamlit App
app/main.py - Frame Backend
app/backend.py
Gesture Recognition
- Sign Pipeline
signlens/pipeline.py - YOLO Detector
signlens/yolo_detector.py - Landmark Features
signlens/landmark.py - Sign Classifier
signlens/recognizer.py
Text Composition
- Prediction Stabilizer
signlens/stabilizer.py - Sentence Builder
signlens/sentence.py - Composed Text
Speech Output
- Speech Conversion
app/backend.py - ElevenLabs
- Generated Audio
Other components
- User
- Camera or Video
Connections 16
- User → Streamlit App: uses
README.md - Camera or Video → Streamlit App: provides frames
README.md - Streamlit App → Frame Backend: calls
app/main.py - Frame Backend → YOLO Detector: detects hand
app/backend.py - Frame Backend → Landmark Features: extracts features
app/backend.py - Frame Backend → Sign Classifier: classifies features
app/backend.py - Sign Pipeline → YOLO Detector: detects hand
signlens/pipeline.py - Sign Pipeline → Landmark Features: extracts features
signlens/pipeline.py - Sign Pipeline → Sign Classifier: classifies features
signlens/pipeline.py - Sign Pipeline → Prediction Stabilizer: stabilizes predictions
signlens/pipeline.py - Sign Pipeline → Sentence Builder: edits sentence
signlens/pipeline.py - Frame Backend → Composed Text: returns prediction
app/backend.py - Streamlit App → Speech Conversion: requests speech
app/main.py - Speech Conversion → ElevenLabs: converts text
app/backend.py - ElevenLabs → Generated Audio: returns audio
- Streamlit App → Generated Audio: plays audio
app/main.py