How rishabhshah13/real-time-slr works
Main Components
A. Input Layer:
• Webcam Interface
• Video Input Processing (MediaPipe)
• Landmark Extraction System
B. Processing Layer:
• Two Main Recognition Modules:
1. Fingerspelling Recognition System
• Letters Model
• Numbers Model
2. Gloss Recognition System
C. Application Interfaces:
• OpenCV Local Interface
• Streamlit Web Interface
D. Model Management:
• Model Storage
• Configuration Management
[Input Sources] → [Preprocessing] → [Recognition Systems] → [User Interfaces]
[Webcam/Video Input] → [MediaPipe Processing] → [Landmark Extraction]
• ↓
• [Feature Processing]
• ┌─ [Fingerspelling Models] ─┐
[Processed Features]─┤ ├─→ [Prediction Output]
• [Gloss Models] ───────┘
Key Relationships to Highlight
• Data flow from video input through landmark extraction to feature processing
• Parallel processing paths for fingerspelling and gloss recognition
• Demonstrate how the configuration system interfaces with all components
• Two separate user interface paths (OpenCV and Streamlit)
Technical Details
• Use of MediaPipe for hand landmark extraction
• 1D CNN and Transformer architecture in the model components
• Configuration management system (config.yaml)
• Inference pipeline flow
Implementation-Specific Elements
• Scripts/inference directory components as the core processing unit
• Models directory as the model storage system
• Relationship between st_app.py and cv2main.py as alternative interfaces
• Utility functions layer that supports all components