How gabguerin/sign-language-recognition--mediapipe-dtw works
Project type and purpose
• This is a stand-alone computer-vision/ML pipeline in Python for real-time Sign Language Recognition.
• It uses MediaPipe for landmark extraction and Dynamic Time Warping (DTW) for time-series similarity.
File structure
• data/
– videos/ (raw sign videos, organized by label)
– dataset/ (preprocessed or placeholder for processed data)
• main.py (orchestrator for loading model, starting webcam, handling user input)
• sign_recorder.py (captures, processes, and predicts signs via DTW + voting)
• yt_download.py + yt_links.csv (automated dataset creation via YouTube + ffmpeg)
• models/
– hand_model.py (encapsulates MediaPipe hand landmarks → feature vectors)
– pose_model.py, sign_model.py (structure for pose or sign sequences)
• utils/ (helper functions)
– mediapipe_utils.py (init & run MediaPipe Holistic)
– landmark_utils.py (normalize/rotate/scale landmarks)
– dataset_utils.py (file I/O, preprocessing pipelines)
– dtw.py (DTW distance computation)
• webcam_manager.py (abstraction over OpenCV webcam feed)
• requirements.txt (dependencies: MediaPipe, OpenCV, NumPy, fastdtw, etc.)
README for additional insights
• Workflow: ingest videos → extract landmarks → compute angles (invariant to scale/orientation) → build per-sign models → at runtime record gesture → compute DTW against all reference signs → voting & thresholding → output label.
Elements
• Clear labels for each block (e.g., “MediaPipe Utils,” “HandModel Feature Extraction,” “DTW Engine”).
• Blue for offline/batch components
• Green for real-time/inference components
• Orange for external services
• Gray for data stores (video folders, model files)
• Rectangles for code modules
• Cloud/music-note for YouTube/ffmpeg external service
Final advice
• Be very detailed: break “Preprocessing” into sub-blocks for dataset_utils, mediapipe_utils, landmark_utils.
• Separate the project into as many independent components as possible to highlight modularity.
• Bidirectional control (e.g., main.py calls webcam_manager & sign_recorder, sign_recorder returns prediction).