How paresh-0007/signlanguagedetectionusinglstm works
Project Type and Purpose
• This is a Python‐based machine-learning application for real-time American Sign Language (ASL) recognition.
• It consists of two main pipelines:
• Training pipeline – collects hand-gesture data, preprocesses, trains an LSTM model, generates artifacts (model.json, model.h5) and logging data.
• Inference (serving) pipeline – captures live video, detects hand landmarks using MediaPipe, feeds preprocessed feature sequences into the trained LSTM, and displays recognized letters.
File Structure Analysis
• Top-level: LICENSE, README.md, requirement.txt, plus the code folder SignLanguageDetectionUsingLstm/.
• SignLanguageDetectionUsingLstm/:
• Data collection & organization: makefolders.py, collectdata.py
• Model training: trainmodel.py, model.json (architecture), model.h5 (weights)
• Inference/app: app.py
• Experiment artifacts: Logs/ (TensorBoard events), Confusion.ipynb, PPT_…pptx, Report_….pdf
Key Architecture Insights from README
• Real-time webcam capture (OpenCV) + Google MediaPipe hand-landmark extraction.
• Sequence-based classification using a TensorFlow-Keras LSTM.
• Output mapping from predicted label to alphabet letter.
• Logging via TensorBoard for monitoring training.
a. Main Components:
• User Webcam (video feed input)
• MediaPipe Hand Detector (landmark extractor)
• LSTM Inference Engine (loads model.json + model.h5)
• Visualization/UI (OpenCV window with predicted text overlay)
• File System (dataset folders, model artifacts, TensorBoard logs)
• Training Pipeline (collectdata.py → makefolders.py → trainmodel.py → produce artifacts & Logs)
b. Relationships & Data Flow:
– Training flow: Raw webcam/images → makefolders.py → collectdata.py → dataset folders → preprocessing → trainmodel.py → model.json + model.h5 + Logs → stored in file system.
– Inference flow: Webcam → MediaPipe → preprocessing → LSTM model → prediction → UI.
c. Architectural Patterns & Principles:
– Pipeline pattern: separation of concerns between data collection, preprocessing, model training, and inference.
– Clear modularization of utility functions vs. orchestration scripts.
– Use of external library services (MediaPipe, TensorFlow).
d. Technologies & Frameworks:
– Python 3.9, OpenCV for capture & display
– MediaPipe for hand-landmark detection
– TensorFlow/Keras LSTM for sequence classification
– TensorBoard for training logs & visualization