How letho1608/vsl_detect works
Main Components
• Data Collection Component:
• Video Downloader & Text Extraction: Represent this block by combining the functionality of the video downloader (handled by Combine.py) and text extraction (which produces Label.csv) from sign language videos.
• Data Processor:
• This component involves the conversion of raw video data into training datasets. Use nodes for CreateData.py (for landmark extraction via MediaPipe) and CheckData.py (for ensuring data integrity).
• Model Training:
• Model trainer (Training.py) which applies an LSTM-based deep learning architecture. Include checkpoints (Models/checkpoints) and the final trained model (Models/final_model.keras).
• Main Application (Real-time Recognition):
• This part, implemented in Main.py, handles real-time sign language recognition using the trained model, PyQt5 for GUI, and additional functionalities like Vietnamese text-to-speech.
Relationships and Data Flow
• The output from the Data Processor serves as input for the “Model Training” component. Indicate that checkpoint models are stored during training.
• Once training is complete, the final model is loaded by the “Main Application” for real-time recognition.
• Data flow from raw video/text to the processed dataset
• Hand-off from data processing to training
• Deployment of the trained model in the recognition module