How emnikhil/sign-language-to-text-conversion works
Main Components
• Data Acquisition & Preparation:
– Folder Creation (FoldersCreation.py) and dataset organization (dataSet/trainingData and dataSet/testingData directories, each with subdirectories for “0” and A–Z).
– TrainingDataCollection.py and TestingDataCollection.py for capturing and saving images.
• Image Preprocessing Module:
– Code in the README shows image filtering, grayscale conversion, gaussian blur, thresholding, and ROI extraction.
• Model Training & Storage:
– The Models folder contains multiple model files (.h5 and .json) along with Jupyter notebooks (Model.ipynb) used for training the CNN.
– This module includes the CNN architecture, detailed in the README (Convolutional Layers, Pooling, Fully Connected Layers, and Output Layer with SoftMax).
• Inference & Classification Module:
– Application.py comprises the live application where frames from a webcam are processed, predictions are made using the trained CNN, and ambiguous cases are handled via additional classifiers (specific sets for D, R, U; T, K, D, I; and S, M, N).
• GUI Layer:
– The system includes a graphical user interface (likely created using tkinter and PIL) that displays the real-time gesture-to-text conversion, as detailed under “Creating a GUI” in the README.
• External Dependencies & Libraries:
– Listed libraries include TensorFlow, Keras, OpenCV, Pillow, among others.
Relationships & Data Flow
• The data starts with collection/capturing (from webcam) and is stored in the organized dataset directories.
• The image preprocessing module reads raw images, applies filters, and outputs processed images.
• These processed images are then fed into the CNN for training/updating weights (offline training phase) or used directly for inference (online, real-time recognition).
• The classification process may branch: if ambiguity is detected, additional classifier modules (for problematic symbol groups) are invoked.
• The results (predicted letters) are then aggregated to form text/sentences, which are displayed in the GUI.
• The file structure (Folders for models, datasets, images, PDFs, and presentations) supports documentation, training, testing, and presentation aspects.
Architectural Patterns & Design Principles
• Modular Design: Each functionality is separated into distinct modules (data collection, preprocessing, training, inference, and GUI) which helps with testing and maintenance.
• Pipeline/Data Flow Architecture: There is a clear linear flow—from image capture and processing, through classification, to final text output.
• Multi-Layered Classification: The design incorporates layered classification to improve accuracy on ambiguous classes, as noted in the README.
• Use of Pre-trained Models & Serialization: The Models directory indicates that models are likely pre-trained and stored in serialized formats (.h5 and .json), which are loaded during inference.