How arshad221b/sign-language-recognition works
Project type and purpose
• This is a machine-learning based image-processing tool for sign-language recognition. It combines OpenCV for real-time ROI (region of interest) extraction with deep-learning (CNN) models (Keras/TensorFlow or PyTorch) to classify hand‐sign images.
File structure
• Top-level scripts & notebooks:
• ROIinOpenCV.py – real-time video capture, hand segmentation, live inference
• cnn.py, HandHisto.py, ISL.py, opencv.py – supporting scripts for model training, histogram features, image preprocessing
• ASL*.ipynb & sign_language_pytorch.ipynb – exploratory notebooks (Keras/TensorFlow vs PyTorch workflows)
• Model artifacts: CNNmodel.h5, model.h5, model_weights.h5
• Config & metadata: requirements.txt, _config.yml, LICENSE, README.md
• Paper PDF: islpaper.pdf
README for additional insights
• Dependencies: TensorFlow, Keras, OpenCV (and PyTorch for the alternate notebook)
• Dataset: Sign-Language-MNIST (downloaded externally)
• Primary “run” entry point: ROIinOpenCV.py
Main components & relationships
a. Data Ingestion
• External source: Kaggle dataset (Sign-Language-MNIST)
• Storage: local filesystem (raw .csv or images)
b. Preprocessing Module
• Scripts: HandHisto.py / opencv.py
• Tasks: ROI extraction, resizing, normalization, histogram equalization
c. Model Training
• Frameworks: Keras/TensorFlow (cnn.py) or PyTorch (sign_language_pytorch.ipynb)
• Output: serialized model files (.h5, model_weights.h5)
d. Model Artifacts
• Stored on disk: CNNmodel.h5 / model.h5 + weights
e. Inference Module
• Script: ROIinOpenCV.py (uses live webcam feed, applies preprocessing, loads model, outputs predicted sign)
f. Notebooks (ASL*.ipynb)
• Experimental workflows: prototyping architectures, hyperparameter tuning
g. External Interfaces
• Webcam or video stream (input)
• Console/UI (text overlay or print prediction)
Data flow: Dataset → Preprocessing → Training → Model Artifacts → Inference → User/UI
Architectural patterns & principles
• Pipeline architecture: distinct stages for data prep, training, inference
• Separation of concerns:
• Preprocessing code decoupled from model definitions
• Model training separate from inference logic
• Extensibility: ability to swap out Keras/TensorFlow vs PyTorch backends
• Modularity: standalone scripts/notebooks for each major task