How surbhit20/slurred-speech-recognition-deeplearning works
Project type and purpose
• This is a machine-learning research pipeline for automatic speech recognition (ASR), specifically fine-tuning a base end-to-end ASR model to handle slurred or impaired speech via transfer learning.
• Key indicators: Jupyter notebooks under “Notebooks Code,” a large “Dataset” tree of .flac files, a “Best-Fit-TrainedModel” folder of .h5 weights, and README’s focus on training/fine-tuning workflows.
File structure
• Top-level directories:
• Best-Fit-TrainedModel – final and intermediate .h5 model checkpoints
• Dataset – organized audio corpora (normal vs. slurred speech), each with .flac files, transcripts, test scripts
• Notebooks Code – Jupyter notebooks for exploratory data analysis, base training, transfer-learning experiments
• Papers – reference ASR architecture papers (DeepSpeech, DeepSpeech2)
• Project_Update_Reports – progress/presentation PDFs
• README.md, btp_report.pdf, prediction.txt – documentation and outputs
• No web-app code in repo; data-collection app references are external (Heroku, GitHub Pages).
README for additional insights
• High-level solution: train a state-of-the-art end-to-end ASR (base model) on standard speech, then transfer-learn on impaired speech.
• Model architecture graphic (RNN/CTC pipeline) indicates: audio input → feature extraction (MFCC) → convolutional layers → recurrent layers → fully connected + CTC loss → text output.
• Base performance metrics (loss, CER, WER) and dataset description (Librispeech, custom slurred dataset).
Core components
a. Main components
1. Data Sources
• Librispeech (pre-collected hours of typical speech)
• Slurred-Speech Dataset (small, organized by speaker/transcript)
• External Data-Collection Web App (Django + Flask/Heroku)
2. Data Preprocessing
• Audio ingestion (.flac reading)
• Feature extraction (MFCC or spectrogram computation)
• Transcript alignment and dataset curation
3. Base Model Training
• End-to-end ASR training pipeline using TensorFlow/Keras (Conv+RNN+CTC)
4. Transfer Learning / Fine-Tuning
• Load base model weights (.h5)
• Freeze lower layers, retrain upper layers on impaired speech
5. Model Evaluation
• Compute WER/CER metrics
• Generate prediction.txt for test audio
6. Model Storage & Versioning
• Best-Fit-TrainedModel directory holding checkpoints and final h5 files
7. Inference / Deployment
• Jupyter or script-based inference calling model.predict on new audio
b. Relationships & interactions
• Data flows from Data Sources → Preprocessing → Base Training → Fine-Tuning → Evaluation → Stored Models → Inference
• External app feeds new slurred audio back into the Dataset for iterative retraining
c. Architectural patterns / design principles
• Pipeline pattern: clear separation between data ingestion, training, evaluation, and deployment
• Modular: preprocessing, model definition, training, and fine-tuning are distinct modules (often in separate notebooks)
• Transfer Learning: reuse base-model weights, then fine-tune on smaller custom data
d. Key technologies & frameworks
• TensorFlow / Keras (model definition, CTC loss)
• Python audio libraries (librosa or similar for MFCC)
• Jupyter notebooks for experimentation
• Flask/Django + Heroku for data-collection frontend (external)