# surbhit20/slurred-speech-recognition-deeplearning architecture

> Architecture diagram of the GitHub repository surbhit20/slurred-speech-recognition-deeplearning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/surbhit20/slurred-speech-recognition-deeplearning
- Repository: https://github.com/surbhit20/slurred-speech-recognition-deeplearning

## Overview

##### Project type and purpose
- This is a machine-learning research pipeline for automatic speech recognition (ASR), specifically fine-tuning a base end-to-end ASR model to handle slurred or impaired speech via transfer learning.
- Key indicators: Jupyter notebooks under “Notebooks Code,” a large “Dataset” tree of .flac files, a “Best-Fit-TrainedModel” folder of .h5 weights, and README’s focus on training/fine-tuning workflows.

##### File structure
- Top-level directories:
- Best-Fit-TrainedModel – final and intermediate .h5 model checkpoints
- Dataset – organized audio corpora (normal vs. slurred speech), each with .flac files, transcripts, test scripts
- Notebooks Code – Jupyter notebooks for exploratory data analysis, base training, transfer-learning experiments
- Papers – reference ASR architecture papers (DeepSpeech, DeepSpeech2)
- Project_Update_Reports – progress/presentation PDFs
- README.md, btp_report.pdf, prediction.txt – documentation and outputs
- No web-app code in repo; data-collection app references are external (Heroku, GitHub Pages).

##### README for additional insights
- High-level solution: train a state-of-the-art end-to-end ASR (base model) on standard speech, then transfer-learn on impaired speech.
- Model architecture graphic (RNN/CTC pipeline) indicates: audio input → feature extraction (MFCC) → convolutional layers → recurrent layers → fully connected + CTC loss → text output.
- Base performance metrics (loss, CER, WER) and dataset description (Librispeech, custom slurred dataset).

##### Core components
a. Main components
1. Data Sources
- Librispeech (pre-collected hours of typical speech)
- Slurred-Speech Dataset (small, organized by speaker/transcript)
- External Data-Collection Web App (Django + Flask/Heroku)
2. Data Preprocessing
- Audio ingestion (.flac reading)
- Feature extraction (MFCC or spectrogram computation)
- Transcript alignment and dataset curation
3. Base Model Training
- End-to-end ASR training pipeline using TensorFlow/Keras (Conv+RNN+CTC)
4. Transfer Learning / Fine-Tuning
- Load base model weights (.h5)
- Freeze lower layers, retrain upper layers on impaired speech
5. Model Evaluation
- Compute WER/CER metrics
- Generate prediction.txt for test audio
6. Model Storage & Versioning
- Best-Fit-TrainedModel directory holding checkpoints and final h5 files
7. Inference / Deployment
- Jupyter or script-based inference calling model.predict on new audio
b. Relationships & interactions
- Data flows from Data Sources → Preprocessing → Base Training → Fine-Tuning → Evaluation → Stored Models → Inference
- External app feeds new slurred audio back into the Dataset for iterative retraining
c. Architectural patterns / design principles
- Pipeline pattern: clear separation between data ingestion, training, evaluation, and deployment
- Modular: preprocessing, model definition, training, and fine-tuning are distinct modules (often in separate notebooks)
- Transfer Learning: reuse base-model weights, then fine-tune on smaller custom data
d. Key technologies & frameworks
- TensorFlow / Keras (model definition, CTC loss)
- Python audio libraries (librosa or similar for MFCC)
- Jupyter notebooks for experimentation
- Flask/Django + Heroku for data-collection frontend (external)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Services
    DCApp["Data-Collection App<br/>(Django/Heroku)"]:::external

    %% Data Sources
    subgraph "Data Sources"
        Librispeech["Standard Speech Dataset<br/>(English)"]:::data
        SlurredTrain["Slurred Speech Training Dataset"]:::data
        SlurredTest["Slurred Speech Test Dataset"]:::data
    end

    %% Preprocessing
    Preprocess["Audio Preprocessing & Feature Extraction"]:::compute

    %% Model Training
    BaseTrain["Base ASR Model Training"]:::compute
    Transfer["Hindi Impaired-Speech Fine-Tuning"]:::compute

    %% Evaluation & Storage
    Eval["Model Evaluation<br/>(WER/CER)"]:::compute
    Prediction["Model Evaluation Output<br/>(prediction.txt)"]:::storage
    ModelStore["Model Storage & Versioning<br/>(.h5 checkpoints)"]:::storage

    %% Inference
    Inference["Inference & Deployment"]:::compute

    %% Notebooks Code
    subgraph "Notebooks Code"
        NB1["Audio Preprocessing Notebook<br/>BTP Part1.ipynb"]:::notebook
        NB2["Base ASR Training Notebook<br/>BTP Part2.ipynb"]:::notebook
        NB3["English ASR Training Notebook<br/>BTP_English.ipynb"]:::notebook
        NB4["Hindi Fine-Tuning Notebook<br/>BTP__Hindi.ipynb"]:::notebook
        NB5["Complete Pipeline & Eval Notebook<br/>BTP_final.ipynb"]:::notebook
        NB6["Auxiliary Images"]:::notebook
    end

    %% Dataset Validation/Test Scripts
    subgraph "Dataset Validation/Test Scripts"
        DS1["Train (Hindi) Test Script"]:::data
        DS2["Test (Hindi) Test Script"]:::data
        DS3["Standard Speech Test Script"]:::data
    end

    %% Papers & Reports
    subgraph "Reference Papers"
        P1["DeepSpeech.pdf"]:::external
        P2["DeepSpeech2.pdf"]:::external
    end
    subgraph "Project Reports"
        R1["Project_Update_Reports/"]:::storage
        R2["btp_report.pdf"]:::storage
    end

    %% Connections
    DCApp -->|feeds new audio| SlurredTrain
    Librispeech -->|.flac| Preprocess
    SlurredTrain -->|.flac| Preprocess
    SlurredTest -->|.flac| Preprocess

    Preprocess -->|MFCC tensors| BaseTrain
    BaseTrain -->|base model weights| ModelStore
    ModelStore -->|load weights| Transfer
    Transfer -->|fine-tuned weights| ModelStore
    Transfer -->|predictions| Eval
    Eval -->|metrics| Prediction
    ModelStore -->|deployed model| Inference

    %% Notebook Arrows
    Preprocess --> NB1
    BaseTrain --> NB2
    BaseTrain --> NB3
    Transfer --> NB4
    Eval --> NB5
    NB6 -.-> NB1
    NB6 -.-> NB2
    NB6 -.-> NB3
    NB6 -.-> NB4
    NB6 -.-> NB5

    %% Validation/Test Scripts Links
    SlurredTrain --> DS1
    SlurredTest --> DS2
    Librispeech --> DS3

    %% Papers & Reports Links
    BaseTrain --> P1
    BaseTrain --> P2
    ModelStore --> R1
    ModelStore --> R2

    %% Click Events
    click SlurredTrain "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/tree/main/Dataset/SlurredSpeech-Dataset-Train(Hindi)"
    click SlurredTest "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/tree/main/Dataset/SlurredSpeech-Dataset-Test(Hindi)"
    click Librispeech "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/tree/main/Dataset/SlurredSpeech-Dataset-Train"
    click NB1 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Notebooks Code/BTP Part1.ipynb"
    click NB2 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Notebooks Code/BTP Part2.ipynb"
    click NB3 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Notebooks Code/BTP_English.ipynb"
    click NB4 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Notebooks Code/BTP__Hindi.ipynb"
    click NB5 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Notebooks Code/BTP_final.ipynb"
    click NB6 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/tree/main/Notebooks Code/Images/"
    click P1 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Papers/DeepSpeech.pdf"
    click P2 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Papers/DeepSpeech2.pdf"
    click ModelStore "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Best-Fit-TrainedModel/*.h5"
    click Prediction "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/prediction.txt"
    click DS1 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Dataset/SlurredSpeech-Dataset-Train(Hindi)/1/2/test.py"
    click DS2 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Dataset/SlurredSpeech-Dataset-Test(Hindi)/1/2/test.py"
    click DS3 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/Dataset/SlurredSpeech-Dataset-Train/1/4/test.py"
    click R1 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/tree/main/Project_Update_Reports/"
    click R2 "https://github.com/surbhit20/slurred-speech-recognition-deeplearning/blob/main/btp_report.pdf"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef compute fill:#90EE90,stroke:#000;
    classDef storage fill:#FFA500,stroke:#000;
    classDef external fill:#DDA0DD,stroke:#000;
    classDef notebook fill:#FFB6C1,stroke:#000;
```
