# rupeshkeesaram/applied-ml-speech-to-text architecture

> Architecture diagram of the GitHub repository rupeshkeesaram/applied-ml-speech-to-text, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rupeshkeesaram/applied-ml-speech-to-text
- Repository: https://github.com/rupeshkeesaram/applied-ml-speech-to-text

## Overview

##### Main Components
- Frontend (User Interface):
- The Streamlit app inside the "app" folder, with a dedicated pages subdirectory (e.g., 1_Audio_File.py and 2_Youtube_Video.py) to handle multiple UI pages.
- Backend Processing:
- Data management:
▪ Directory "data" contains raw audio, interim data, processed outputs, and user-uploaded files.
- Core source code under "src":
▪ "src/data_processing": Preprocessing of audio input.
▪ "src/feature_extraction": Module for extracting features from audio.
▪ "src/model": Functions for loading and fine-tuning the ASR model (also includes utilities for transcript creation and practice model loading).
▪ "src/evaluation": Scripts to assess the performance of the ASR model.
▪ "src/utils": Utility functions including extracting audio from YouTube and loading audio data.
- Configuration management:
▪ "configs", which likely holds environment configurations or parameter settings (even though the README refers to "config" as centralized settings).
- Model Management:
- The "models" folder hosts the base model archives and extracted model files.
- Experimentation and Documentation:
- "notebooks" directory for colloaborative work and model fine-tuning experiments.
- "docs" folder includes extensive documentation (Project Description, Resources, Comparison documents).

##### Key Processes and Data Flow
- User Interaction:
- The user accesses the Streamlit app through a web browser.
- The app sends audio files or YouTube links (from the app/pages scripts) to the backend processing routines.
- Data Processing Pipeline:
- Raw audio files are ingested from "data/raw" or "data/uploaded" and then processed through "src/data_processing" routines.
- Feature engineering is performed in "src/feature_extraction".
- Model Interaction:
- The processed audio features are passed into the ASR model (loaded and possibly fine-tuned using code in "src/model").
- Evaluation of model outputs is done via scripts located in "src/evaluation".
- Logging and Monitoring:
- Global logs in "logs/global_log.log" capture important runtime events.
- Docker Containerization:
- The Dockerfile and associated configuration (like .dockerignore) encapsulate the entire project, ensuring that the environment and dependencies (defined in packages.txt and requirements.txt) are consistent during deployment.

##### Architectural Patterns and Design Principles
- Layered Architecture:
- Clearly separate the presentation layer (Streamlit app), processing layers (data, feature extraction, model inference), and supporting modules (utils, configuration, logging).
- Modularity:
- The project is modularized by domain (e.g., data processing vs. model operations), which enhances maintainability and scalability.
- Containerization:
- Deployment via Docker emphasizes environment consistency and ease of deployment.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    U("User (Web Browser)"):::frontend

    %% Frontend Layer
    subgraph "Frontend Layer"
        FS("Streamlit App"):::frontend
        FS1("Audio File Page"):::frontend
        FS2("Youtube Video Page"):::frontend
        FS --> FS1
        FS --> FS2
    end

    %% Data Management Layer
    subgraph "Data Management"
        DM("Data Folder"):::backend
        DM1("Raw"):::backend
        DM2("Interim"):::backend
        DM3("Processed"):::backend
        DM4("Uploaded"):::backend
        DM --> DM1
        DM --> DM2
        DM --> DM3
        DM --> DM4
    end

    %% Processing & Feature Extraction Layer
    subgraph "Processing & Feature Extraction"
        DP("Data Processing"):::backend
        FE("Feature Extraction"):::backend
        DP --> FE
    end

    %% Model Handling & Evaluation Layer
    subgraph "Model Handling & Evaluation"
        M1("Load Model"):::backend
        M2("Fine-tune Model"):::backend
        M3("Transcripts"):::backend
        M4("Practice Model"):::backend
        MF("Model Files"):::backend
        EV("Evaluation"):::backend
        M1 --> M2
        M2 --> M3
        M3 --> M4
        M1 --> MF
        M1 --> EV
    end

    %% Utilities and Configurations
    subgraph "Utilities & Configurations"
        UT1("Extract YouTube Audio"):::backend
        UT2("Load Audio"):::backend
        CF("Configurations"):::backend
    end

    %% Containerization & Deployment
    subgraph "Containerization & Deployment"
        DF("Dockerfile"):::infra
        DC(".dockerignore"):::infra
        PK("packages.txt"):::infra
        RT("requirements.txt"):::infra
    end

    %% Experimentation & Documentation
    subgraph "Experimentation & Documentation"
        NB("Notebooks"):::infra
        DOC("Documentation"):::infra
    end

    %% Logging
    LOG("Global Logging"):::infra

    %% Connections between layers
    U -->|"accesses"| FS
    FS -->|"sends_audio_input"| DP
    DM -->|"provides_raw_data"| DP
    DM -->|"supplies_input"| FE
    FE -->|"extracts_features_to"| M1
    EV -->|"returns_results_to"| FS

    %% Utilities / Configurations connected to processing and model handling
    UT1 ---|"supports"| DP
    UT1 ---|"assists"| M1
    UT2 ---|"supports"| DP
    UT2 ---|"assists"| M1
    CF ---|"configures"| DP
    CF ---|"configures"| M1

    %% Logging connected to key stages
    LOG --- FS
    LOG --- DP
    LOG --- M1

    %% Containerization supports overall system (dashed connections)
    DF --- FS
    DF --- DP
    DF --- M1
    DF --- UT1
    DF --- UT2

    %% Experimentation & Documentation link to model development
    NB --- M1
    DOC --- M1

    %% Click Events for Frontend
    click FS "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/app/"
    click FS1 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/app/pages/1_Audio_File.py"
    click FS2 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/app/pages/2_Youtube_Video.py"

    %% Click Events for Data Management
    click DM "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/data/"
    click DM1 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/data/raw"
    click DM2 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/data/interim"
    click DM3 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/data/processed"
    click DM4 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/data/uploaded"

    %% Click Events for Processing & Feature Extraction
    click DP "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/data_processing/pre_process.py"
    click FE "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/feature_extraction/feature_extraction.py"

    %% Click Events for Model Handling & Evaluation
    click M1 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/model/load_model.py"
    click M2 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/model/finetune_model.py"
    click M3 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/model/transcripts.py"
    click M4 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/model/practice_load_model.py"
    click MF "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/models/base/extracted/base.pt"
    click EV "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/evaluation/evaluation.py"

    %% Click Events for Utilities & Configurations
    click UT1 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/utils/extract_youtube_audio.py"
    click UT2 "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/src/utils/load_audio.py"
    click CF "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/configs/"

    %% Click Events for Containerization & Deployment
    click DF "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/Dockerfile"
    click DC "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/.dockerignore"
    click PK "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/packages.txt"
    click RT "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/blob/main/requirements.txt"

    %% Click Events for Experimentation & Documentation
    click NB "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/notebooks/"
    click DOC "https://github.com/rupeshkeesaram/applied-ml-speech-to-text/tree/main/docs/"

    %% Styling
    classDef frontend fill:#AEDFF7,stroke:#333,stroke-width:2px;
    classDef backend fill:#F7E8AA,stroke:#333,stroke-width:2px;
    classDef infra fill:#C3FDB8,stroke:#333,stroke-width:2px;
```
