# rishig10/voice-based-pattern-detection architecture

> Architecture diagram of the GitHub repository rishig10/voice-based-pattern-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rishig10/voice-based-pattern-detection
- Repository: https://github.com/rishig10/voice-based-pattern-detection

## Overview

##### Project Type and Purpose
- This is an open-source data‐processing and ML pipeline (not a web or mobile app).
- Its goal is to ingest raw voice recordings, extract speech/text and acoustic features, train a Random Forest model to distinguish “control” vs. “dementia,” then serialize the model for later inference.

##### File Structure
- Top level:
- audio_data/ — input dataset (two subdirs: control/, dementia/)
- notebook.ipynb & notebook_converted.py — the main training workflow
- test_model.py — standalone inference script
- saved_model_top/ — holds serialized artifacts: best_rf_model_top.pkl, scaler.pkl, feature_names.pkl
- README.md, .gitignore, etc.
- Patterns: a batch‐oriented ETL + model‐train pipeline, then an offline inference tool.

##### README for Insights
- Uses OpenAI Whisper for speech‐to‐text (needs ffmpeg).
- Extracts pause/hesitation metrics and acoustic features via librosa/pydub.
- Trains and evaluates a Random Forest (scikit‐learn), persists model+scaler+feature list.

##### Components and Interactions
A. Main Components
1. Data Store: File system folder “audio_data” with class subfolders
2. Preprocessing Pipeline (Notebook/Script):
a. Audio Loader → ffmpeg (via pydub)
b. Speech-to-Text Service (OpenAI Whisper model)
c. Acoustic Feature Extractor (librosa/pydub)
d. Text-Processing Module (pause/hesitation metrics)
e. Dataframe Builder (pandas) and Scaler (scikit-learn)
3. Model Trainer: Random Forest trainer + cross-validation → selects best model
4. Artifact Store: saved_model_top/ directory (model.pkl, scaler.pkl, feature_names.pkl)
5. Inference Module: test_model.py reads new audio file → runs same preprocessing → loads artifacts → outputs prediction & confidence

B. Relationships and Data Flow

C. Architectural Patterns
- Batch ETL + Model Training Stage separated from Runtime Inference Stage
- Pipeline broken into discrete steps (modular feature‐extractor, transcription, scaling)
- Reuse of code/modules for both training and inference

D. Key Technologies / Libraries
- OpenAI Whisper (speech-to-text)
- ffmpeg (audio decoding) via pydub
- librosa (acoustic features)
- pandas / numpy (data management)
- scikit-learn (scaler + Random Forest)
- pickle (model serialization)

##### Detail
- Sub‐component in the preprocessing stage (Transcriber, Acoustic Feature Extractor, Text Metrics).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Stores
    A["Audio Data Store\n(audio_data/)"]:::dataStore
    S["Artifact Store\n(saved_model_top/)"]:::dataStore

    %% Preprocessing Pipeline
    subgraph "Preprocessing Pipeline"
        direction TB
        AL["Audio Loader\n(pydub + ffmpeg)"]:::process
        ST["Transcriber\n(OpenAI Whisper)"]:::process
        AF["Acoustic Feature Extractor\n(librosa/pydub)"]:::process
        TM["Text-Processing Module\n(pause/hesitation)"]:::process
        DF["Dataframe Builder & Scaler\n(pandas + sklearn)"]:::process
    end

    %% Model Training
    MT["Model Training\n(Random Forest + CV)"]:::training

    %% Inference
    IF["Inference Module\n(test_model.py)"]:::inference

    %% Data Flow
    A -->|".wav files"| AL
    AL -->|"raw audio"| ST
    ST -->|"transcript"| AF
    AF -->|"acoustic features"| TM
    TM -->|"text metrics"| DF
    DF -->|"feature_vectors.csv"| MT
    MT -->|"best_rf_model_top.pkl\nscaler.pkl\nfeature_names.pkl"| S

    %% Inference Flow
    S -->|"artifacts"| IF
    IF -->|"prediction & confidence"| O["Output"]:::external

    %% Manual Triggers
    A -.->|"Run notebook.ipynb or notebook_converted.py"| MT
    S -.->|"Invoke test_model.py"| IF

    %% Click Events
    click A "https://github.com/rishig10/voice-based-pattern-detection/tree/main/audio_data/"
    click S "https://github.com/rishig10/voice-based-pattern-detection/tree/main/saved_model_top/"
    click AL "https://github.com/rishig10/voice-based-pattern-detection/blob/main/notebook.ipynb"
    click ST "https://github.com/rishig10/voice-based-pattern-detection/blob/main/notebook.ipynb"
    click AF "https://github.com/rishig10/voice-based-pattern-detection/blob/main/notebook.ipynb"
    click TM "https://github.com/rishig10/voice-based-pattern-detection/blob/main/notebook.ipynb"
    click DF "https://github.com/rishig10/voice-based-pattern-detection/blob/main/notebook.ipynb"
    click MT "https://github.com/rishig10/voice-based-pattern-detection/blob/main/notebook_converted.py"
    click IF "https://github.com/rishig10/voice-based-pattern-detection/blob/main/test_model.py"
    click S "https://github.com/rishig10/voice-based-pattern-detection/tree/main/saved_model_top/"
    click S "https://github.com/rishig10/voice-based-pattern-detection/tree/main/saved_model_top/"
    click S "https://github.com/rishig10/voice-based-pattern-detection/tree/main/saved_model_top/"
    click S "https://github.com/rishig10/voice-based-pattern-detection/blob/main/saved_model_top/best_rf_model_top.pkl"
    click S "https://github.com/rishig10/voice-based-pattern-detection/blob/main/saved_model_top/scaler.pkl"
    click S "https://github.com/rishig10/voice-based-pattern-detection/blob/main/saved_model_top/feature_names.pkl"

    %% Styles
    classDef dataStore fill:#aaccee,stroke:#333,stroke-width:1px
    classDef process fill:#a6e0a6,stroke:#333,stroke-width:1px
    classDef training fill:#f4b183,stroke:#333,stroke-width:1px
    classDef inference fill:#ffe599,stroke:#333,stroke-width:1px
    classDef external fill:#dddddd,stroke:#333,stroke-width:1px
```
