# rayanwang/speech_emotion_recognition_blstm architecture

> Architecture diagram of the GitHub repository rayanwang/speech_emotion_recognition_blstm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rayanwang/speech_emotion_recognition_blstm
- Repository: https://github.com/rayanwang/speech_emotion_recognition_blstm

## Overview

##### Main Components
a. Data Sources
- Raw audio datasets (e.g., Berlin speech dataset)
b. Data Ingestion & Preprocessing
- dataset.py: loads raw audio and labels
- wav_extraction.py + pydub (utility/audio, utility/audiosegment): extracts WAV segments
- sentence_extraction.py + webrtcvad: splits audio into utterances
c. Feature Engineering
- External libs: librosa, pyAudioAnalysis
- utility/metrics_util.py, globalvars.py: compute & normalize features
- stFeatureSpeed patch in pyAudioAnalysis for mfcc, spectral features
- .p file storage of feature streams
d. Model Training & Hyperparameter Tuning
- DataAugmentationTraining.py: optional data-augmentation routines
- find_best_model.py: Hyperas + Keras (TensorFlow backend) for tuning (optimizers, batch_size, epochs)
- model_cross_validation.py: k-fold or speaker-independent CV using scikit-learn
e. Prediction & Evaluation
- prediction.py: loads saved model (.h5) + features → emotion labels
- utility/networks.py: BLSTM architecture definition
- utility/metrics_util.py: accuracy, confusion matrix
f. Utilities & Configuration
- utility/globalvars.py: global paths, hyperparameter defaults
- LICENSE, README.md, environment (Python, CUDA)

##### Architectural Patterns
- Pipeline pattern: strictly ordered stages
- Modular code: separate scripts for each pipeline stage
- Reusable utility modules for I/O, feature calculation, network definitions
- Hyperparameter-search sub-pipeline (Hyperas orchestrator calling find_best_model.py)

##### Technologies
- Python (2.7/3.6)
- Keras 2.1.5 on TensorFlow 1.6
- librosa, pyAudioAnalysis, pydub, webrtcvad
- Hyperas + scikit-learn
- NVIDIA GPU (CUDA)

##### Essential Elements
- Component labels (script names, utilities)
- Data stores: raw audio, feature pickles, model files

## Diagram (Mermaid)

```mermaid
graph TD
    %% Environment
    Env["Environment\n(Python 2.7/3.6, CUDA GPU)"]:::external

    %% Raw Data
    RawData["Raw Audio Datasets\n(e.g., Berlin)"]:::data

    %% Preprocessing Stage
    subgraph "Preprocessing" 
        DS["dataset.py\n(Load audio & labels)"]:::process
        WE["wav_extraction.py\n(Extract WAV)"]:::process
        SE["sentence_extraction.py\n(Utterance split)"]:::process
        AH1["utility/audio.py"]:::process
        AH2["utility/audiosegment.py"]:::process
    end

    %% Feature Engineering Stage
    subgraph "Feature Engineering"
        MU["utility/metrics_util.py\n(Compute & normalize)"]:::process
        GV["utility/globalvars.py\n(Constants & paths)"]:::config
        Libs["External Libs\n(librosa, pyAudioAnalysis)"]:::external
        FeatureStore["Feature Pickle Files\n(.p)"]:::data
    end

    %% Training & Tuning Stage
    subgraph "Model Training & Tuning"
        AUG["DataAugmentationTraining.py"]:::process
        HYP["find_best_model.py\n(Hyperas+Keras)"]:::process
        CV["model_cross_validation.py"]:::process
        ModelStore["Trained Model Files\n(.h5)"]:::data
    end

    %% Prediction & Evaluation Stage
    subgraph "Prediction & Evaluation"
        PRED["prediction.py\n(Predict labels)"]:::process
        NET["utility/networks.py\n(BLSTM arch)"]:::process
        ME["utility/metrics_util.py\n(Eval metrics)"]:::process
        Outputs["Emotion Outputs"]:::data
    end

    %% Utilities & Config
    subgraph "Utilities & Config"
        README["README.md"]:::config
        LICENSE["LICENSE"]:::config
    end

    %% Flow
    Env --> RawData
    RawData --> DS
    DS --> WE
    WE --> SE
    SE --> AH1
    AH1 --> AH2
    AH2 --> MU
    GV --> MU
    Libs --> MU
    MU --> FeatureStore
    FeatureStore --> AUG
    AUG --> HYP
    FeatureStore --> CV
    HYP --> ModelStore
    CV --> ME
    ModelStore --> PRED
    PRED --> NET
    NET --> ME
    ME --> Outputs

    %% Click Events
    click DS "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/dataset.py"
    click WE "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/wav_extraction.py"
    click SE "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/sentence_extraction.py"
    click AH1 "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/utility/audio.py"
    click AH2 "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/utility/audiosegment.py"
    click MU "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/utility/metrics_util.py"
    click GV "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/utility/globalvars.py"
    click AUG "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/DataAugmentationTraining.py"
    click HYP "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/find_best_model.py"
    click CV "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/model_cross_validation.py"
    click PRED "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/prediction.py"
    click NET "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/utility/networks.py"
    click README "https://github.com/rayanwang/speech_emotion_recognition_blstm/blob/master/README.md"
    click LICENSE "https://github.com/rayanwang/speech_emotion_recognition_blstm/tree/master/LICENSE"

    %% Styles
    classDef process fill:#bbdefb,stroke:#1e88e5,color:#0d47a1;
    classDef data fill:#c8e6c9,stroke:#2e7d32,color:#1b5e20;
    classDef external fill:#ffe0b2,stroke:#ef6c00,color:#e65100;
    classDef config fill:#f0f4c3,stroke:#827717,color:#33691e;
```
