# renovamen/speech-emotion-recognition architecture

> Architecture diagram of the GitHub repository renovamen/speech-emotion-recognition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/renovamen/speech-emotion-recognition
- Repository: https://github.com/renovamen/speech-emotion-recognition

## Overview

##### Project type and purpose
- This is a machine-learning pipeline/tool for Speech Emotion Recognition (SER).
- It provides end-to-end functionality: raw audio → feature extraction → model training → checkpointing → inference/prediction → visualization.

##### File structure
- Top-level “scripts”:
- preprocess.py – orchestrates feature extraction & data preprocessing
- train.py – runs model training based on a YAML config
- predict.py – loads a checkpointed model to infer emotion on new audio
- Core modules:
- extract_feats/ – two interchangeable feature extractors (librosa.py and opensmile.py)
- models/
– base.py & dnn/dnn.py – base classes
– dnn/cnn.py and dnn/lstm.py – neural architectures
– ml.py – classical ML (SVM & MLP via scikit-learn)
- utils/ – shared helpers (file operations, CLI opts, plotting)
- Data and metadata:
- configs/*.yaml – all run-time parameters (paths, model hyperparams, feature set)
- features/ – serialized feature outputs (.p or .csv)
- checkpoints/ – serialized trained model weights & configs
- Support files: requirements.txt, LICENSE, README.md

##### README for additional insights
- Lists external libraries: TensorFlow/Keras (for CNN/LSTM), scikit-learn (SVM/MLP), joblib, librosa, SciPy, pandas, matplotlib, NumPy, optional Opensmile.
- Describes multi-dataset support, configurable feature sets (IS09, IS10, etc.).
- Emphasizes two feature-extraction backends and four model families.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    DataSources["Data Sources\n(RAVDESS, SAVEE, EMO-DB, CASIA)"]:::external

    %% Configuration Service
    Config["Configuration Service\n(configs/)"]:::config
    click Config "https://github.com/renovamen/speech-emotion-recognition/tree/master/configs/"

    %% Preprocessing Layer
    Preprocess["Preprocessing Layer\npreprocess.py"]:::compute
    click Preprocess "https://github.com/renovamen/speech-emotion-recognition/blob/master/preprocess.py"
    FileUtils["File Utilities\nutils/files.py"]:::util
    click FileUtils "https://github.com/renovamen/speech-emotion-recognition/blob/master/utils/files.py"

    %% Feature Extraction Layer
    subgraph "Feature Extraction"
        direction TB
        Librosa["Librosa Extractor\nextract_feats/librosa.py"]:::compute
        click Librosa "https://github.com/renovamen/speech-emotion-recognition/blob/master/extract_feats/librosa.py"
        Opensmile["Opensmile Extractor\nextract_feats/opensmile.py"]:::compute
        click Opensmile "https://github.com/renovamen/speech-emotion-recognition/blob/master/extract_feats/opensmile.py"
    end

    %% Feature Store
    FeatureStore["Feature Store\nfeatures/"]:::dataStore
    click FeatureStore "https://github.com/renovamen/speech-emotion-recognition/tree/master/features/"

    %% Training Layer
    Train["Training Layer\ntrain.py"]:::compute
    click Train "https://github.com/renovamen/speech-emotion-recognition/blob/master/train.py"
    subgraph "Models"
        direction TB
        Base["Base Classes\nmodels/base.py"]:::module
        click Base "https://github.com/renovamen/speech-emotion-recognition/blob/master/models/base.py"
        DNNBase["DNN Base\nmodels/dnn/dnn.py"]:::module
        click DNNBase "https://github.com/renovamen/speech-emotion-recognition/blob/master/models/dnn/dnn.py"
        CNN["CNN Architecture\nmodels/dnn/cnn.py"]:::module
        click CNN "https://github.com/renovamen/speech-emotion-recognition/blob/master/models/dnn/cnn.py"
        LSTM["LSTM Architecture\nmodels/dnn/lstm.py"]:::module
        click LSTM "https://github.com/renovamen/speech-emotion-recognition/blob/master/models/dnn/lstm.py"
        ML["Classical ML (SVM & MLP)\nmodels/ml.py"]:::module
        click ML "https://github.com/renovamen/speech-emotion-recognition/blob/master/models/ml.py"
        DNNBase --> CNN
        DNNBase --> LSTM
        Base --> DNNBase
        Base --> ML
    end

    %% Checkpoint Repository
    Checkpoint["Checkpoint Repository\ncheckpoints/"]:::dataStore
    click Checkpoint "https://github.com/renovamen/speech-emotion-recognition/tree/master/checkpoints/"

    %% Inference Layer
    Predict["Inference Layer\npredict.py"]:::compute
    click Predict "https://github.com/renovamen/speech-emotion-recognition/blob/master/predict.py"

    %% Visualization & Utils
    UtilsPlot["Visualization Utilities\nutils/plot.py"]:::util
    click UtilsPlot "https://github.com/renovamen/speech-emotion-recognition/blob/master/utils/plot.py"
    UtilsOpts["CLI Options Helper\nutils/opts.py"]:::util
    click UtilsOpts "https://github.com/renovamen/speech-emotion-recognition/blob/master/utils/opts.py"

    %% External Libraries
    TensorFlow["TensorFlow/Keras"]:::external
    SKL["scikit-learn"]:::external
    LibrosaLib["librosa"]:::external
    OpensmileExt["Opensmile CLI"]:::external

    %% Data Flow
    DataSources -->|"raw .wav"| Preprocess
    Preprocess -->|"preprocessed .wav"| Librosa
    Preprocess -->|"preprocessed .wav"| Opensmile
    Librosa -->|"features .csv/.p"| FeatureStore
    Opensmile -->|"features .csv/.p"| FeatureStore

    %% Config Dependencies
    Config --> Preprocess
    Config --> Train
    Config --> Predict

    %% Feature Store to Training
    FeatureStore -->|"features"| Train
    Train --> Base

    %% Training to Checkpoints
    Train -->|"models .h5/.json/.m"| Checkpoint

    %% Inference Flow
    Checkpoint --> Predict
    Predict -->|"predictions"| UtilsPlot

    %% Utils Dependencies
    FileUtils --> Preprocess
    FileUtils --> Train
    FileUtils --> Predict
    UtilsOpts --> Preprocess
    UtilsOpts --> Train
    UtilsOpts --> Predict

    %% Models invoked by Training
    Train --> Models

    %% External integrations
    Train --> TensorFlow
    Train --> SKL
    Librosa --> LibrosaLib
    Opensmile --> OpensmileExt

    %% Styles
    classDef dataStore fill:#BBDEFB,stroke:#0D47A1,color:#0D47A1;
    classDef compute fill:#C8E6C9,stroke:#1B5E20,color:#1B5E20;
    classDef external fill:#FFE0B2,stroke:#E65100,color:#E65100;
    classDef config fill:#FFF9C4,stroke:#F57F17,color:#F57F17;
    classDef util fill:#E1BEE7,stroke:#4A148C,color:#4A148C;
    classDef module fill:#B3E5FC,stroke:#01579B,color:#01579B;
```
