# dinhthixuanbinh/voicefilter architecture

> Architecture diagram of the GitHub repository dinhthixuanbinh/voicefilter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dinhthixuanbinh/voicefilter
- Repository: https://github.com/dinhthixuanbinh/voicefilter

## Overview

##### Project type and purpose
- This is a machine-learning research project (TensorFlow/Keras implementation) for speech separation (“VoiceFilter”) targeted at isolating a primary speaker’s voice from a mixed audio stream.
- It ingests LibriSpeech data, builds a custom dataset (spectrograms + speaker embeddings), trains a deep neural network, and evaluates restoration quality (SDR).

##### File-structure analysis
- Top level:
- README.md, VoiceFilter.ipynb – documentation and exploratory notebook
- requirements.txt – Python dependencies
- configuration/ – Python modules defining audio I/O parameters (Audio.py) and hyperparameters (HyperParams.py)
- dataset/ – scripts to generate and load the dataset:
- dataset_creation.py (mixing, reference embedding)
- directory_structure.py (folder organization)
- preload_training_dataset.py (data generators)
- speech_collection.py (Librispeech subset selection)
- model/ – core network definition:
- model.py (functional Keras model)
- sequence_generator.py (time-series input handling)
- train_test/ – orchestration scripts:
- train.py (training loop, callbacks)
- testing.py (evaluation, SDR measurement)

##### Key insights from README
- Two-level architecture:
a. Data-prep pipeline: converting audio → magnitude spectrogram + d-vector embedding.
b. Model pipeline: takes spectrogram + d-vector → mask → output spectrogram → reconstruct audio.
- External dependency: pre-trained speaker embedding extractor (from mindslab-ai).
- Metrics: MSE loss, Adam optimizer, evaluation via SDR.
- Deployment/environment: Python 3.6.9, Google Colab / GPU.

a. Main components
- Data sources:
– LibriSpeech raw audio
– Pre-trained embedding model (external service or library)
- Data preprocessing pipeline:
– Audio Loader & Mixer
– Spectrogram converter (librosa)
– Embedding generator
– Custom Keras data generator
- Model training pipeline:
– Keras functional model (model.py)
– Hyperparameters module (HyperParams.py)
– Training scripts with callbacks
– Checkpoint & logging
- Evaluation pipeline:
– Testing script computing SDR
– Audio reconstructor (spectrogram + phase)
- Artifacts & assets:
– Stored model weights
– Audio samples (assets/audio_samples)

b. Relationships and data flows
- From LibriSpeech → dataset_creation.py → spectrograms
- From spectrograms + embedding extractor → data generator → model training
- From model → mask → audio reconstructor → output audio
- From trained model → testing.py → SDR metric
- Feedback loop: training metrics → hyperparameter tuning.

c. Architectural patterns or principles
- Modular separation of concerns:
- configuration vs. data vs. model vs. orchestration
- Pipeline pattern for data preprocessing → training → evaluation
- Use of custom generators and callbacks (demonstrating extensibility).

d. Key technologies
- Python 3.6, TensorFlow/Keras
- Librosa (audio processing)
- NumPy, multiprocessing for data loading
- Jupyter Notebook for exploration

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Documentation and Dependencies
    README["README.md"]:::docs
    NOTEBOOK["VoiceFilter.ipynb"]:::docs
    REQS["requirements.txt"]:::docs

    %% External Service
    EMBEDDING["Pre-trained Embedding Service"]:::external

    %% Data Preparation Layer
    subgraph "Data Preparation" 
        direction TB
        CONFIG_AUDIO["Audio.py"]:::config
        CONFIG_HYPER["HyperParams.py"]:::config
        DS_CREATE["dataset_creation.py"]:::dataprep
        DS_DIR["directory_structure.py"]:::dataprep
        DS_PRELOAD["preload_training_dataset.py"]:::dataprep
        DS_SPEECH["speech_collection.py"]:::dataprep
        SPECTRO["Spectrogram Store"]:::artifacts
    end

    %% Model Training Layer
    subgraph "Model Training"
        direction TB
        MODEL_DEF["model.py"]:::training
        SEQ_GEN["sequence_generator.py"]:::training
        TRAIN_SCRIPT["train.py"]:::training
        CHECKPOINTS["Model Checkpoints"]:::artifacts
        LOGS["Training Logs"]:::artifacts
    end

    %% Evaluation Layer
    subgraph "Evaluation"
        direction TB
        TEST_SCRIPT["testing.py"]:::evaluation
        RECON["Audio Reconstructor"]:::evaluation
        SDR["SDR Metrics"]:::artifacts
        ASSET_AUDIO["assets/audio_samples/"]:::artifacts
        ASSET_IMG["assets/images/"]:::artifacts
    end

    %% Notebook & Docs to Data Prep
    README --> DS_CREATE
    NOTEBOOK --> DS_CREATE
    REQS --> DS_CREATE

    %% Config to Data Prep
    CONFIG_AUDIO -.-> DS_CREATE
    CONFIG_HYPER -.-> DS_CREATE

    %% Data Prep Flow
    DS_SPEECH --> DS_CREATE
    EMBEDDING --> DS_PRELOAD
    DS_CREATE --> SPECTRO
    SPECTRO --> DS_PRELOAD
    DS_PRELOAD -->|spectrogram + embeddings| MODEL_DEF

    %% Config to Training
    CONFIG_HYPER -.-> TRAIN_SCRIPT

    %% Model Definition to Training
    MODEL_DEF --> TRAIN_SCRIPT
    SEQ_GEN --> TRAIN_SCRIPT

    %% Training Outputs
    TRAIN_SCRIPT --> CHECKPOINTS
    TRAIN_SCRIPT --> LOGS

    %% Evaluation Flow
    CHECKPOINTS --> TEST_SCRIPT
    TEST_SCRIPT --> SDR
    SDR --> RECON
    RECON --> ASSET_AUDIO
    RECON --> ASSET_IMG

    %% External Embedding used by Data Prep
    EMBEDDING --> DS_PRELOAD

    %% Click Events
    click README "https://github.com/dinhthixuanbinh/voicefilter/blob/master/README.md"
    click NOTEBOOK "https://github.com/dinhthixuanbinh/voicefilter/blob/master/VoiceFilter.ipynb"
    click REQS "https://github.com/dinhthixuanbinh/voicefilter/blob/master/requirements.txt"
    click CONFIG_AUDIO "https://github.com/dinhthixuanbinh/voicefilter/blob/master/configuration/Audio.py"
    click CONFIG_HYPER "https://github.com/dinhthixuanbinh/voicefilter/blob/master/configuration/HyperParams.py"
    click DS_CREATE "https://github.com/dinhthixuanbinh/voicefilter/blob/master/dataset/dataset_creation.py"
    click DS_DIR "https://github.com/dinhthixuanbinh/voicefilter/blob/master/dataset/directory_structure.py"
    click DS_PRELOAD "https://github.com/dinhthixuanbinh/voicefilter/blob/master/dataset/preload_training_dataset.py"
    click DS_SPEECH "https://github.com/dinhthixuanbinh/voicefilter/blob/master/dataset/speech_collection.py"
    click MODEL_DEF "https://github.com/dinhthixuanbinh/voicefilter/blob/master/model/model.py"
    click SEQ_GEN "https://github.com/dinhthixuanbinh/voicefilter/blob/master/model/sequence_generator.py"
    click TRAIN_SCRIPT "https://github.com/dinhthixuanbinh/voicefilter/blob/master/train_test/train.py"
    click TEST_SCRIPT "https://github.com/dinhthixuanbinh/voicefilter/blob/master/train_test/testing.py"

    %% Styles
    classDef docs fill:#E6E6FA,stroke:#333,stroke-width:1px
    classDef config fill:#F4A460,stroke:#333,stroke-width:1px
    classDef dataprep fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef training fill:#90EE90,stroke:#333,stroke-width:1px
    classDef evaluation fill:#FFD580,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef artifacts fill:#FFE4E1,stroke:#333,stroke-width:1px
```
