# yeyupiaoling/audioclassification-pytorch architecture

> Architecture diagram of the GitHub repository yeyupiaoling/audioclassification-pytorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yeyupiaoling/audioclassification-pytorch
- Repository: https://github.com/yeyupiaoling/audioclassification-pytorch

## Overview

##### Project type and purpose
- This is an open-source audio classification system built on PyTorch.
- It provides end-to-end CLI-driven pipelines: data preparation, feature extraction, training, evaluation and inference.
- Core use cases: environment-sound classification, speaker verification, real-time audio recognition.

##### File-structure analysis
Top-level scripts (entry points)
- create_data.py       ← raw-audio → train/test lists
- extract_features.py ← raw-audio → saved features
- train.py            ← configs → model training
- eval.py             ← trained model → metrics + confusion plots
- infer.py            ← single audio → prediction
- infer_record.py     ← streaming mic → continuous prediction
- record_audio.py     ← record mic → raw wav files
Configuration
- configs/*.yml       ← data augment, model, optimizer, scheduler, preprocess parameters
Core library package “macls/”
- data_utils/         ← Reader, Featurizer, CollateFn, DataLoader wrappers
- models/             ← Model definitions (EcapaTdnn, ResNetSE, etc.) + pooling layers + utils
- optimizer/          ← Scheduler, optimizer wrappers
- metric/             ← Accuracy, loss, confusion-matrix generation
- utils/              ← Checkpointing, argument parsing, logging, record I/O
- trainer.py          ← High-level Trainer class orchestrating epochs, forward/backward, val/test
- predict.py          ← Prediction utilities for batch inference
Docs & tooling
- docs/images/        ← sample outputs, confusion matrices
- tools/download_*.sh ← external data download
- requirements.txt, setup.py

##### README insights
- Config-driven: everything—from feature type to model architecture to optimizer—is controlled via YAML.
- Extensible models / pooling / preprocessors.
- VisualDL used for training logging and real-time monitoring.

##### Main components
A. Data Sources
- Raw audio datasets (e.g., UrbanSound8K, custom dialect datasets)
- Noise datasets for augmentation
B. Data Preparation
- create_data.py → generates train/test lists
C. Feature Extraction
- extract_features.py or on-the-fly via macls/data_utils/featurizer
- Feature store on disk (`dataset/features/`)
D. Training Pipeline
- train.py invokes macls/trainer.Trainer
- DataLoader (macls/data_utils) → batch of features/labels
- Model (macls/models) → forward pass
- Loss & Metric (macls/metric) → compute loss/accuracy
- Optimizer & Scheduler (macls/optimizer) → update weights
- Checkpointing (macls/utils/checkpoint) → save best model
- VisualDL server ← logs & scalars
E. Evaluation Pipeline
- eval.py → loads checkpoint → runs evaluation loop → outputs metrics & confusion images
F. Inference
- infer.py → single or batch audio → preprocess → model → label prediction
- infer_record.py + record_audio.py → real-time streaming inference
G. Configuration
- YAML files feed into all CLI scripts via unified argument parser
H. External Services/Tools
- VisualDL for dashboards
- HuggingFace models (optional) as preprocessors
- Shell scripts in tools/ for dataset download

##### Relationships and interactions
Raw Audio → Data Preparation → Feature Extraction → Training/Inference
Training → Model Artifacts → Evaluation & Inference
Configs → all pipelines (inject parameters)
- Trainer depends on DataUtils, Models, Optimizer, Metric, Utils
- Inference reuses DataUtils + Models + Utils

##### Architectural patterns & principles
- Modular separation by concern (data, models, training, utils)
- Config-driven orchestration (YAML + argparse)
- Plugin-style extensibility (drop-in new model in macls/models or new pooling in macls/models/pooling.py)
- Batch vs. streaming inference reuse core modules

##### Essential elements
- Clear labels per component (e.g., “Trainer”, “Featurizer”)
- Containers grouping “Data”, “Core Library”, “Scripts”, “Monitoring”

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Storage Layer
    subgraph "Data Storage" 
        direction TB
        RawAudio["Raw Audio Datasets"]:::data
        NoiseData["Noise Datasets"]:::data
        FeatureStore["Feature Store<br/>(dataset/features/)"]:::data
        ModelArtifacts["Model Artifacts<br/>(saved models)"]:::data
    end

    %% Configuration
    Config["Configuration YAML<br/>(configs/*.yml)"]:::config

    %% External Scripts
    ExternalDownload["External Data Download Script<br/>(tools/download_language_data.sh)"]:::external

    %% CLI Scripts
    subgraph "CLI Scripts"
        direction TB
        DataPrep["Data Preparation<br/>(create_data.py)"]:::process
        FeatureExtScript["Feature Extraction CLI<br/>(extract_features.py)"]:::process
        trainScript["Training Pipeline CLI<br/>(train.py)"]:::process
        evalScript["Evaluation Pipeline CLI<br/>(eval.py)"]:::process
        inferScript["Single-File Inference CLI<br/>(infer.py)"]:::process
        inferRecordScript["Streaming Inference CLI<br/>(infer_record.py)"]:::process
        recordAudioScript["Audio Recording Utility<br/>(record_audio.py)"]:::process
    end

    %% Core Library
    subgraph "macls Core Library"
        direction TB
        subgraph "Data Utils"
            direction TB
            DataReader["Reader Module"]:::process
            Featurizer["Featurizer Module"]:::process
            CollateFn["Collate Fn Module"]:::process
        end
        subgraph "Models"
            direction TB
            EcapaTDNN["Ecapa-TDNN"]:::process
            ResNetSE["ResNet-SE"]:::process
            Res2Net["Res2Net"]:::process
            ERes2Net["ERes2Net"]:::process
            CAMpp["CAM++"]:::process
            PANNs["PANNs"]:::process
            TDNN["TDNN"]:::process
            Pooling["Pooling Layers & Utils"]:::process
        end
        Optimizer["Optimizer & Scheduler Module"]:::process
        Metric["Metric Module"]:::process
        Checkpoint["Checkpoint Utility"]:::process
        Utils["General Utilities"]:::process
        Trainer["Trainer Core Class"]:::process
        PredictUtil["Prediction Utilities"]:::process
    end

    %% Monitoring
    subgraph "Monitoring & Outputs"
        direction TB
        VisualDL["VisualDL Server"]:::external
        ConfusionPlots["Confusion Images<br/>(docs/images/)"]:::external
    end

    %% Data Flow & Control Flow
    RawAudio -->|provides raw audio| DataPrep
    NoiseData -->|provides noise| DataPrep
    ExternalDownload -->|downloads datasets| RawAudio

    DataPrep -->|generates lists| FeatureExtScript
    Config --> DataPrep

    FeatureExtScript -->|calls featurizer| Featurizer
    Featurizer -->|saves features| FeatureStore
    Config --> FeatureExtScript

    FeatureStore -->|inputs features| trainScript
    Config --> trainScript

    trainScript -->|uses| DataReader
    trainScript -->|uses| CollateFn
    trainScript -->|instantiates| Trainer

    Trainer -->|loads| Optimizer
    Trainer -->|loads| Metric
    Trainer -->|calls| EcapaTDNN
    Trainer -->|calls| ResNetSE
    Trainer -->|calls| Res2Net
    Trainer -->|calls| ERes2Net
    Trainer -->|calls| CAMpp
    Trainer -->|calls| PANNs
    Trainer -->|calls| TDNN
    Trainer -->|uses| Pooling
    Trainer -->|uses| Utils

    Trainer -->|saves| Checkpoint
    Checkpoint --> ModelArtifacts
    Trainer -->|logs| VisualDL

    ModelArtifacts --> evalScript
    Config --> evalScript
    evalScript -->|outputs| ConfusionPlots

    ModelArtifacts --> inferScript
    Config --> inferScript
    inferScript -->|calls| Utils
    inferScript -->|calls| PredictUtil
    inferScript -->|calls| Trainer

    recordAudioScript -->|provides audio stream| inferRecordScript
    inferRecordScript -->|uses| Utils
    inferRecordScript -->|uses| PredictUtil
    inferRecordScript -->|uses| Trainer
    Config --> inferRecordScript

    %% Click Events
    click DataPrep "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/create_data.py"
    click FeatureExtScript "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/extract_features.py"
    click Featurizer "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/data_utils/featurizer.py"
    click DataReader "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/data_utils/reader.py"
    click CollateFn "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/data_utils/collate_fn.py"
    click trainScript "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/train.py"
    click Trainer "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/trainer.py"
    click EcapaTDNN "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/ecapa_tdnn.py"
    click ResNetSE "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/resnet_se.py"
    click Res2Net "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/res2net.py"
    click ERes2Net "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/eres2net.py"
    click CAMpp "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/campplus.py"
    click PANNs "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/panns.py"
    click TDNN "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/tdnn.py"
    click Pooling "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/pooling.py"
    click Pooling "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/models/utils.py"
    click Optimizer "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/optimizer/scheduler.py"
    click Metric "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/metric/metrics.py"
    click Checkpoint "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/utils/checkpoint.py"
    click Utils "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/utils/utils.py"
    click PredictUtil "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/macls/predict.py"
    click evalScript "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/eval.py"
    click inferScript "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/infer.py"
    click inferRecordScript "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/infer_record.py"
    click recordAudioScript "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/record_audio.py"
    click Config "https://github.com/yeyupiaoling/audioclassification-pytorch/tree/master/configs/"
    click ExternalDownload "https://github.com/yeyupiaoling/audioclassification-pytorch/blob/master/tools/download_language_data.sh"

    %% Styles
    classDef data fill:#ccffcc,stroke:#2d7732,color:#2d7732
    classDef process fill:#cce5ff,stroke:#336699,color:#336699
    classDef config fill:#fff0b3,stroke:#b38f00,color:#b38f00
    classDef external fill:#ffe0b3,stroke:#cc8400,color:#cc8400

    %% Legend
    subgraph "Legend"
        direction TB
        D["Data Storage"]:::data --- P["Processing Modules"]:::process
        C["Config/CLI"]:::config --- E["External Tools"]:::external
    end
```
