# nttcslab/dcase2023_task2_baseline_ae architecture

> Architecture diagram of the GitHub repository nttcslab/dcase2023_task2_baseline_ae, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nttcslab/dcase2023_task2_baseline_ae
- Repository: https://github.com/nttcslab/dcase2023_task2_baseline_ae

## Overview

##### Project Type and Purpose
- This is an open-source research baseline tool for first-shot unsupervised anomalous sound detection on the DCASE Challenge Task 2 (2023, 2024, 2025).
- It implements two scoring modes: a simple MSE autoencoder and a selective Mahalanobis distance AE, written in PyTorch.
- Orchestration is driven by shell scripts (train/test/summarize), and core logic lives in Python modules under `datasets/`, `networks/`, and top-level scripts.

##### File Structure and Architectural Patterns
- Top-level CLI/Scripts
- 01_train_2025t2.sh, 02a_test_2025t2.sh, 02b_test_2025t2.sh, 03_summarize_results.sh
- train.py (entry point for training)
- common.py (shared utilities)
- baseline.yaml (configuration)
- Data Layer
- `data/` contains raw & processed WAV files for dev/eval across multiple years and machine types
- `datasets/` holds dataset loaders, path configs (YAML/CSV), ground-truth CSVs
- Model Layer
- `networks/` defines base_model, AE architectures, criterion (MSE, Mahalanobis), and top-level models.py
- Persistence
- `models/` (checkpoints, saved_model)
- `results/` (raw anomaly scores, aggregated metrics)
- `logs/` (training logs)
- Utilities
- `tools/` for legacy support, data download scripts, plotting, renaming, result export
- `dockerfile`, `requirements.txt` for environment reproducibility

##### Key Components & Interactions
- External Data Sources → data_download_*.sh (or auto-download) → `data/dcaseXXXXt2/`
- Dataset Loader (datasets/dcase*_loader.py & loader_common.py) reads raw audio, applies preprocessing (feature extraction configured in baseline.yaml), outputs tensors
- Training Orchestrator (train.py invoked by 01_train_2025t2.sh)
- Loads config → instantiates Dataset & DataLoader → builds AE model(s) via networks/ → trains with MSE loss → after final epoch computes covariance matrices for Mahalanobis mode → saves checkpoints (models/)
- Testing Orchestrator (02a_*.sh / 02b_*.sh)
- Loads trained model & covariance (if Mahalanobis) → runs inference on test set → computes anomaly scores (MSE or Mahalanobis) → writes per-file CSV in results/ → for development also computes AUC, pAUC, precision, recall, F1 → writes summary CSV
- Summarization (03_summarize_results.sh)
- Gathers result CSVs → aggregates into a single table under results/.../summarize/

##### Architectural Patterns & Design Principles
- Layered architecture:
- Presentation/Orchestration layer (shell scripts, CLI)
- Application logic layer (train.py, common.py)
- Domain/model layer (networks/, criterion)
- Data abstraction layer (datasets/, data/)
- Configuration-driven: baseline.yaml centralizes hyperparameters and file paths
- Modular & extensible: networks/ can be extended with new AE variants; datasets/ can support new years/machine types
- Separation of concerns: data downloading, preprocessing, model training, scoring, and result export are decoupled

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    Zenodo("Zenodo Dataset Hosting"):::external
    DCASE_EVAL("DCASE Task2 Evaluator"):::external

    %% Utilities Layer
    subgraph "Utilities Layer"
        direction TB
        Tools("tools/"):::util
        Docker("dockerfile<br/>requirements.txt"):::util
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DataRaw("data/ (Raw & Processed Audio)"):::data
        Datasets("datasets/"):::python
    end

    %% Configuration
    BaselineYAML("baseline.yaml"):::config

    %% Application Logic Layer
    subgraph "Application Logic"
        direction TB
        Common("common.py"):::python
        TrainCLI("train.py"):::python
    end

    %% Shell Script Layer
    subgraph "Shell Script Layer"
        direction TB
        ScriptTrain("01_train_2025t2.sh"):::script
        ScriptTestA("02a_test_2025t2.sh"):::script
        ScriptTestB("02b_test_2025t2.sh"):::script
        ScriptSumm("03_summarize_results.sh"):::script
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        Networks("networks/"):::python
    end

    %% Persistence Layer
    subgraph "Persistence & Results"
        direction TB
        ModelsPersist("models/ (Checkpoints & Saved Models)"):::model
        ResultsPersist("results/ (Raw Scores & Metrics)"):::data
        Logs("logs/"):::data
    end

    %% Core Pipeline
    Zenodo -->|"hosts raw data"| DataRaw
    Tools -->|data_download scripts| DataRaw
    DataRaw -->|audio files| Datasets
    BaselineYAML -->|configs| Datasets
    BaselineYAML -->|configs| TrainCLI
    Datasets -->|tensors| TrainCLI
    Common -->|utilities| TrainCLI

    ScriptTrain -->|invokes| TrainCLI
    TrainCLI -->|builds AE model| Networks
    Networks -->|trained weights| ModelsPersist
    TrainCLI -->|logs| Logs

    ScriptTestA -->|invokes| TrainCLI
    ScriptTestB -->|invokes| TrainCLI
    TrainCLI -->|loads checkpoints| ModelsPersist
    TrainCLI -->|inference| Networks
    Networks -->|anomaly scores.csv| ResultsPersist
    ResultsPersist -->|metrics| ResultsPersist

    ScriptSumm -->|aggregates| ResultsPersist
    ScriptSumm -->|uses| Common
    ScriptSumm -->|outputs summary| ResultsPersist

    DCASE_EVAL -->|evaluation| ResultsPersist

    %% Scoring Modes Branch
    subgraph "Scoring Modes"
        direction TB
        SimpleAE("Simple AE (MSE)"):::model
        Mahalanobis("Selective Mahalanobis AE"):::model
    end
    Networks -->|MSE scores| SimpleAE
    Networks -->|covariance| Mahalanobis
    SimpleAE --> ResultsPersist
    Mahalanobis --> ResultsPersist

    %% Click Events
    click ScriptTrain "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/01_train_2025t2.sh"
    click ScriptTestA "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/02a_test_2025t2.sh"
    click ScriptTestB "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/02b_test_2025t2.sh"
    click ScriptSumm "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/03_summarize_results.sh"
    click TrainCLI "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/train.py"
    click Common "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/common.py"
    click BaselineYAML "https://github.com/nttcslab/dcase2023_task2_baseline_ae/blob/main/baseline.yaml"
    click DataRaw "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/data/"
    click Datasets "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/datasets/"
    click Networks "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/networks/"
    click ModelsPersist "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/models/"
    click ResultsPersist "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/results/"
    click Logs "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/logs/"
    click Tools "https://github.com/nttcslab/dcase2023_task2_baseline_ae/tree/main/tools/"

    %% Styles
    classDef script fill:#f9f,stroke:#333,stroke-dasharray: 5 5;
    classDef python fill:#bbf,stroke:#333;
    classDef data fill:#bfb,stroke:#333,stroke-width:2px;
    classDef model fill:#ffb,stroke:#333;
    classDef config fill:#fbf,stroke:#333,stroke-width:2px;
    classDef external fill:#eef,stroke:#333,stroke-dasharray: 2 2;
    classDef util fill:#ffd,stroke:#333;
```
