# y3162/shinoda-lab architecture

> Architecture diagram of the GitHub repository y3162/shinoda-lab, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/y3162/shinoda-lab
- Repository: https://github.com/y3162/shinoda-lab

## Overview

This repository is a collection of Shinoda Lab speech-research projects rather than one application. A central workflow prepares utterance/noise configurations, trains speech-enhancement models, then evaluates enhanced noisy speech with ASR; a separate ASR-error-modeling project trains a text-prediction model from recognition results. The map includes those distinct stages and the documented model families. Sampled source confirms enhancement loading/inference and parts of ASR inference/training; several database and evaluation links are inferred only where filenames explicitly describe the operation, so uncertain wiring is omitted.

Main entry points: [`post-se-mixing/src/database/observation_asr_results_fill_common.py`](https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/observation_asr_results_fill_common.py) (Enhanced-audio ASR), [`asr-error-modeling/src/train/ntp.py`](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/train/ntp.py) (Text prediction training), [`speech-enhancement/src/database`](https://github.com/y3162/shinoda-lab/tree/main/speech-enhancement/src/database) (Dataset preparation), [`speech-enhancement/src/se_mamba/train.py`](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/se_mamba/train.py) (SE training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Audio data"]
  node_speech_corpora["Speech corpora"]
  node_noise_corpora["Noise corpora"]
  node_data_prep["Dataset preparation"]
  node_noise_configs[("Noise configurations")]
end

subgraph group_enhancement["Speech enhancement"]
  node_se_training["SE training<br/>[train.py]"]
  node_mp_training["MP-SENet training<br/>[train.py]"]
  node_mamba_pp_training["Mamba++ training<br/>[train.py]"]
  node_audio_mamba_training["Audio Mamba training<br/>[train.py]"]
  node_se_models["Enhancement models"]
  node_se_api["Enhancement API<br/>[api.py]"]
  node_checkpoints["Model checkpoints"]
end

subgraph group_evaluation["ASR evaluation"]
  node_observation_runs[("Evaluation runs")]
  node_se_asr_eval["Enhanced-audio ASR"]
  node_asr_api["ASR model API<br/>[api.py]"]
  node_asr_models["ASR models"]
  node_observation_results[("ASR observations")]
end

subgraph group_error["Error modeling"]
  node_asr_inference["Noisy ASR inference"]
  node_asr_results[("Recognition results")]
  node_pair_dataset["ASR pair dataset<br/>[dataset.py]"]
  node_ntp_training["Text prediction training<br/>[ntp.py]"]
  node_ntp_model["Transformer model<br/>[transformer.py]"]
  node_tokenizer["Text tokenizers<br/>[tokenizer.py]"]
end

node_researcher(("Researcher"))
node_external_asr{{"Pretrained ASR"}}

node_researcher -->|"prepares data"| node_data_prep
node_speech_corpora -->|"supplies speech"| node_data_prep
node_noise_corpora -->|"supplies noise"| node_data_prep
node_data_prep -->|"creates configs"| node_noise_configs
node_researcher -->|"trains model"| node_se_training
node_se_training -->|"builds"| node_se_models
node_mp_training -->|"saves weights"| node_checkpoints
node_se_training -->|"saves weights"| node_checkpoints
node_mamba_pp_training -->|"saves weights"| node_checkpoints
node_audio_mamba_training -->|"saves weights"| node_checkpoints
node_researcher -->|"records run"| node_observation_runs
node_observation_runs -->|"provides run"| node_se_asr_eval
node_se_asr_eval -->|"enhances audio"| node_se_api
node_se_api -->|"loads weights"| node_checkpoints
node_se_asr_eval -->|"transcribes mixes"| node_asr_api
node_asr_api -->|"dispatches"| node_asr_models
node_asr_models -.->|"loads pretrained"| node_external_asr
node_se_asr_eval -->|"writes observations"| node_observation_results
node_noise_configs -->|"configures noise"| node_asr_inference
node_asr_inference -.->|"transcribes audio"| node_external_asr
node_asr_inference -->|"stores hypotheses"| node_asr_results
node_asr_results -->|"provides pairs"| node_pair_dataset
node_researcher -->|"trains predictor"| node_ntp_training
node_ntp_training -->|"loads pairs"| node_pair_dataset
node_ntp_training -->|"encodes text"| node_tokenizer
node_ntp_training -->|"optimizes"| node_ntp_model

click node_data_prep "https://github.com/y3162/shinoda-lab/tree/main/speech-enhancement/src/database"
click node_noise_configs "https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/database/03_2_insert_noise_configs_from_demand.py"
click node_se_training "https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/se_mamba/train.py"
click node_mp_training "https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/mp_senet/train.py"
click node_mamba_pp_training "https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/se_mamba_pp/train.py"
click node_audio_mamba_training "https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/audio_mamba/train.py"
click node_se_models "https://github.com/y3162/shinoda-lab/tree/main/speech-enhancement/src/se_mamba/model"
click node_se_api "https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/api.py"
click node_observation_runs "https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/04_1_create_table_observation_eval_runs.py"
click node_se_asr_eval "https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/observation_asr_results_fill_common.py"
click node_asr_api "https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/asr/api.py"
click node_asr_models "https://github.com/y3162/shinoda-lab/tree/main/post-se-mixing/src/asr"
click node_observation_results "https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/05_1_create_table_observation_asr_results.py"
click node_asr_inference "https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/database/04_2_noisy_inference.py"
click node_asr_results "https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/database/04_1_create_table_asr_results.py"
click node_pair_dataset "https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/train/dataset.py"
click node_ntp_training "https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/train/ntp.py"
click node_ntp_model "https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/model/transformer.py"
click node_tokenizer "https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/model/tokenizer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_speech_corpora,node_noise_corpora,node_data_prep,node_noise_configs toneBlue
class node_se_training,node_mp_training,node_mamba_pp_training,node_audio_mamba_training,node_se_models,node_se_api,node_checkpoints toneAmber
class node_observation_runs,node_se_asr_eval,node_asr_api,node_asr_models,node_observation_results toneMint
class node_asr_inference,node_asr_results,node_pair_dataset,node_ntp_training,node_ntp_model,node_tokenizer toneRose
class node_researcher,node_external_asr toneIndigo
```

## Components

### Audio data

- **Speech corpora**
- **Noise corpora**
- [Dataset preparation](https://github.com/y3162/shinoda-lab/tree/main/speech-enhancement/src/database): `speech-enhancement/src/database`
- [Noise configurations](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/database/03_2_insert_noise_configs_from_demand.py): `speech-enhancement/src/database/03_2_insert_noise_configs_from_demand.py`

### Speech enhancement

- [SE training](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/se_mamba/train.py): `speech-enhancement/src/se_mamba/train.py`
- [MP-SENet training](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/mp_senet/train.py): `speech-enhancement/src/mp_senet/train.py`
- [Mamba++ training](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/se_mamba_pp/train.py): `speech-enhancement/src/se_mamba_pp/train.py`
- [Audio Mamba training](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/audio_mamba/train.py): `speech-enhancement/src/audio_mamba/train.py`
- [Enhancement models](https://github.com/y3162/shinoda-lab/tree/main/speech-enhancement/src/se_mamba/model): `speech-enhancement/src/se_mamba/model`
- [Enhancement API](https://github.com/y3162/shinoda-lab/blob/main/speech-enhancement/src/api.py): `speech-enhancement/src/api.py`
- **Model checkpoints**

### ASR evaluation

- [Evaluation runs](https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/04_1_create_table_observation_eval_runs.py): `post-se-mixing/src/database/04_1_create_table_observation_eval_runs.py`
- [Enhanced-audio ASR](https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/observation_asr_results_fill_common.py): `post-se-mixing/src/database/observation_asr_results_fill_common.py`
- [ASR model API](https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/asr/api.py): `post-se-mixing/src/asr/api.py`
- [ASR models](https://github.com/y3162/shinoda-lab/tree/main/post-se-mixing/src/asr): `post-se-mixing/src/asr`
- [ASR observations](https://github.com/y3162/shinoda-lab/blob/main/post-se-mixing/src/database/05_1_create_table_observation_asr_results.py): `post-se-mixing/src/database/05_1_create_table_observation_asr_results.py`

### Error modeling

- [Noisy ASR inference](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/database/04_2_noisy_inference.py): `asr-error-modeling/src/database/04_2_noisy_inference.py`
- [Recognition results](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/database/04_1_create_table_asr_results.py): `asr-error-modeling/src/database/04_1_create_table_asr_results.py`
- [ASR pair dataset](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/train/dataset.py): `asr-error-modeling/src/train/dataset.py`
- [Text prediction training](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/train/ntp.py): `asr-error-modeling/src/train/ntp.py`
- [Transformer model](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/model/transformer.py): `asr-error-modeling/src/model/transformer.py`
- [Text tokenizers](https://github.com/y3162/shinoda-lab/blob/main/asr-error-modeling/src/model/tokenizer.py): `asr-error-modeling/src/model/tokenizer.py`

### Other components

- **Researcher**
- **Pretrained ASR**

## Connections

- Researcher → Dataset preparation: prepares data
- Speech corpora → Dataset preparation: supplies speech
- Noise corpora → Dataset preparation: supplies noise
- Dataset preparation → Noise configurations: creates configs
- Researcher → SE training: trains model
- SE training → Enhancement models: builds
- MP-SENet training → Model checkpoints: saves weights
- SE training → Model checkpoints: saves weights
- Mamba++ training → Model checkpoints: saves weights
- Audio Mamba training → Model checkpoints: saves weights
- Researcher → Evaluation runs: records run
- Evaluation runs → Enhanced-audio ASR: provides run
- Enhanced-audio ASR → Enhancement API: enhances audio
- Enhancement API → Model checkpoints: loads weights
- Enhanced-audio ASR → ASR model API: transcribes mixes
- ASR model API → ASR models: dispatches
- ASR models → Pretrained ASR: loads pretrained
- Enhanced-audio ASR → ASR observations: writes observations
- Noise configurations → Noisy ASR inference: configures noise
- Noisy ASR inference → Pretrained ASR: transcribes audio
- Noisy ASR inference → Recognition results: stores hypotheses
- Recognition results → ASR pair dataset: provides pairs
- Researcher → Text prediction training: trains predictor
- Text prediction training → ASR pair dataset: loads pairs
- Text prediction training → Text tokenizers: encodes text
- Text prediction training → Transformer model: optimizes