# chinjouli/dysaug-vc architecture

> Architecture diagram of the GitHub repository chinjouli/dysaug-vc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chinjouli/dysaug-vc
- Repository: https://github.com/chinjouli/dysaug-vc

## Overview

This research repository evaluates multilingual dysarthric-speech recognition and voice-conversion-related augmentation. Its main executable workflow prepares speech/text pairs, fine-tunes MMS adapters, runs test-set transcription, and scores CER/WER by severity. A separate objective-analysis script trains an XGBoost classifier on prosody features and compares synthesized datasets; the README also identifies subjective-evaluation samples. The graph follows visible calls and data use; augmentation inputs and some analysis artifacts are not implemented in the sampled code.

Main entry points: [`asr/mms_experiment.py`](https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_experiment.py) (Fine-tuning workflow), [`asr/mms_inference.py`](https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_inference.py) (Test inference), [`asr/geter.py`](https://github.com/chinjouli/dysaug-vc/blob/main/asr/geter.py) (Severity error scoring), [`analysis/obj_eval_xgboost.py`](https://github.com/chinjouli/dysaug-vc/blob/main/analysis/obj_eval_xgboost.py) (Severity classifier)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_asr["ASR experiments"]
  node_speech_data[("Speech and text")]
  node_experiment["Fine-tuning workflow<br/>[mms_experiment.py]"]
  node_normalizer["Text normalizer<br/>[whispernorm.py]"]
  node_collator["CTC batch collator<br/>[mms_utils.py]"]
  node_mms["MMS pretrained model"]
  node_trainer["Trainer"]
  node_adapter["Saved adapter"]
  node_inference["Test inference<br/>[mms_inference.py]"]
  node_test_data[("Test audio and CSV")]
end

subgraph group_evaluation["Evaluation"]
  node_transcriptions["Transcription output"]
  node_scorer["Severity error scoring<br/>[geter.py]"]
  node_normalizer_eval["Evaluation normalizer<br/>[whispernorm.py]"]
  node_error_rates["CER and WER"]
end

subgraph group_analysis["Objective analysis"]
  node_feature_data[("Prosody feature datasets")]
  node_classifier["Severity classifier"]
  node_classification["Dataset classifications"]
  node_subjective["Subjective samples"]
end

node_researcher(("Researcher"))

node_researcher -->|"starts training"| node_experiment
node_speech_data -->|"provides pairs"| node_experiment
node_experiment -->|"normalizes text"| node_normalizer
node_experiment -->|"batches features"| node_collator
node_experiment -->|"loads model"| node_mms
node_experiment -->|"configures and trains"| node_trainer
node_trainer -->|"saves weights"| node_adapter
node_researcher -->|"starts inference"| node_inference
node_test_data -->|"provides test inputs"| node_inference
node_inference -->|"loads model"| node_mms
node_adapter -.->|"checkpoint input"| node_inference
node_inference -->|"writes hypotheses"| node_transcriptions
node_transcriptions -->|"reads hypotheses"| node_scorer
node_test_data -->|"reads severity labels"| node_scorer
node_scorer -->|"normalizes text"| node_normalizer_eval
node_scorer -->|"computes CER and WER"| node_error_rates
node_researcher -->|"runs analysis"| node_classifier
node_feature_data -->|"loads features"| node_classifier
node_classifier -->|"writes classifications"| node_classification

click node_experiment "https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_experiment.py"
click node_normalizer "https://github.com/chinjouli/dysaug-vc/blob/main/asr/whispernorm.py"
click node_collator "https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_utils.py"
click node_inference "https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_inference.py"
click node_scorer "https://github.com/chinjouli/dysaug-vc/blob/main/asr/geter.py"
click node_normalizer_eval "https://github.com/chinjouli/dysaug-vc/blob/main/asr/whispernorm.py"
click node_classifier "https://github.com/chinjouli/dysaug-vc/blob/main/analysis/obj_eval_xgboost.py"
click node_subjective "https://github.com/chinjouli/dysaug-vc/blob/main/analysis/subj_eval_sample.pdf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_speech_data,node_experiment,node_normalizer,node_collator,node_mms,node_trainer,node_adapter,node_inference,node_test_data toneBlue
class node_transcriptions,node_scorer,node_normalizer_eval,node_error_rates toneAmber
class node_feature_data,node_classifier,node_classification,node_subjective toneMint
class node_researcher toneIndigo
```

## Components

### ASR experiments

- **Speech and text**
- [Fine-tuning workflow](https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_experiment.py): `asr/mms_experiment.py`
- [Text normalizer](https://github.com/chinjouli/dysaug-vc/blob/main/asr/whispernorm.py): `asr/whispernorm.py`
- [CTC batch collator](https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_utils.py): `asr/mms_utils.py`
- **MMS pretrained model**
- **Trainer**
- **Saved adapter**
- [Test inference](https://github.com/chinjouli/dysaug-vc/blob/main/asr/mms_inference.py): `asr/mms_inference.py`
- **Test audio and CSV**

### Evaluation

- **Transcription output**
- [Severity error scoring](https://github.com/chinjouli/dysaug-vc/blob/main/asr/geter.py): `asr/geter.py`
- [Evaluation normalizer](https://github.com/chinjouli/dysaug-vc/blob/main/asr/whispernorm.py): `asr/whispernorm.py`
- **CER and WER**

### Objective analysis

- **Prosody feature datasets**
- [Severity classifier](https://github.com/chinjouli/dysaug-vc/blob/main/analysis/obj_eval_xgboost.py): `analysis/obj_eval_xgboost.py`
- **Dataset classifications**
- [Subjective samples](https://github.com/chinjouli/dysaug-vc/blob/main/analysis/subj_eval_sample.pdf): `analysis/subj_eval_sample.pdf`

### Other components

- **Researcher**

## Connections

- Researcher → Fine-tuning workflow: starts training (evidence: `asr/mms_experiment.py`)
- Speech and text → Fine-tuning workflow: provides pairs (evidence: `asr/mms_experiment.py`)
- Fine-tuning workflow → Text normalizer: normalizes text (evidence: `asr/mms_experiment.py`)
- Fine-tuning workflow → CTC batch collator: batches features (evidence: `asr/mms_experiment.py`)
- Fine-tuning workflow → MMS pretrained model: loads model (evidence: `asr/mms_experiment.py`)
- Fine-tuning workflow → Trainer: configures and trains (evidence: `asr/mms_experiment.py`)
- Trainer → Saved adapter: saves weights
- Researcher → Test inference: starts inference (evidence: `asr/mms_inference.py`)
- Test audio and CSV → Test inference: provides test inputs (evidence: `asr/mms_inference.py`)
- Test inference → MMS pretrained model: loads model (evidence: `asr/mms_inference.py`)
- Saved adapter → Test inference: checkpoint input
- Test inference → Transcription output: writes hypotheses (evidence: `asr/mms_inference.py`)
- Transcription output → Severity error scoring: reads hypotheses (evidence: `asr/geter.py`)
- Test audio and CSV → Severity error scoring: reads severity labels (evidence: `asr/geter.py`)
- Severity error scoring → Evaluation normalizer: normalizes text (evidence: `asr/geter.py`)
- Severity error scoring → CER and WER: computes CER and WER (evidence: `asr/geter.py`)
- Researcher → Severity classifier: runs analysis (evidence: `analysis/obj_eval_xgboost.py`)
- Prosody feature datasets → Severity classifier: loads features (evidence: `analysis/obj_eval_xgboost.py`)
- Severity classifier → Dataset classifications: writes classifications (evidence: `analysis/obj_eval_xgboost.py`)
