How chinjouli/dysaug-vc works
This research repository evaluates multilingual dysarthric-speech recognition and voice-conversion-related augmentation. Its main executable workflow prepares speech/text pairs, fine-tunes MMS adapters, runs test-set transcription, and scores CER/WER by severity. A separate objective-analysis script trains an XGBoost classifier on prosody features and compares synthesized datasets; the README also identifies subjective-evaluation samples. The graph follows visible calls and data use; augmentation inputs and some analysis artifacts are not implemented in the sampled code.
Main entry points: mms_experiment.py Fine-tuning workflow · mms_inference.py Test inference · geter.py Severity error scoring · obj_eval_xgboost.py Severity classifier
Components 18
ASR experiments
- Speech and text
- Fine-tuning workflow
asr/mms_experiment.py - Text normalizer
asr/whispernorm.py - CTC batch collator
asr/mms_utils.py - MMS pretrained model
- Trainer
- Saved adapter
- Test inference
asr/mms_inference.py - Test audio and CSV
Evaluation
- Transcription output
- Severity error scoring
asr/geter.py - Evaluation normalizer
asr/whispernorm.py - CER and WER
Objective analysis
- Prosody feature datasets
- Severity classifier
analysis/obj_eval_xgboost.py - Dataset classifications
- Subjective samples
analysis/subj_eval_sample.pdf
Other components
- Researcher
Connections 19
- Researcher → Fine-tuning workflow: starts training
asr/mms_experiment.py - Speech and text → Fine-tuning workflow: provides pairs
asr/mms_experiment.py - Fine-tuning workflow → Text normalizer: normalizes text
asr/mms_experiment.py - Fine-tuning workflow → CTC batch collator: batches features
asr/mms_experiment.py - Fine-tuning workflow → MMS pretrained model: loads model
asr/mms_experiment.py - Fine-tuning workflow → Trainer: configures and trains
asr/mms_experiment.py - Trainer → Saved adapter: saves weights
- Researcher → Test inference: starts inference
asr/mms_inference.py - Test audio and CSV → Test inference: provides test inputs
asr/mms_inference.py - Test inference → MMS pretrained model: loads model
asr/mms_inference.py - Saved adapter → Test inference: checkpoint input
- Test inference → Transcription output: writes hypotheses
asr/mms_inference.py - Transcription output → Severity error scoring: reads hypotheses
asr/geter.py - Test audio and CSV → Severity error scoring: reads severity labels
asr/geter.py - Severity error scoring → Evaluation normalizer: normalizes text
asr/geter.py - Severity error scoring → CER and WER: computes CER and WER
asr/geter.py - Researcher → Severity classifier: runs analysis
analysis/obj_eval_xgboost.py - Prosody feature datasets → Severity classifier: loads features
analysis/obj_eval_xgboost.py - Severity classifier → Dataset classifications: writes classifications
analysis/obj_eval_xgboost.py