How chaituviking/prosodyrlvr works
ProsodyRLVR is a reproducible pilot for testing whether GRPO fine-tuning of Llasa speech synthesis reduces prosodic expressiveness while improving intelligibility. The workflow selects LibriTTS evaluation utterances, probes checkpoint and prompt behavior, synthesizes matched base/GRPO speech, measures CER and word timing, then evaluates prosody and prepares blind listening pairs. The graph also preserves the separate Elise speaker-drift reference branch. Stage ordering and several subsystem links are documented; source excerpts provide stronger evidence for module calls than for the overall orchestration.
Main entry points: synthesize_llasa.py Matched speech synthesis · diagnose_llasa.py Prompt diagnosis · eval_cer.py ASR and CER · m1_collapse.py Pilot metrics and verdict
Components 16
Evaluation data
- Evaluation manifest
scripts/build_eval_set.py - Elise recordings
scripts/fetch_elise.py
Model preparation
- Checkpoint smoke probe
src/smoke_llasa.py - Prompt diagnosis
src/diagnose_llasa.py - Matched speech synthesis
src/synthesize_llasa.py
Automated analysis
- ASR and CER
scripts/eval_cer.py - Whisper word timings
scripts/eval_cer.py - F0 and rate metrics
src/prosody_metrics.py - Pilot metrics and verdict
scripts/m1_collapse.py
Human review
- Blind A/B pairs
scripts/make_ab_pairs.py
Other components
- Researcher
- LibriTTS-R
- Llasa checkpoints
- XCodec2 codec
- Whisper + wav2vec2
- Human listener
Connections 29
- Researcher → Evaluation manifest: selects utterances
README.md - LibriTTS-R → Evaluation manifest: provides speech
scripts/build_eval_set.py - Researcher → Elise recordings: fetches reference
scripts/fetch_elise.py - Researcher → Checkpoint smoke probe: runs probe
README.md - Checkpoint smoke probe → Llasa checkpoints: loads checkpoints
src/smoke_llasa.py - Checkpoint smoke probe → XCodec2 codec: loads codec
src/smoke_llasa.py - Checkpoint smoke probe → Prompt diagnosis: tests prompted generation
src/diagnose_llasa.py - Prompt diagnosis → Llasa checkpoints: loads GRPO model
src/diagnose_llasa.py - Prompt diagnosis → XCodec2 codec: loads codec
src/diagnose_llasa.py - Prompt diagnosis → Whisper + wav2vec2: transcribes probes
src/diagnose_llasa.py - Prompt diagnosis → Matched speech synthesis: calls engine
src/diagnose_llasa.py - Researcher → Matched speech synthesis: runs synthesis
README.md - Evaluation manifest → Matched speech synthesis: supplies targets
README.md - Matched speech synthesis → Llasa checkpoints: generates speech
src/synthesize_llasa.py - Matched speech synthesis → XCodec2 codec: decodes speech
src/synthesize_llasa.py - Matched speech synthesis → Prompt diagnosis: reads recommendation
src/synthesize_llasa.py - Researcher → ASR and CER: runs evaluation
README.md - Matched speech synthesis → ASR and CER: provides audio
README.md - ASR and CER → Whisper + wav2vec2: transcribes audio
scripts/eval_cer.py - ASR and CER → Whisper word timings: writes timestamps
scripts/eval_cer.py - ASR and CER → F0 and rate metrics: excludes runaways
scripts/eval_cer.py - Researcher → Pilot metrics and verdict: runs analysis
README.md - Matched speech synthesis → Pilot metrics and verdict: provides generations
scripts/m1_collapse.py - Elise recordings → Pilot metrics and verdict: provides reference audio
scripts/m1_collapse.py - Whisper word timings → Pilot metrics and verdict: provides word intervals
scripts/m1_collapse.py - F0 and rate metrics → Pilot metrics and verdict: computes prosody
scripts/m1_collapse.py - Researcher → Blind A/B pairs: builds listening set
README.md - Matched speech synthesis → Blind A/B pairs: provides matched audio
scripts/make_ab_pairs.py - Blind A/B pairs → Human listener: presents blinded pairs
scripts/make_ab_pairs.py