# chaituviking/prosodyrlvr architecture

> Architecture diagram of the GitHub repository chaituviking/prosodyrlvr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chaituviking/prosodyrlvr
- Repository: https://github.com/chaituviking/prosodyrlvr

## Overview

ProsodyRLVR is a reproducible pilot for testing whether GRPO fine-tuning of Llasa speech synthesis reduces prosodic expressiveness while improving intelligibility. The workflow selects LibriTTS evaluation utterances, probes checkpoint and prompt behavior, synthesizes matched base/GRPO speech, measures CER and word timing, then evaluates prosody and prepares blind listening pairs. The graph also preserves the separate Elise speaker-drift reference branch. Stage ordering and several subsystem links are documented; source excerpts provide stronger evidence for module calls than for the overall orchestration.

Main entry points: [`src/synthesize_llasa.py`](https://github.com/chaituviking/prosodyrlvr/blob/main/src/synthesize_llasa.py) (Matched speech synthesis), [`src/diagnose_llasa.py`](https://github.com/chaituviking/prosodyrlvr/blob/main/src/diagnose_llasa.py) (Prompt diagnosis), [`scripts/eval_cer.py`](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/eval_cer.py) (ASR and CER), [`scripts/m1_collapse.py`](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/m1_collapse.py) (Pilot metrics and verdict)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Evaluation data"]
  node_evalset["Evaluation manifest<br/>[build_eval_set.py]"]
  node_elise["Elise recordings<br/>[fetch_elise.py]"]
end

subgraph group_generation["Model preparation"]
  node_smoke["Checkpoint smoke probe<br/>[smoke_llasa.py]"]
  node_diagnose["Prompt diagnosis<br/>[diagnose_llasa.py]"]
  node_synth["Matched speech synthesis"]
end

subgraph group_analysis["Automated analysis"]
  node_cer["ASR and CER<br/>[eval_cer.py]"]
  node_wordtimes["Whisper word timings<br/>[eval_cer.py]"]
  node_metrics["F0 and rate metrics<br/>[prosody_metrics.py]"]
  node_verdict["Pilot metrics and verdict<br/>[m1_collapse.py]"]
end

subgraph group_review["Human review"]
  node_pairs["Blind A/B pairs<br/>[make_ab_pairs.py]"]
end

node_operator(("Researcher"))
node_libritts[("LibriTTS-R")]
node_llasa["Llasa checkpoints"]
node_codec["XCodec2 codec"]
node_asr["Whisper + wav2vec2"]
node_listener(("Human listener"))

node_operator -->|"selects utterances"| node_evalset
node_libritts -->|"provides speech"| node_evalset
node_operator -->|"fetches reference"| node_elise
node_operator -->|"runs probe"| node_smoke
node_smoke -->|"loads checkpoints"| node_llasa
node_smoke -->|"loads codec"| node_codec
node_smoke -.->|"tests prompted generation"| node_diagnose
node_diagnose -->|"loads GRPO model"| node_llasa
node_diagnose -->|"loads codec"| node_codec
node_diagnose -->|"transcribes probes"| node_asr
node_diagnose -->|"calls engine"| node_synth
node_operator -->|"runs synthesis"| node_synth
node_evalset -->|"supplies targets"| node_synth
node_synth -->|"generates speech"| node_llasa
node_synth -->|"decodes speech"| node_codec
node_synth -->|"reads recommendation"| node_diagnose
node_operator -->|"runs evaluation"| node_cer
node_synth -->|"provides audio"| node_cer
node_cer -->|"transcribes audio"| node_asr
node_cer -->|"writes timestamps"| node_wordtimes
node_cer -->|"excludes runaways"| node_metrics
node_operator -->|"runs analysis"| node_verdict
node_synth -->|"provides generations"| node_verdict
node_elise -->|"provides reference audio"| node_verdict
node_wordtimes -->|"provides word intervals"| node_verdict
node_metrics -->|"computes prosody"| node_verdict
node_operator -->|"builds listening set"| node_pairs
node_synth -->|"provides matched audio"| node_pairs
node_pairs -->|"presents blinded pairs"| node_listener

click node_evalset "https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/build_eval_set.py"
click node_elise "https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/fetch_elise.py"
click node_smoke "https://github.com/chaituviking/prosodyrlvr/blob/main/src/smoke_llasa.py"
click node_diagnose "https://github.com/chaituviking/prosodyrlvr/blob/main/src/diagnose_llasa.py"
click node_synth "https://github.com/chaituviking/prosodyrlvr/blob/main/src/synthesize_llasa.py"
click node_cer "https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/eval_cer.py"
click node_wordtimes "https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/eval_cer.py"
click node_metrics "https://github.com/chaituviking/prosodyrlvr/blob/main/src/prosody_metrics.py"
click node_verdict "https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/m1_collapse.py"
click node_pairs "https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/make_ab_pairs.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_evalset,node_elise toneBlue
class node_smoke,node_diagnose,node_synth,node_libritts toneAmber
class node_cer,node_wordtimes,node_metrics,node_verdict toneMint
class node_pairs toneRose
class node_operator,node_llasa,node_codec,node_asr,node_listener toneIndigo
```

## Components

### Evaluation data

- [Evaluation manifest](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/build_eval_set.py): `scripts/build_eval_set.py`
- [Elise recordings](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/fetch_elise.py): `scripts/fetch_elise.py`

### Model preparation

- [Checkpoint smoke probe](https://github.com/chaituviking/prosodyrlvr/blob/main/src/smoke_llasa.py): `src/smoke_llasa.py`
- [Prompt diagnosis](https://github.com/chaituviking/prosodyrlvr/blob/main/src/diagnose_llasa.py): `src/diagnose_llasa.py`
- [Matched speech synthesis](https://github.com/chaituviking/prosodyrlvr/blob/main/src/synthesize_llasa.py): `src/synthesize_llasa.py`

### Automated analysis

- [ASR and CER](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/eval_cer.py): `scripts/eval_cer.py`
- [Whisper word timings](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/eval_cer.py): `scripts/eval_cer.py`
- [F0 and rate metrics](https://github.com/chaituviking/prosodyrlvr/blob/main/src/prosody_metrics.py): `src/prosody_metrics.py`
- [Pilot metrics and verdict](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/m1_collapse.py): `scripts/m1_collapse.py`

### Human review

- [Blind A/B pairs](https://github.com/chaituviking/prosodyrlvr/blob/main/scripts/make_ab_pairs.py): `scripts/make_ab_pairs.py`

### Other components

- **Researcher**
- **LibriTTS-R**
- **Llasa checkpoints**
- **XCodec2 codec**
- **Whisper + wav2vec2**
- **Human listener**

## Connections

- Researcher → Evaluation manifest: selects utterances (evidence: `README.md`)
- LibriTTS-R → Evaluation manifest: provides speech (evidence: `scripts/build_eval_set.py`)
- Researcher → Elise recordings: fetches reference (evidence: `scripts/fetch_elise.py`)
- Researcher → Checkpoint smoke probe: runs probe (evidence: `README.md`)
- Checkpoint smoke probe → Llasa checkpoints: loads checkpoints (evidence: `src/smoke_llasa.py`)
- Checkpoint smoke probe → XCodec2 codec: loads codec (evidence: `src/smoke_llasa.py`)
- Checkpoint smoke probe → Prompt diagnosis: tests prompted generation (evidence: `src/diagnose_llasa.py`)
- Prompt diagnosis → Llasa checkpoints: loads GRPO model (evidence: `src/diagnose_llasa.py`)
- Prompt diagnosis → XCodec2 codec: loads codec (evidence: `src/diagnose_llasa.py`)
- Prompt diagnosis → Whisper + wav2vec2: transcribes probes (evidence: `src/diagnose_llasa.py`)
- Prompt diagnosis → Matched speech synthesis: calls engine (evidence: `src/diagnose_llasa.py`)
- Researcher → Matched speech synthesis: runs synthesis (evidence: `README.md`)
- Evaluation manifest → Matched speech synthesis: supplies targets (evidence: `README.md`)
- Matched speech synthesis → Llasa checkpoints: generates speech (evidence: `src/synthesize_llasa.py`)
- Matched speech synthesis → XCodec2 codec: decodes speech (evidence: `src/synthesize_llasa.py`)
- Matched speech synthesis → Prompt diagnosis: reads recommendation (evidence: `src/synthesize_llasa.py`)
- Researcher → ASR and CER: runs evaluation (evidence: `README.md`)
- Matched speech synthesis → ASR and CER: provides audio (evidence: `README.md`)
- ASR and CER → Whisper + wav2vec2: transcribes audio (evidence: `scripts/eval_cer.py`)
- ASR and CER → Whisper word timings: writes timestamps (evidence: `scripts/eval_cer.py`)
- ASR and CER → F0 and rate metrics: excludes runaways (evidence: `scripts/eval_cer.py`)
- Researcher → Pilot metrics and verdict: runs analysis (evidence: `README.md`)
- Matched speech synthesis → Pilot metrics and verdict: provides generations (evidence: `scripts/m1_collapse.py`)
- Elise recordings → Pilot metrics and verdict: provides reference audio (evidence: `scripts/m1_collapse.py`)
- Whisper word timings → Pilot metrics and verdict: provides word intervals (evidence: `scripts/m1_collapse.py`)
- F0 and rate metrics → Pilot metrics and verdict: computes prosody (evidence: `scripts/m1_collapse.py`)
- Researcher → Blind A/B pairs: builds listening set (evidence: `README.md`)
- Matched speech synthesis → Blind A/B pairs: provides matched audio (evidence: `scripts/make_ab_pairs.py`)
- Blind A/B pairs → Human listener: presents blinded pairs (evidence: `scripts/make_ab_pairs.py`)
