How manifestro/aqyn works
Aqyn is a research speech-model project whose implemented end-to-end product is Stage 1 streaming text-to-speech: prepare speech and word alignments as Mimi-token data, train the frame-rate model, then synthesize or evaluate audio. Its model combines a sliding text window, configurable temporal backbones and within-frame codebook prediction. The README also describes future hearing/turn-taking and a slow LLM cortex, but these are planned rather than implemented, so they are not shown as runtime components. Relationships are based on the README and SOURCE INDEX references; unsampled internals are not inferred.
Main entry points: train.py Training loop · cli.py Aqyn CLI · prepare.py Corpus preparation · evaluate.py Speech evaluation
Components 17
Data preparation
- Corpus preparation
src/aqyn/prepare.py - Text normalization
src/aqyn/text.py - Word alignment
src/aqyn/align.py - Mimi codec
src/aqyn/codec.py - Token datasets
src/aqyn/data.py
Speech model
- Frame-rate TTS
src/aqyn/models/tts.py - Backbone blocks
src/aqyn/models/blocks.py - CfC recurrence
src/aqyn/models/cfc.py
Training
- Experiment config
src/aqyn/config.py - Training loop
src/aqyn/train.py - Model checkpoints
src/aqyn/checkpoint.py
Inference and evaluation
- Audio synthesis
src/aqyn/synthesize.py - Speech evaluation
src/aqyn/evaluate.py - ASR service
- Model benchmark
src/aqyn/bench.py
Other components
- Researcher
- Aqyn CLI
src/aqyn/cli.py
Connections 23
- Researcher → Aqyn CLI: runs commands
README.md - Aqyn CLI → Corpus preparation: dispatches prepare
README.md - Aqyn CLI → Training loop: dispatches train
README.md - Aqyn CLI → Audio synthesis: dispatches synth
README.md - Aqyn CLI → Speech evaluation: dispatches eval
README.md - Aqyn CLI → Model benchmark: dispatches bench
README.md - Corpus preparation → Text normalization: builds vocabulary
src/aqyn/prepare.py - Corpus preparation → Word alignment: aligns transcripts
src/aqyn/prepare.py - Corpus preparation → Mimi codec: encodes audio
src/aqyn/prepare.py - Corpus preparation → Token datasets: writes token data
README.md - Training loop → Experiment config: loads settings
src/aqyn/train.py - Training loop → Token datasets: loads batches
src/aqyn/train.py - Training loop → Frame-rate TTS: trains model
src/aqyn/train.py - Training loop → Model checkpoints: saves state
src/aqyn/train.py - Training loop → Mimi codec: decodes samples
src/aqyn/train.py - Frame-rate TTS → Backbone blocks: uses blocks
src/aqyn/models/tts.py - Backbone blocks → CfC recurrence: uses CfC mixer
src/aqyn/models/blocks.py - Audio synthesis → Model checkpoints: loads model
src/aqyn/synthesize.py - Audio synthesis → Mimi codec: decodes tokens
src/aqyn/synthesize.py - Speech evaluation → Model checkpoints: loads model
src/aqyn/evaluate.py - Speech evaluation → Token datasets: reads held-out data
src/aqyn/evaluate.py - Speech evaluation → Mimi codec: decodes audio
src/aqyn/evaluate.py - Speech evaluation → ASR service: transcribes audio
src/aqyn/evaluate.py