How sidneyowallah/dholuo-tts works
Purpose and Entry Points
This repository builds and serves a POS-aware Dholuo text-to-speech system. Raw text is grammatically tagged, converted to tone-marked IPA, then synthesized by gender-specific VITS models. Primary runtime entry points are the FastAPI app in api/main.py, local pipeline CLI scripts such as train_vits.py, and the Next.js demo in web/app/page.tsx.
Linguistic Inference Pipeline
tagger.py loads the fine-tuned AfroXLMR POS model and emits (word, POS) pairs. phonemizer.py applies Dholuo grapheme rules, digraph handling, vowel normalization, and POS-driven tone injection; it can invoke the tagger for raw-text input. This boundary isolates linguistic normalization from acoustic inference.
Paths: tagger.py, phonemizer.py, generate_lexicon.py.
Dataset Preparation and Training
The offline pipeline transcribes DhoNam audio, cleans/splits records by gender, applies POS labels, creates IPA metadata, and trains separate VITS voices. POS training is independently managed from TTS training, allowing the tagger and acoustic checkpoints to evolve separately.
Paths: transcribe_audio.py, apply_pos_tags.py, create_vits_metadata.py, train_pos_local.py, train_vits.py.
Model and Artifact Lifecycle
Inference expects a POS model plus male/female TTS checkpoints. Model acquisition is designed to work from local artifacts or the Hugging Face Hub, avoiding model files in the repository. Training uses AfroXLMR (Davlan/afro-xlmr-large-76L) for tagging and VITS for acoustic generation; GPU training targets NVIDIA hardware.
Paths: api/models.py, api/inference.py, scripts/upload_models_to_hub.py.
Serving API
FastAPI initializes application configuration and routes, coordinates model loading, runs text → POS → IPA → waveform inference, and returns synthesized audio plus linguistic information for the UI. A cache layer prevents repeated expensive synthesis/model work. API concerns are kept under api/, separate from reusable NLP modules at the repository root.
Paths: api/main.py, api/routes.py, api/inference.py, api/cache.py, api/config.py.
Web Demo Client
The standalone Next.js frontend accepts text and selected voice, calls the API through a client abstraction, and renders playback plus linguistic/phonetic output. Its runtime boundary is an environment-configured API endpoint rather than direct Python/model access.
Paths: web/app/page.tsx, web/hooks/useTts.ts, web/lib/api.ts, web/components/LinguisticPanel.tsx.
Deployment and Infrastructure
Production deployment uses Docker Compose for the API, Redis cache, and Nginx reverse proxy. A separate Space-oriented Dockerfile supports Hugging Face hosting; the frontend is intended for Vercel deployment. External dependencies include Hugging Face Hub, Redis, espeak-ng, PyTorch/Coqui TTS, and Transformers.
Paths: docker/docker-compose.yml, docker/Dockerfile, docker/Dockerfile.space, docker/nginx.conf.
Main entry points: inference.py Synthesis orchestration · routes.py TTS routes · useTts.ts TTS client hook · main.py FastAPI app
Components 21
Offline training
Dataset, POS, and VITS training pipelines.
- DhoNam audio
- Transcription
transcribe_audio.py - POS training · Independently produces the POS model artifact.
train_pos_local.py - POS labeling
apply_pos_tags.py - VITS metadata
create_vits_metadata.py - Voice training · Trains separate male and female voices.
train_vits.py - Model publishing
scripts/upload_models_to_hub.py
Python API
Serving and inference boundary.
- FastAPI app
api/main.py - TTS routes
api/routes.py - Synthesis orchestration · Runs text through POS, IPA, and selected voice synthesis.
api/inference.py - Model loader · Loads local or Hub POS and VITS models.
api/models.py - Synthesis cache
api/cache.py
Web demo
Standalone Next.js client.
- Demo page
web/app/page.tsx - TTS client hook
web/hooks/useTts.ts - API client · Uses an environment-configured API endpoint.
web/lib/api.ts - Linguistic panel · Displays POS and phonetic output.
web/components/LinguisticPanel.tsx
Deployment
Container and hosting topology.
- Production stack · Runs API, Redis, and Nginx.
docker/docker-compose.yml - Space image
docker/Dockerfile.space
Other components
- POS and VITS artifacts · Local artifacts or Hugging Face Hub models.
- POS tagger · Emits word and POS pairs using fine-tuned AfroXLMR.
tagger.py - Tone IPA phonemizer · Applies grapheme rules and POS-driven tone injection.
phonemizer.py
Connections 23
- DhoNam audio → Transcription: transcribe
- Transcription → POS labeling: clean and split
- POS training → POS and VITS artifacts: trains POS model
- POS labeling → VITS metadata: labeled records
- VITS metadata → Voice training: IPA metadata
- Voice training → POS and VITS artifacts: trains voice checkpoints
- POS and VITS artifacts → Model publishing: publish
- POS and VITS artifacts → Model loader: load
- Model loader → POS tagger: POS model
- POS tagger → Tone IPA phonemizer: POS pairs
- FastAPI app → TTS routes: registers
- TTS routes → Synthesis cache: check/store
- TTS routes → Synthesis orchestration: synthesis request
- Synthesis orchestration → POS tagger: tag raw text
- Synthesis orchestration → Tone IPA phonemizer: generate tone IPA
- Synthesis orchestration → Model loader: select voice model
- Demo page → TTS client hook: submits text and voice
- TTS client hook → API client: request synthesis
- API client → TTS routes: HTTP API
- TTS client hook → Linguistic panel: render linguistic output
- Production stack → FastAPI app: hosts API
- Production stack → Synthesis cache: provides Redis
- Space image → FastAPI app: hosts API