How lavanyamahalle14/lavanya_tts works
Project Type and Purpose
• This is a full-stack, multilingual Text-to-Speech (TTS) web application built on Flask (Python) that integrates FastSpeech2 and HiFi-GAN models for synthetic speech across many Indian languages. It provides a web UI for users to input text, a REST API for inference, and serves generated audio via static hosting.
High-Level File Structure & Architectural Clues
• Top-level:
• app.py, gunicorn_config.py, Dockerfile, render.yaml, Procfile → Deployment & entry-point for the Flask web server
• static/, templates/ → Frontend assets (HTML, CSS, static audio directory)
• Fastspeech2_HS/ → Core TTS engine with subfolders:
– api.py, app.py → Flask blueprint or microservice serving inference endpoints
– inference.py, text_preprocess_for_inference.py, get_phone_mapped_python.py → Orchestration scripts for preprocessing and running TTS
– phone_dict/, charmap/, ssn_parser/, ssn_parser_new/, unified_parser/ → Phoneme-mapping and text normalization pipelines
– [language]/[gender]/model/ → Pretrained FastSpeech2 model files and stats
– hifigan/ → HiFi-GAN vocoder modules
• README details: environment setup, dependency installation, local run steps, Docker & cloud deployments
Main Architectural Components
a. Presentation Layer (Frontend)
• Web Browser
• Static Server (Flask static/ directory) hosting index.html and generated audio files
b. Application Layer (Flask Web App)
• REST API endpoints (app.py/api.py at root and in Fastspeech2_HS/)
• Request router → gunicorn workers
• CORS handling
c. Processing Layer
• Text Preprocessing Module
• Character mapping (charmap/Text_Cleaning.py, multilingualcharmap.json)
• Phoneme Parser (ssn_parser & unified_parser, phoneReplace.py)
• Number-to-words converter (num_to_words.py)
• TTS Inference Module
• FastSpeech2 model loader (config.yaml, model.pth)
• Mel-spectrogram generator (inference.py)
• Vocoder Module
• HiFi-GAN generator (hifigan/models.py, generator checkpoints)
d. Model & Data Storage
• File system directories for model binaries (.pth, .npz stats) tracked via Git LFS
• Phone dictionaries and charmaps
• Temporary output directories (Fastspeech2_HS/tmp, static/audio)
Component Interactions & Data Flow
1. User enters text in browser → HTTP POST to Flask API
2. Flask API receives JSON payload → passes text to Text Preprocessing Module
3. Preprocessing outputs normalized phoneme sequence → passed to FastSpeech2 Inference
4. FastSpeech2 Inference generates mel-spectrogram → sent to HiFi-GAN Vocoder
5. Vocoder returns raw waveform → saved to static/audio/<filename>.wav
6. Flask responds with URL to generated audio → front end plays audio
Architectural Patterns & Principles
• Layered Architecture: clearly separate Presentation, Application, Processing, and Storage
• Modular Design: text-preprocessing, inference, and vocoder are independent modules
• Singleton Model Loading: load models once at application start, reuse across requests
• Configuration-driven: config.yaml and environment variables (e.g., TTS_MODEL_ROOT) locate model files
Final Checklist
• [ ] Clearly labeled components with names matching code (e.g., Fastspeech2_HS/inference.py)
• [ ] Annotations for deployment context (Docker, cloud)
• [ ] Enough granularity to show tripartite split: Preprocessor → FastSpeech2 → HiFi-GAN within the TTS Engine