How kartikeyagrawal2007/punarvaani works
Punarvaani is a Hindi visual-speech recognition system: silent video is cropped to mouth frames, a CTC vision model predicts Hindi phonemes, and a Sarvam language model reconstructs Devanagari text. Its runtime entry points support inference as well as dataset training and evaluation; data preparation, synthetic-data generation, and readiness checks are additional product capabilities. The graph combines sampled source references with README-described stages; unsampled model and preprocessing internals are not asserted beyond those documented relationships.
Main entry points: inference.py Inference API · main.py CLI workflow · dataset.py Video dataset · face_cropper.py Face and mouth cropper
Components 16
Runtime and orchestration
- CLI workflow
main.py - Inference API
inference.py - HPI configuration
config.py
Video and visual recognition
- Face and mouth cropper
face_cropper.py - Mouth geometry
mouth_geometry.py - Batch lip preprocessing
scripts/preprocess_lips.py - ViT CTC recognizer
Models/ML_VALLR.py - Classic VALLR reference
Models/VALLR.py
Hindi language processing
- Sarvam translator
Models/Sarvam.py - Hindi G2P
Data/hindi_g2p.py
Data and readiness
- Video dataset
Data/dataset.py - Evaluation metrics
Data/metrics.py - Synthetic dataset generator
Data/synthetic.py - Dataset and adapter checks
scripts/preflight.py
Other components
- Video user
- Hindi transcript
Connections 19
- Video user → Inference API: submits video
README.md - Inference API → Face and mouth cropper: extracts mouth frames
inference.py - Face and mouth cropper → Mouth geometry: computes crop boxes
face_cropper.py - Inference API → ViT CTC recognizer: runs recognition
inference.py - ViT CTC recognizer → Sarvam translator: passes phonemes
README.md - Inference API → Sarvam translator: translates phonemes
inference.py - Inference API → HPI configuration: loads Hindi inventory
inference.py - Inference API → Hindi transcript: returns transcription
README.md - CLI workflow → Video dataset: builds datasets
main.py - CLI workflow → ViT CTC recognizer: trains or evaluates
main.py - CLI workflow → Evaluation metrics: computes error rates
main.py - Video dataset → Hindi G2P: converts transcripts
Data/dataset.py - Video dataset → Face and mouth cropper: uses frame cropping
Data/dataset.py - Batch lip preprocessing → Face and mouth cropper: uses cropper
scripts/preprocess_lips.py - Synthetic dataset generator → Video dataset: writes dataset format
Data/synthetic.py - Dataset and adapter checks → Hindi G2P: checks phoneme handling
scripts/preflight.py - Dataset and adapter checks → Sarvam translator: checks adapter
scripts/preflight.py - CLI workflow → HPI configuration: loads phoneme inventory
main.py - Sarvam translator → Hindi transcript: reconstructs Hindi text
README.md