How sahahahil/shruti---avsr-in-nepali-language- works
Shruti is a Nepali audio-visual speech-recognition application: users submit video or use live audio/video capture to obtain a transcription. The sampled backend shows HTTP AVSR processing and a WebSocket path for streaming input, with inference and configurable audio/video fusion. The README provides little architectural detail, and several frontend pages and backend routes are unsampled, so uncertain component wiring is omitted.
Main entry points: inference.py AVSR inference · offline/page.tsx Offline page · RealtimeAVSR.tsx Live AVSR · realtime/page.tsx Realtime page
Components 16
User experience
- Offline page
frontend/nepali-avsr-frontend/src/app/offline/page.tsx - File uploader
frontend/nepali-avsr-frontend/src/components/offline/FileUploader.tsx - Output display
frontend/nepali-avsr-frontend/src/components/offline/OutputDisplay.tsx - Realtime page
frontend/nepali-avsr-frontend/src/app/realtime/page.tsx - Live AVSR
frontend/nepali-avsr-frontend/src/components/RealtimeAVSR.tsx - Webcam capture
frontend/nepali-avsr-frontend/src/components/RealtimeWebcam.tsx - Audio capture
frontend/nepali-avsr-frontend/src/components/RealtimeAudio.tsx - API client
frontend/nepali-avsr-frontend/src/lib/api-client.ts
Recognition service
- HTTP and socket API
backend/app.py
Inference
- AVSR inference
backend/inference.py - Video preprocessing
backend/utils/preprocess_video.py - Audio extraction
backend/utils/extract_audio.py - Frame extraction
backend/utils/extract_frames.py - Audio preprocessing
backend/utils/preprocess_audio.py - CTC decoding
backend/utils/decode_ctc.py
Other components
- User
Connections 14
- User → Offline page: opens
- User → Realtime page: opens
- Offline page → File uploader: offers upload
- Offline page → Output display: shows output
- Realtime page → Live AVSR: presents
- API client → HTTP and socket API: sends AVSR request
- HTTP and socket API → AVSR inference: runs inference
- AVSR inference → Video preprocessing: preprocesses frames
- AVSR inference → Audio extraction: extracts audio
- AVSR inference → Frame extraction: extracts frames
- AVSR inference → Audio preprocessing: prepares audio
- AVSR inference → CTC decoding: decodes output
- Webcam capture → Live AVSR: provides video
- Audio capture → Live AVSR: provides audio