How pengzhendong/streaming-sensevoice works
Project type and purpose
• This is a Python-based streaming speech-to-text inference application and demo suite built around the open-source SenseVoice model.
• It provides three primary modes:
• Batch transcription of WAV files (main.py)
• Real-time microphone transcription (realtime.py)
• Real-time transcription over WebSocket (realtime_ws_server_demo.py + realtime_ws_client.html)
File structure
• Top level:
• main.py – batch transcription CLI
• realtime.py – microphone streaming CLI
• realtime_ws_server_demo.py – FastAPI WebSocket server demo
• realtime_ws_client.html – browser client for WebSocket demo
• streaming_sensevoice/ – core Python package
• data/, images/ – sample inputs and screenshots
• streaming_sensevoice package:
• sensevoice.py – wrapper around the SenseVoice model (loading, single-chunk inference)
• streaming_sensevoice.py – orchestrates chunked inference, buffering, streaming callbacks
README for additional insights
• Describes usage scenarios and dependencies:
• requirements.txt for core inference
• requirements-ws-demo.txt for FastAPI/WebSocket demo
• Mentions external Recorder library for capturing MP3 chunks in the browser
a. Main components
1. Client input sources
– WAV file loader
– Microphone capture (PyAudio or sounddevice)
– WebSocket frontend (Recorder-powered browser page)
2. Application entry points
– main.py (batch)
– realtime.py (mic streaming)
– FastAPI WebSocket server (realtime_ws_server_demo.py)
3. Core library
– streaming_sensevoice.StreamingSenseVoice
– streaming_sensevoice.SenseVoiceWrapper
4. External model
– SenseVoice inference engine (loaded via sensevoice.py)
5. Output sinks
– Console (stdout)
– WebSocket response messages
b. Relationships / data flow
• Client source ➔ Application entry point ➔ StreamingSenseVoice handler ➔ SenseVoiceWrapper (chunk inference) ➔ results streamed back to entry point ➔ output sink
• In WebSocket mode: Browser client ➔ WebSocket server ➔ StreamingSenseVoice ➔ server sends back transcript frames ➔ Browser UI
c. Architectural patterns
• Modular separation: CLI vs. streaming library vs. WebSocket service
• Streaming pipeline: chunk buffer ➔ model inference ➔ callback
• Single-process, event-driven I/O for WebSocket demo (FastAPI + uvicorn)
d. Key technologies / libraries
• Python 3.x
• SenseVoice (external model)
• FastAPI (WebSocket server)
• Recorder.js (frontend MP3 capture)
• sounddevice or PyAudio (mic capture)
• asyncio or threading for chunk dispatch
Model structure (internal package focus)
• sensevoice.py: SenseVoiceWrapper – loads model, exposes infer_chunk()
• streaming_sensevoice.py: StreamingSenseVoice – manages chunk queue, streaming loop, event callbacks