# pengzhendong/streaming-sensevoice architecture

> Architecture diagram of the GitHub repository pengzhendong/streaming-sensevoice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pengzhendong/streaming-sensevoice
- Repository: https://github.com/pengzhendong/streaming-sensevoice

## Overview

##### Project type and purpose
- This is a Python-based streaming speech-to-text inference application and demo suite built around the open-source SenseVoice model.
- It provides three primary modes:
- Batch transcription of WAV files (main.py)
- Real-time microphone transcription (realtime.py)
- Real-time transcription over WebSocket (realtime_ws_server_demo.py + realtime_ws_client.html)

##### File structure
- Top level:
- main.py – batch transcription CLI
- realtime.py – microphone streaming CLI
- realtime_ws_server_demo.py – FastAPI WebSocket server demo
- realtime_ws_client.html – browser client for WebSocket demo
- streaming_sensevoice/ – core Python package
- data/, images/ – sample inputs and screenshots
- streaming_sensevoice package:
- sensevoice.py – wrapper around the SenseVoice model (loading, single-chunk inference)
- streaming_sensevoice.py – orchestrates chunked inference, buffering, streaming callbacks

##### README for additional insights
- Describes usage scenarios and dependencies:
- requirements.txt for core inference
- requirements-ws-demo.txt for FastAPI/WebSocket demo
- Mentions external Recorder library for capturing MP3 chunks in the browser

a. Main components
1. Client input sources
– WAV file loader
– Microphone capture (PyAudio or sounddevice)
– WebSocket frontend (Recorder-powered browser page)
2. Application entry points
– main.py (batch)
– realtime.py (mic streaming)
– FastAPI WebSocket server (realtime_ws_server_demo.py)
3. Core library
– streaming_sensevoice.StreamingSenseVoice
– streaming_sensevoice.SenseVoiceWrapper
4. External model
– SenseVoice inference engine (loaded via sensevoice.py)
5. Output sinks
– Console (stdout)
– WebSocket response messages
b. Relationships / data flow
- Client source ➔ Application entry point ➔ StreamingSenseVoice handler ➔ SenseVoiceWrapper (chunk inference) ➔ results streamed back to entry point ➔ output sink
- In WebSocket mode: Browser client ➔ WebSocket server ➔ StreamingSenseVoice ➔ server sends back transcript frames ➔ Browser UI
c. Architectural patterns
- Modular separation: CLI vs. streaming library vs. WebSocket service
- Streaming pipeline: chunk buffer ➔ model inference ➔ callback
- Single-process, event-driven I/O for WebSocket demo (FastAPI + uvicorn)
d. Key technologies / libraries
- Python 3.x
- SenseVoice (external model)
- FastAPI (WebSocket server)
- Recorder.js (frontend MP3 capture)
- sounddevice or PyAudio (mic capture)
- asyncio or threading for chunk dispatch

##### Model structure (internal package focus)
- sensevoice.py: SenseVoiceWrapper – loads model, exposes infer_chunk()
- streaming_sensevoice.py: StreamingSenseVoice – manages chunk queue, streaming loop, event callbacks

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Input Sources
    subgraph "Client Input Sources"
        direction TB
        WAV["WAV File Loader"]:::browser
        MIC["Microphone Capture"]:::browser
        BROWSER["Browser Client (Recorder.js)"]:::browser
    end

    %% Application Entry Points
    subgraph "Application Entry Points"
        direction TB
        MAIN["Batch Transcription CLI\n(main.py)"]:::cli
        REALTIME["Real-time Mic CLI\n(realtime.py)"]:::cli
        WS_SERVER["FastAPI WebSocket Server\n(realtime_ws_server_demo.py)"]:::infra
    end

    %% Core Library
    subgraph "Core Library (streaming_sensevoice/)" 
        direction TB
        SSV["StreamingSenseVoice\n(streaming_sensevoice.py)"]:::lib
        SWR["SenseVoiceWrapper\n(sensevoice.py)"]:::lib
    end

    %% External Model
    MODEL["SenseVoice Model\n(Inference Engine)"]:::external

    %% Output Sinks
    subgraph "Output Sinks"
        direction TB
        CONSOLE["Console (stdout)"]:::infra
        WS_RESP["WebSocket Responses"]:::infra
        UI["Browser UI"]:::browser
    end

    %% Resources
    subgraph "Resources & Assets"
        direction TB
        DATA["Sample WAV Inputs\ndata/"]:::external
        IMAGES["Demo Screenshots\nimages/"]:::external
        REQ1["requirements.txt"]:::external
        REQ2["requirements-ws-demo.txt"]:::external
    end

    %% Data Flows - Batch CLI
    WAV -->|"load file"| MAIN
    MAIN -->|"stream chunks"| SSV
    SSV -->|"infer_chunk()"| SWR
    SWR -->|"invoke model"| MODEL
    MODEL -->|"transcript"| SWR
    SWR -->|"results"| SSV
    SSV -->|"print text"| MAIN
    MAIN -->|"stdout"| CONSOLE

    %% Data Flows - Mic CLI
    MIC -->|"capture audio"| REALTIME
    REALTIME -->|"stream chunks"| SSV

    %% Data Flows - WebSocket Demo
    BROWSER -->|"MP3 chunks"| WS_SERVER
    WS_SERVER -->|"stream chunks"| SSV
    SSV -->|"infer_chunk()"| SWR
    SWR -->|"invoke model"| MODEL
    MODEL -->|"transcript"| SWR
    SWR -->|"results"| SSV
    SSV -->|"send frames"| WS_SERVER
    WS_SERVER -->|"WebSocket messages"| WS_RESP
    WS_RESP -->|"display text"| UI

    %% Click Events
    click MAIN "https://github.com/pengzhendong/streaming-sensevoice/blob/master/main.py"
    click REALTIME "https://github.com/pengzhendong/streaming-sensevoice/blob/master/realtime.py"
    click WS_SERVER "https://github.com/pengzhendong/streaming-sensevoice/blob/master/realtime_ws_server_demo.py"
    click BROWSER "https://github.com/pengzhendong/streaming-sensevoice/blob/master/realtime_ws_client.html"
    click SSV "https://github.com/pengzhendong/streaming-sensevoice/tree/master/streaming_sensevoice/"
    click SWR "https://github.com/pengzhendong/streaming-sensevoice/blob/master/streaming_sensevoice/sensevoice.py"
    click SSV "https://github.com/pengzhendong/streaming-sensevoice/blob/master/streaming_sensevoice/streaming_sensevoice.py"
    click DATA "https://github.com/pengzhendong/streaming-sensevoice/tree/master/data/"
    click IMAGES "https://github.com/pengzhendong/streaming-sensevoice/tree/master/images/"
    click REQ1 "https://github.com/pengzhendong/streaming-sensevoice/blob/master/requirements.txt"
    click REQ2 "https://github.com/pengzhendong/streaming-sensevoice/blob/master/requirements-ws-demo.txt"

    %% Styles
    classDef cli fill:#D6EAF8,stroke:#2980B9,color:#1B4F72
    classDef browser fill:#D6EAF8,stroke:#2980B9,color:#1B4F72
    classDef lib fill:#D5F5E3,stroke:#27AE60,color:#145A32
    classDef infra fill:#FAD7A0,stroke:#D68910,color:#7E5109
    classDef external fill:#E8DAEF,stroke:#8E44AD,color:#4A235A
```
