# 0x5446/api4sensevoice architecture

> Architecture diagram of the GitHub repository 0x5446/api4sensevoice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/0x5446/api4sensevoice
- Repository: https://github.com/0x5446/api4sensevoice

## Overview

Step 1. Identify the Main Components
- API Server (server.py): This component exposes a REST API (/transcribe) for single sentence audio transcription. It involves receiving an audio file, performing pre-processing, handling VAD and performing ASR inference, and then returning JSON responses.
- WebSocket Server (server_wss.py): This component handles real-time streaming recognition. It listens on a different port and supports a WebSocket endpoint (/ws/transcribe). It also can enable speaker verification based on a query parameter (sv) and uses pre-registered speaker audio files.
- Speech Recognition / Processing Model (model.py): Although not fully detailed, this module likely encapsulates the ASR logic, VAD detection, and speaker verification logic. It represents the core business logic that processes audio input.
- Speaker Verification Resources (speaker directory): This folder contains sample speaker audio files (in the required format) used to verify the identity of speakers during streaming recognition.
- Client Testing Page (client_wss.html): This static HTML file serves as a testing ground for the WebSocket interface, enabling manual testing of streaming recognition.
- Dependencies & Configuration: Configuration and dependency management are handled via requirements.txt and external dependency instructions in the README. Additionally, there is use of FastAPI, Uvicorn, and FFmpeg (installed with conda).

Step 2. Outline Relationships and Interactions Between Components
- Client Interaction: Clients can interact with the system in two ways:
- REST API: Clients send audio files through a POST request to the /transcribe endpoint provided by server.py.
- WebSocket: Clients can use the client testing page or their own clients to connect to the WebSocket endpoint (/ws/transcribe) of server_wss.py using a secure WebSocket (wss://). This handles streaming PCM audio and returns JSON responses.
- Audio Processing Flow:
1. Audio input is received (either as a complete file or as streaming PCM).
2. VAD detection is triggered (enabled within the ASR engine, possibly via external models referenced in the README).
3. Speech is transcribed using the recognition model.
4. (If enabled) Speaker verification is applied by comparing live input against preset audio samples in the speaker directory.
- Internal Dependencies: Both servers leverage the same underlying ASR model (model.py) to perform speech recognition and processing.

Step 3. Highlight Architectural Patterns and Design Principles
- Separation of Concerns: The system clearly splits responsibilities between the REST API and the WebSocket server, allowing both single sentence and real-time streaming services to coexist.
- Modular Design: By isolating the ASR engine in model.py and externalizing configuration (via command-line arguments, config files, and dependency management in requirements.txt), the architecture supports ease of modification and extension.
- Use of Micro-services-like Approach: Although running within a single repository, separate server files indicate a logical separation similar to microservices for handling diverse client interactions.

- REST API Server (server.py)
- WebSocket Server (server_wss.py)
- ASR/VAD/Speaker Verification Engine (model.py)
- Speaker Repository (speaker folder)
- Labels and Descriptions:
- Additional parameters such as SSL (certfile/keyfile), port configurations, and command-line parameters.
- Visual Differentiation:

Because this project is essentially a speech recognition and streaming service application:
- Separation between REST API handling and real-time WebSocket processing.
- Security (SSL certificate handling) as a configuration parameter for both server components.
- A clear feedback loop from each server to clients (i.e., responses to HTTP requests and JSON responses along WebSocket).

Step 6. Practical Tips for Maximum Detail
- Component with its role (e.g., “ASR Engine – processes speech, applies VAD, and verifies speaker identity”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients
    subgraph "Clients"
        REST_Client["REST Client"]
        WS_Client["WebSocket Client (client_wss.html)"]
    end

    %% Servers
    subgraph "Servers"
        API_Server["API Server (/transcribe) (server.py)"]
        WS_Server["WebSocket Server (/ws/transcribe) (server_wss.py)"]
    end

    %% Core Processing
    subgraph "Core Processing"
        ASR_Engine["ASR/VAD/Speaker Verification Engine (model.py)"]
        SpeakerRepo["Speaker Repository (speaker)"]
    end

    %% Dependencies & Configuration
    subgraph "Dependencies & Config"
        Req["Dependencies (requirements.txt)"]
        Readme["Documentation (README.md)"]
    end

    %% Flow Connections
    REST_Client -->|"HTTP_POST"| API_Server
    API_Server -->|"ProcessesAudio"| ASR_Engine
    ASR_Engine -->|"ReturnsTranscription"| API_Server
    API_Server -->|"ResponseToClient"| REST_Client

    WS_Client -->|"WSConnect"| WS_Server
    WS_Server -->|"StreamsAudio"| ASR_Engine
    ASR_Engine -->|"ReturnsTranscription"| WS_Server
    WS_Server -->|"JSONResponse"| WS_Client
    WS_Server -->|"SpeakerVerify"| SpeakerRepo

    %% Dependencies Connections
    API_Server -.->|"Uses"| Req
    API_Server -.->|"Uses"| Readme
    WS_Server -.->|"Uses"| Req
    WS_Server -.->|"Uses"| Readme
    ASR_Engine -.->|"Uses"| Req
    ASR_Engine -.->|"Uses"| Readme

    %% Styling Classes
    class REST_Client,WS_Client client
    class API_Server,WS_Server server
    class ASR_Engine engine
    class SpeakerRepo resource
    class Req,Readme config

    %% Class Definitions with Colors
    classDef client fill:#9fdfbf,stroke:#333,stroke-width:2px;
    classDef server fill:#a3c4f3,stroke:#333,stroke-width:2px;
    classDef engine fill:#ffe599,stroke:#333,stroke-width:2px;
    classDef resource fill:#f4cccc,stroke:#d22,stroke-width:2px;
    classDef config fill:#f9cb9c,stroke:#d22,stroke-width:2px;

    %% Click Events
    click API_Server "https://github.com/0x5446/api4sensevoice/blob/main/server.py"
    click WS_Server "https://github.com/0x5446/api4sensevoice/blob/main/server_wss.py"
    click ASR_Engine "https://github.com/0x5446/api4sensevoice/blob/main/model.py"
    click SpeakerRepo "https://github.com/0x5446/api4sensevoice/tree/main/speaker/"
    click WS_Client "https://github.com/0x5446/api4sensevoice/blob/main/client_wss.html"
    click Req "https://github.com/0x5446/api4sensevoice/blob/main/requirements.txt"
    click Readme "https://github.com/0x5446/api4sensevoice/blob/main/README.md"
```
