How mawwalker/stt-server works
Project Type and Purpose
• This is a C++ WebSocket-based streaming ASR (Automatic Speech Recognition) server.
• It exposes two endpoints (/sttRealtime for streaming, /oneshot for one-shot recognition), manages WebSocket connections, runs voice‐activity detection (VAD), and invokes the sherpa-onnx inference engine on pre-loaded ONNX models.
• It is primarily a backend service (open-source tool) packaged for local or Docker deployment.
File Structure Analysis
• Top-level: main.cpp (entry), build scripts (build.sh, Makefile, CMakeLists.txt), Docker files, .env support.
• include/ & src/: Core components:
• asr_engine.* — model loading & inference orchestration
• oneshot_asr_session.* & asr_session.* — session state machine for streaming vs one-shot
• websocket_server.* & connection_manager — WebSocket acceptor and per-client routing
• model_pool.* — a pool of loaded ASR/VAD models
• logger.* — unified logging
• Deployment/config: docker-compose.yml, Dockerfile, .env.example
• Utilities: install_sherpa_onnx*.sh, setup_env.sh, sherpa_config.sh
Key Insights from README
• Unified configuration: .env file drives both local and Docker behavior.
• Processing flow: WebSocket connect → ASRSession → VAD → partial/final inference → send JSON.
a. Main Components to Include
• WebSocket Clients (Python, JS, etc.)
• WebSocket ASR Server
• Connection Manager
• Session Factory (ASRSession for streaming, OneShotASRSession)
• Logger
• ASREngine / ModelPool
• sherpa-onnx Inference Engine
• Silero VAD Engine
• Asset Store (ONNX Model files on disk)
b. Relationships & Data Flows
1. Client opens WS → WebSocket Server (Listener)
2. Server creates an ASRSession instance via Connection Manager
3. ASRSession receives PCM audio frames → forwards to VAD in ASREngine
4. VAD marks speech segments → ASREngine invokes sherpa-onnx recognizer
5. Partial/final results → ASRSession packages JSON → sends back over WS
6. ModelPool holds pre-loaded ONNX models for reuse across sessions
c. Architectural Patterns & Principles
• Modular separation: Networking (WebSocket) vs Session logic vs Inference vs I/O
• Resource pooling: ModelPool to avoid repeated model loads
• Single responsibility: Each class (Engine, Session, Server, Logger) has a clear domain
• Configuration abstraction: .env + server_config handle environment detection
d. Technologies & Frameworks
• C++17, WebSocket++ or asio for networking
• sherpa-onnx C++ API for ASR
• Silero VAD ONNX model
• CMake, Makefile, Docker