How kizuna-ai-lab/sokuji works
Sokuji is a desktop and browser-extension app for real-time, two-way speech translation, with cloud and fully offline modes. The main workflow captures microphone and meeting/system audio, processes speech through provider-backed or local ASR → translation → TTS, then presents subtitles and routes translated audio for others to hear. Major documented capabilities include provider selection, local model management, session display and extension/desktop use. The README documents the product flow; sampled source confirms local engine and model-management details, but not every runtime connection.
Main entry points: MainPanel.tsx Translation session · LocalInferenceClient.ts Local inference client · modelStore.ts Model store · ClientFactory.ts Client factory
Components 24
Session experience
- Translation session
src/components/MainPanel/MainPanel.tsx - Live subtitles
src/components/Subtitle/SubtitleApp.tsx - Conversation display
src/stores/conversationDisplayStore.ts - Provider settings
src/components/Settings/Settings.tsx
Audio capture and output
- Audio capture
src/lib/modern-audio/ModernBrowserAudioService.ts - System audio
electron/audio-host.js - Meeting audio
extension/content/content.js - Translated audio
src/lib/modern-audio/ModernAudioPlayer.js - Virtual microphone
extension/content/virtual-microphone.js
Speech processing
- Client operations
src/services/ClientOperations.ts - Client factory
src/services/clients/ClientFactory.ts - Cloud provider clients
src/services/clients/index.ts
Local inference
- Local inference client
src/services/clients/LocalInferenceClient.ts - Speech recognition
src/lib/local-inference/engine/AsrEngine.ts - Text translation
src/lib/local-inference/engine/TranslationEngine.ts - Speech synthesis
src/lib/local-inference/engine/TtsEngine.ts - Local model management
src/lib/local-inference/ModelManager.ts - Model store
src/stores/modelStore.ts - Native inference sidecar
sidecar/sokuji_sidecar/server.py
Desktop and extension
- Desktop shell
electron/main.js - Browser extension
extension/background/background.js - Subtitle surfaces
src/components/Subtitle/surfaces/getSubtitleSurface.ts
Other components
- Meeting participants
- Cloud speech services
Connections 31
- Meeting participants → Audio capture: speaks into
- Meeting participants → System audio: meeting audio
- Meeting participants → Meeting audio: meeting audio
- Audio capture → Translation session: audio input
- System audio → Translation session: audio input
- Meeting audio → Translation session: audio input
- Provider settings → Translation session: configures
- Translation session → Client operations: starts session
- Client operations → Client factory: creates client
- Client factory → Cloud provider clients: selects provider
- Cloud provider clients → Cloud speech services: sends audio
- Cloud speech services → Cloud provider clients: returns speech
- Client factory → Local inference client: selects local
- Local inference client → Speech recognition: transcribes
- Local inference client → Text translation: translates text
- Local inference client → Speech synthesis: synthesizes speech
- Speech recognition → Local inference client: transcript
- Text translation → Local inference client: translated text
- Speech synthesis → Local inference client: audio output
- Local model management → Model store: manages models
- Model store → Speech recognition: provides models
- Model store → Text translation: provides models
- Model store → Speech synthesis: provides models
- Local inference client → Native inference sidecar: uses native engine
- Translation session → Conversation display: updates display
- Translation session → Live subtitles: updates subtitles
- Translation session → Translated audio: plays translated audio
- Translated audio → Virtual microphone: routes audio
- Desktop shell → System audio: hosts capture
- Browser extension → Meeting audio: hosts capture
- Live subtitles → Subtitle surfaces: renders on