How joru10/opencluely works
Project Type and Purpose
• Project Type: Full-stack web application (real-time interview assistant).
• Purpose: Captures microphone audio, performs low-latency ASR (speech-to-text) + speaker identification, sends transcript/context to an LLM to generate suggestions, displays suggestions in a browser UI.
• Key Indicators:
– Top-level asr/ package for all speech logic
– webapp/backend/ (FastAPI) and webapp/frontend/ (JS modules)
– Entry scripts: start-webapp.sh, vosk_asr_server_refactored.py
File Structure
• asr/
– server.py: WebSocket ASR endpoint
– vosk_recognizer.py: Vosk-based speech decoding
– speaker_identification.py: Voiceprint matching
– voice_training.py: Builds custom voice profile
• webapp/
– backend/: FastAPI app (app.py)
– frontend/: Vanilla JS UI (index.html + ES modules in voice/)
• Root-level helpers and scripts
– Dockerfiles (asr, frontend, backend)
– start-all-services.sh, start-webapp.sh
– Alternate ASR servers & test stubs (asr-stub/, mock_asr_server.py)
– “backend/” directory: utility scripts (whisper_transcriber.py, direct_asr.py, profile_manager.py)
• Config/deployment files
– .env, config.py, devcontainer.json
README for Additional Insights
• Project follows a modular architecture (ASR package, webapp, entry scripts)
• Two ASR implementations (refactored vs. compatibility) toggled via USE_REFACTORED
• Technology stack: Python3.9+, FastAPI, websockets, Vosk, vanilla JS, Web Audio API, optional LLM server
Main Components and Their Interactions
a. ASR Service
– Component: asr/server.py or vosk_asr_server_refactored.py
– Interfaces:
• WebSocket endpoint ← audio stream from browser
• Returns (timestamped) text + speaker ID
b. Backend API
– Component: webapp/backend/app.py (FastAPI)
– Responsibilities:
• Receives transcripts + speaker labels via WebSocket or HTTP
• Manages session state (saved_context.json)
• Queries LLM server for context-aware suggestions
• Exposes suggestion API to frontend
c. Frontend UI
– Component: webapp/frontend/
– Modules under voice/: capture audio, open WebSocket, display transcripts, speaker-specific logic, trigger voice training
– Main-app.js: orchestrates modules, updates DOM
d. Voice Profile Training
– Component: asr/voice_training.py & handlers/training.py
– Flow: Browser records audio → sends to ASR service → builds voice profile → stores in user_voice_profile.*
e. External Services
– LLM server (optional) for AI suggestions
– Vosk model files under model/ or vosk-model-spk-0.4/
Architectural Patterns and Design Principles
• Modularization: clear separation of concerns (ASR, API, UI)
• ES Modules on frontend for maintainability
• Compatibility layers for gradual refactoring (controlled by env var)
• Use of WebSocket for real-time bidirectional data flow
• Dockerization for each service