# joru10/opencluely architecture

> Architecture diagram of the GitHub repository joru10/opencluely, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/joru10/opencluely
- Repository: https://github.com/joru10/opencluely

## Overview

##### Project Type and Purpose
- Project Type: Full-stack web application (real-time interview assistant).
- Purpose: Captures microphone audio, performs low-latency ASR (speech-to-text) + speaker identification, sends transcript/context to an LLM to generate suggestions, displays suggestions in a browser UI.
- Key Indicators:
– Top-level `asr/` package for all speech logic
– `webapp/backend/` (FastAPI) and `webapp/frontend/` (JS modules)
– Entry scripts: `start-webapp.sh`, `vosk_asr_server_refactored.py`

##### File Structure
- asr/
– server.py: WebSocket ASR endpoint
– vosk_recognizer.py: Vosk-based speech decoding
– speaker_identification.py: Voiceprint matching
– voice_training.py: Builds custom voice profile
- webapp/
– backend/: FastAPI app (app.py)
– frontend/: Vanilla JS UI (index.html + ES modules in `voice/`)
- Root-level helpers and scripts
– Dockerfiles (asr, frontend, backend)
– start-all-services.sh, start-webapp.sh
– Alternate ASR servers & test stubs (asr-stub/, mock_asr_server.py)
– “backend/” directory: utility scripts (whisper_transcriber.py, direct_asr.py, profile_manager.py)
- Config/deployment files
– .env, config.py, devcontainer.json

##### README for Additional Insights
- Project follows a modular architecture (ASR package, webapp, entry scripts)
- Two ASR implementations (refactored vs. compatibility) toggled via `USE_REFACTORED`
- Technology stack: Python3.9+, FastAPI, websockets, Vosk, vanilla JS, Web Audio API, optional LLM server

##### Main Components and Their Interactions
a. ASR Service
– Component: `asr/server.py` or `vosk_asr_server_refactored.py`
– Interfaces:
- WebSocket endpoint ← audio stream from browser
- Returns (timestamped) text + speaker ID
b. Backend API
– Component: `webapp/backend/app.py` (FastAPI)
– Responsibilities:
- Receives transcripts + speaker labels via WebSocket or HTTP
- Manages session state (saved_context.json)
- Queries LLM server for context-aware suggestions
- Exposes suggestion API to frontend
c. Frontend UI
– Component: `webapp/frontend/`
– Modules under `voice/`: capture audio, open WebSocket, display transcripts, speaker-specific logic, trigger voice training
– Main-app.js: orchestrates modules, updates DOM
d. Voice Profile Training
– Component: `asr/voice_training.py` & `handlers/training.py`
– Flow: Browser records audio → sends to ASR service → builds voice profile → stores in `user_voice_profile.*`
e. External Services
– LLM server (optional) for AI suggestions
– Vosk model files under `model/` or `vosk-model-spk-0.4/`

##### Architectural Patterns and Design Principles
- Modularization: clear separation of concerns (ASR, API, UI)
- ES Modules on frontend for maintainability
- Compatibility layers for gradual refactoring (controlled by env var)
- Use of WebSocket for real-time bidirectional data flow
- Dockerization for each service

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend UI Layer
    subgraph "Frontend UI (Browser)" 
        direction TB
        UI_INDEX["index.html"]:::frontend
        UI_MAIN["main-app.js"]:::frontend
        subgraph "voice ES Modules"
            direction TB
            UI_AUDIO["audio-capture.js"]:::frontend
            UI_VOICE_INDEX["index.js"]:::frontend
            UI_UICOMP["ui-components.js"]:::frontend
            UI_WSCLIENT["websocket-client.js"]:::frontend
        end
        UI_DOCKER["Dockerfile"]:::frontend
    end

    %% ASR Service Layer
    subgraph "ASR Service (Docker)" 
        direction TB
        ASR_SERVER["server.py"]:::realtime
        ASR_RECOG["vosk_recognizer.py"]:::realtime
        ASR_SPKID["speaker_identification.py"]:::realtime
        ASR_VTRAIN["voice_training.py"]:::realtime
        ASR_REF["vosk_asr_server_refactored.py"]:::realtime
        ASR_DOCKER["Dockerfile"]:::realtime
    end

    %% Backend API Layer
    subgraph "Backend API (FastAPI)" 
        direction TB
        API_APP["app.py"]:::backend
        API_MON["profile_monitor.py"]:::backend
        API_CTX["saved_context.json"]:::backend
        API_REQ["requirements.txt"]:::backend
        API_DOCKER["Dockerfile"]:::backend
    end

    %% Voice Training Handler
    subgraph "Voice Profile Training" 
        direction TB
        HANDLER["handlers/training.py"]:::backend
        VT_ASR["asr/voice_training.py"]:::backend
        VT_CREATE["create_voice_profile.py"]:::backend
        VT_PM["profile_manager.py"]:::backend
    end

    %% ASR Stub & Mock
    subgraph "ASR Stub & Mock" 
        direction TB
        STUB["asr-stub/stub_asr.py"]:::realtime
        TEST_CLIENT["asr-stub/test_client.py"]:::realtime
        MOCK_SERVER["mock_asr_server.py"]:::realtime
    end

    %% Configuration & Deployment
    subgraph "Config & Deployment" 
        direction TB
        CFG_ENV[".env"]:::external
        CFG_PY["config.py"]:::external
        CFG_DEV["devcontainer.json"]:::external
        SCRIPTS["start-webapp.sh\nstart-all-services.sh"]:::external
    end

    %% Model Storage
    subgraph "Model Storage" 
        direction TB
        MODEL_DIR["model/"]:::external
        VOSK_MODEL["vosk-model-spk-0.4/"]:::external
    end

    %% Context & Voice Profile Storage
    subgraph "Context & Voice Profile Storage" 
        direction TB
        STORE_CTX["webapp/backend/saved_context.json"]:::external
        STORE_META["user_voice_profile.meta.json"]:::external
        STORE_NPY["user_voice_profile.npy"]:::external
        STORE_PKL["user_voice_profile.pkl"]:::external
    end

    %% External LLM Service
    LLM["LLM Server"]:::external

    %% Data Flows
    UI_AUDIO -->|WebSocket audio stream| ASR_SERVER
    UI_WSCLIENT -->|WebSocket connect| ASR_SERVER
    ASR_SERVER -->|"transcript + speaker ID"| API_APP
    API_APP -->|HTTP/REST| LLM
    LLM -->|suggestions| API_APP
    API_APP -->|HTTP/REST| UI_MAIN
    UI_MAIN -->|display suggestions| UI_UICOMP

    %% Voice training flow
    UI_AUDIO -->|audio chunk| ASR_SERVER
    ASR_SERVER -->|voice data| HANDLER
    HANDLER -->|create profile| VT_ASR
    VT_ASR -->|profile files| STORE_META

    %% ASR Stub testing
    TEST_CLIENT -->|test audio| STUB
    STUB -->|mock transcript| API_APP

    %% Click Events
    click UI_INDEX "https://github.com/joru10/opencluely/blob/main/webapp/frontend/index.html"
    click UI_MAIN "https://github.com/joru10/opencluely/blob/main/webapp/frontend/main-app.js"
    click UI_AUDIO "https://github.com/joru10/opencluely/blob/main/webapp/frontend/voice/audio-capture.js"
    click UI_VOICE_INDEX "https://github.com/joru10/opencluely/blob/main/webapp/frontend/voice/index.js"
    click UI_UICOMP "https://github.com/joru10/opencluely/blob/main/webapp/frontend/voice/ui-components.js"
    click UI_WSCLIENT "https://github.com/joru10/opencluely/blob/main/webapp/frontend/voice/websocket-client.js"
    click UI_DOCKER "https://github.com/joru10/opencluely/tree/main/webapp/frontend/Dockerfile"
    click ASR_SERVER "https://github.com/joru10/opencluely/blob/main/asr/server.py"
    click ASR_RECOG "https://github.com/joru10/opencluely/blob/main/asr/vosk_recognizer.py"
    click ASR_SPKID "https://github.com/joru10/opencluely/blob/main/asr/speaker_identification.py"
    click ASR_VTRAIN "https://github.com/joru10/opencluely/blob/main/asr/voice_training.py"
    click ASR_REF "https://github.com/joru10/opencluely/blob/main/vosk_asr_server_refactored.py"
    click ASR_DOCKER "https://github.com/joru10/opencluely/tree/main/asr/Dockerfile"
    click API_APP "https://github.com/joru10/opencluely/blob/main/webapp/backend/app.py"
    click API_MON "https://github.com/joru10/opencluely/blob/main/webapp/backend/profile_monitor.py"
    click API_CTX "https://github.com/joru10/opencluely/blob/main/webapp/backend/saved_context.json"
    click API_REQ "https://github.com/joru10/opencluely/blob/main/webapp/backend/requirements.txt"
    click API_DOCKER "https://github.com/joru10/opencluely/tree/main/webapp/backend/Dockerfile"
    click HANDLER "https://github.com/joru10/opencluely/blob/main/handlers/training.py"
    click VT_ASR "https://github.com/joru10/opencluely/blob/main/asr/voice_training.py"
    click VT_CREATE "https://github.com/joru10/opencluely/blob/main/create_voice_profile.py"
    click VT_PM "https://github.com/joru10/opencluely/blob/main/profile_manager.py"
    click STUB "https://github.com/joru10/opencluely/blob/main/asr-stub/stub_asr.py"
    click TEST_CLIENT "https://github.com/joru10/opencluely/blob/main/asr-stub/test_client.py"
    click MOCK_SERVER "https://github.com/joru10/opencluely/blob/main/mock_asr_server.py"
    click CFG_ENV "https://github.com/joru10/opencluely/blob/main/.env"
    click CFG_PY "https://github.com/joru10/opencluely/blob/main/config.py"
    click CFG_DEV "https://github.com/joru10/opencluely/blob/main/devcontainer/devcontainer.json"
    click SCRIPTS "https://github.com/joru10/opencluely/blob/main/start-webapp.sh"
    click MODEL_DIR "https://github.com/joru10/opencluely/tree/main/model/"
    click VOSK_MODEL "https://github.com/joru10/opencluely/tree/main/vosk-model-spk-0.4/"
    click STORE_CTX "https://github.com/joru10/opencluely/blob/main/webapp/backend/saved_context.json"
    click STORE_META "https://github.com/joru10/opencluely/blob/main/user_voice_profile.meta.json"
    click STORE_NPY "https://github.com/joru10/opencluely/blob/main/user_voice_profile.npy"
    click STORE_PKL "https://github.com/joru10/opencluely/blob/main/user_voice_profile.pkl"

    %% Styles
    classDef realtime fill:#E0F7FA,stroke:#0277BD,color:#01579B
    classDef backend fill:#E8F5E9,stroke:#2E7D32,color:#1B5E20
    classDef frontend fill:#FFF3E0,stroke:#EF6C00,color:#E65100
    classDef external fill:#ECEFF1,stroke:#546E7A,color:#37474F
```
