# jeffrey0117/speakslow architecture

> Architecture diagram of the GitHub repository jeffrey0117/speakslow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jeffrey0117/speakslow
- Repository: https://github.com/jeffrey0117/speakslow

## Overview

1. **Repository shape**
   This repo is not a single app; it contains a **Windows desktop product**, a **separate web experience/marketing site**, and a set of **Python ASR experiments and model assets**. The desktop app is the core product; the web folders and docs support demos, documentation, and deployment.

2. **Primary product: Electron desktop app**
   The main runtime is an **Electron app**: `main.js` boots the Electron process, `preload.js` exposes safe bridges, and `src/` contains the React UI. `start-electron.js` is the startup entrypoint used during development/package launch. The app is explicitly tuned for **Windows** and local speech input.

3. **UI layer and screens**
   The desktop UI is built with **React + Vite + Tailwind** and is split into settings and workflow views:
   - `src/App.jsx` and `src/main.jsx` drive the main shell
   - `src/settings.jsx`, `src/history.jsx`, `src/history.html`, `src/settings.html` support dedicated windows/pages
   - components like `HotwordsManager`, `DictionaryManager`, `HotkeySettings`, `HistoryView`, and `TranscribeModal` map directly to product features
   The UI is primarily for controlling capture, reviewing history, configuring models, and managing text-processing options.

4. **Native/system boundary**
   Desktop interaction is routed through a clear boundary in `src/helpers/ipc/*` and `src/helpers/ipcHandlers.js`. These IPC modules expose system-level capabilities to the UI: window control, hotkeys, transcription, clipboard behavior, AI, database access, and device/system functions. This is the main seam between renderer code and privileged Electron/main-process code.

5. **Voice capture and hotkey flow**
   The central user flow is: **press a global hotkey → record audio → transcribe locally → post-process text → paste at the cursor**. The hotkey system is implemented around `uiohook-napi` per the README, with hooks like `useRecording`, `useStreamingRecording`, and helpers such as `hotkeyManager`, `windowManager`, `tray`, and `typelessManager` coordinating capture state and user feedback.

6. **Local ASR pipeline**
   Speech recognition is designed to run **fully on-device** using **sherpa-onnx** models managed by `src/helpers/sherpaManager.js` and Python helpers. The repo includes model download tooling (`download_all_models.py`) and bundled model directories under `poc-sherpa/`. The ASR stack in the README is composed of:
   - **Paraformer** for offline recognition
   - **Silero VAD** for speech activity detection
   - **ct-transformer** for punctuation
   - **Zipformer / streaming recognition** for incremental decoding
   The architecture emphasizes low-latency transcription and local privacy.

7. **Text post-processing pipeline**
   After recognition, text goes through local cleanup and formatting before insertion. That logic is split between `src/helpers/text...`-style modules such as `aiTextProcessor`, `commandMode`, `translate`, `tts`, and the standalone `text_processing.py`. Features include punctuation cleanup, filler-word removal, list formatting, whitespace/line handling, and optional AI-assisted rewriting. The repo’s docs suggest this is a deliberate rule-based pipeline before any LLM step.

8. **Persistence and user history**
   The app keeps usage/history locally through **SQLite** via `better-sqlite3` and `src/helpers/database.js` plus IPC wrappers. History-related UI and stats live in `HistoryView` and related components. This persistence layer stores transcription history, recordings, and dashboard metrics so users can search, re-run recognition, and inspect long-term usage without sending data off-device.

9. **Optional AI and integrations**
   AI features are explicitly optional and isolated: `src/helpers/aiPrompts.js`, `aiTextProcessor.js`, `translate.js`, and `tts.js` suggest add-on processing for polishing, translation, or speech output. The README says these can target **OpenAI-compatible services** or **local Ollama/LM Studio**, so the architecture keeps the ASR path local while allowing an external or local LLM as a separate enrichment step.

10. **Python services and experiments**
   `sherpa_server.py`, `sherpa_web_server.py`, `requirements-web.txt`, and the `poc-sherpa/` directory indicate a **Python-side experimentation/service layer** for ASR model testing and possibly web-serving demos. The `poc-sherpa` folder contains demo scripts, comparison reports, and model snapshots, which looks like a research sandbox rather than the production UI path.

11. **Web app and public site**
   The `web/` directory is a separate **TypeScript React/Vite** application with pages like `HomePage`, `GuidePage`, `ComparePage`, and `VoicePage`. It has hooks for `useApi`, `useAudioRecorder`, and `useWebSocket`, so it likely supports an online demo or web-backed comparison flow distinct from the Electron desktop product. Separately, the `website/` folder is a static marketing site with plain HTML/CSS/JS.

12. **Packaging, deployment, and infra**
   The repo includes multiple delivery targets:
   - Electron packaging/build scripts (`build/installer.nsh`, `scripts/build-backend.mjs`, GitHub Actions for Linux/macOS)
   - Dockerized backend/frontend and Nginx configs under `docker/` and `docker-compose*.yml`
   - model download automation and environment examples (`.env.example`, `download_all_models.py`)
   This means the architecture spans **desktop distribution**, **containerized deployment**, and **static web publishing**, but the core product remains the local Electron speech-input workflow.

13. **Documentation as architecture source**
   The `docs/` directory is unusually comprehensive and appears to function as the project’s design record: ASR architecture, streaming recognition, VAD optimization, deployment, feature specs, roadmap, and website specs. For diagramming purposes, those docs reinforce that the repo is organized around a **local-first speech pipeline**, with UI, IPC, ASR models, and optional AI as distinct layers.

Main entry points: [`src/helpers/ipcHandlers.js`](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/ipcHandlers.js) (IPC handlers), [`src/App.jsx`](https://github.com/jeffrey0117/speakslow/blob/main/src/App.jsx) (Desktop UI), [`src/helpers/sherpaManager.js`](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/sherpaManager.js) (Sherpa manager), [`docs/asr-architecture.md`](https://github.com/jeffrey0117/speakslow/blob/main/docs/asr-architecture.md) (Architecture docs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_desktop["Electron desktop app"]
  node_main["Electron main<br/>electron bootstrap<br/>[main.js]"]
  node_preload["Preload bridge<br/>ipc bridge<br/>[preload.js]"]
  node_desktop_ui["Desktop UI<br/>react renderer<br/>[App.jsx]"]
  node_settings_history["Settings & history<br/>aux windows<br/>[settings.jsx]"]
  node_feature_components["Feature panels<br/>ui components"]
  node_hooks["Recording hooks<br/>ui hooks<br/>[useRecording.js]"]
  node_ipc_handlers["IPC handlers<br/>privileged services<br/>[ipcHandlers.js]"]
  node_text_pipeline["Text pipeline<br/>post-processing<br/>[aiTextProcessor.js]"]
  node_store[("Local store<br/>sqlite persistence<br/>[database.js]")]
  node_system_helpers["System helpers<br/>window/tray/hotkeys<br/>[hotkeyManager.js]"]
end

subgraph group_asr["Local ASR pipeline"]
  node_sherpa_manager["Sherpa manager<br/>asr orchestration<br/>[sherpaManager.js]"]
  node_python_asr["Python ASR<br/>server/experiments<br/>[sherpa_server.py]"]
  node_model_assets[("Model assets<br/>onnx models<br/>[silero_vad.onnx]")]
end

subgraph group_web["Web experiences"]
  node_web_app["Web app<br/>vite react<br/>[App.tsx]"]
  node_web_pages["Web pages<br/>[HomePage.tsx]"]
  node_website_site["Marketing site<br/>static site<br/>[index.html]"]
end

subgraph group_infra["Packaging and docs"]
  node_build_packaging["Packaging<br/>build/deploy<br/>[build-backend.mjs]"]
  node_docs["Architecture docs<br/>design docs"]
end

node_main -->|"loads"| node_preload
node_preload -->|"bridges"| node_desktop_ui
node_desktop_ui -->|"composes"| node_feature_components
node_desktop_ui -->|"opens"| node_settings_history
node_feature_components -->|"uses"| node_hooks
node_hooks -->|"orchestrates"| node_system_helpers
node_system_helpers -->|"invokes"| node_ipc_handlers
node_ipc_handlers -->|"persists"| node_store
node_ipc_handlers -->|"starts transcription"| node_sherpa_manager
node_sherpa_manager -->|"loads"| node_model_assets
node_sherpa_manager -->|"emits text"| node_text_pipeline
node_text_pipeline -->|"returns output"| node_ipc_handlers
node_ipc_handlers -->|"reports state"| node_desktop_ui
node_python_asr -.->|"tests"| node_model_assets
node_web_app -->|"routes"| node_web_pages
node_website_site -.->|"reflects"| node_docs
node_build_packaging -->|"packages"| node_main
node_build_packaging -->|"deploys"| node_web_app
node_docs -.->|"specifies"| node_sherpa_manager
node_docs -.->|"describes"| node_desktop_ui

click node_main "https://github.com/jeffrey0117/speakslow/blob/main/main.js"
click node_preload "https://github.com/jeffrey0117/speakslow/blob/main/preload.js"
click node_desktop_ui "https://github.com/jeffrey0117/speakslow/blob/main/src/App.jsx"
click node_settings_history "https://github.com/jeffrey0117/speakslow/blob/main/src/settings.jsx"
click node_feature_components "https://github.com/jeffrey0117/speakslow/blob/main/src/components/HotwordsManager.jsx"
click node_hooks "https://github.com/jeffrey0117/speakslow/blob/main/src/hooks/useRecording.js"
click node_ipc_handlers "https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/ipcHandlers.js"
click node_text_pipeline "https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/aiTextProcessor.js"
click node_store "https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/database.js"
click node_system_helpers "https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/hotkeyManager.js"
click node_sherpa_manager "https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/sherpaManager.js"
click node_python_asr "https://github.com/jeffrey0117/speakslow/blob/main/sherpa_server.py"
click node_model_assets "https://github.com/jeffrey0117/speakslow/blob/main/poc-sherpa/silero_vad.onnx"
click node_web_app "https://github.com/jeffrey0117/speakslow/blob/main/web/src/App.tsx"
click node_web_pages "https://github.com/jeffrey0117/speakslow/blob/main/web/src/pages/HomePage.tsx"
click node_website_site "https://github.com/jeffrey0117/speakslow/blob/main/website/index.html"
click node_build_packaging "https://github.com/jeffrey0117/speakslow/blob/main/scripts/build-backend.mjs"
click node_docs "https://github.com/jeffrey0117/speakslow/blob/main/docs/asr-architecture.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_preload,node_desktop_ui,node_settings_history,node_feature_components,node_hooks,node_ipc_handlers,node_text_pipeline,node_store,node_system_helpers toneBlue
class node_sherpa_manager,node_python_asr,node_model_assets toneAmber
class node_web_app,node_web_pages,node_website_site toneMint
class node_build_packaging,node_docs toneRose
```

## Components

### Electron desktop app

Core Windows speech-input product and renderer/main-process boundary

- [Electron main](https://github.com/jeffrey0117/speakslow/blob/main/main.js): electron bootstrap · Starts the desktop app and owns the privileged runtime · `main.js`
- [Preload bridge](https://github.com/jeffrey0117/speakslow/blob/main/preload.js): ipc bridge · Exposes safe renderer APIs into Electron · `preload.js`
- [Desktop UI](https://github.com/jeffrey0117/speakslow/blob/main/src/App.jsx): react renderer · Main React shell and app entry for the desktop product · `src/App.jsx`
- [Settings & history](https://github.com/jeffrey0117/speakslow/blob/main/src/settings.jsx): aux windows · Dedicated settings and history windows/pages · `src/settings.jsx`
- [Feature panels](https://github.com/jeffrey0117/speakslow/blob/main/src/components/HotwordsManager.jsx): ui components · Hotwords, dictionary, hotkeys, transcribe, and history controls · `src/components/HotwordsManager.jsx`
- [Recording hooks](https://github.com/jeffrey0117/speakslow/blob/main/src/hooks/useRecording.js): ui hooks · Capture and streaming recording state in the renderer · `src/hooks/useRecording.js`
- [IPC handlers](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/ipcHandlers.js): privileged services · Main-process bridge for system, window, transcription, AI, and database actions · `src/helpers/ipcHandlers.js`
- [Text pipeline](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/aiTextProcessor.js): post-processing · Local cleanup, command mode, translation, TTS, and AI rewriting · `src/helpers/aiTextProcessor.js`
- [Local store](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/database.js): sqlite persistence · SQLite-backed history, recordings, and usage data · `src/helpers/database.js`
- [System helpers](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/hotkeyManager.js): window/tray/hotkeys · Global hotkey, tray, window, clipboard, and recovery coordination · `src/helpers/hotkeyManager.js`

### Local ASR pipeline

On-device recognition models, experiments, and Python services

- [Sherpa manager](https://github.com/jeffrey0117/speakslow/blob/main/src/helpers/sherpaManager.js): asr orchestration · Loads local speech models and manages recognition flow · `src/helpers/sherpaManager.js`
- [Python ASR](https://github.com/jeffrey0117/speakslow/blob/main/sherpa_server.py): server/experiments · Python services and comparison scripts for model testing · `sherpa_server.py`
- [Model assets](https://github.com/jeffrey0117/speakslow/blob/main/poc-sherpa/silero_vad.onnx): onnx models · Bundled sherpa-onnx model snapshots and VAD/punctuation assets · `poc-sherpa/silero_vad.onnx`

### Web experiences

Separate TypeScript site and static marketing pages

- [Web app](https://github.com/jeffrey0117/speakslow/blob/main/web/src/App.tsx): vite react · Separate TypeScript web experience for demo and comparison flows · `web/src/App.tsx`
- [Web pages](https://github.com/jeffrey0117/speakslow/blob/main/web/src/pages/HomePage.tsx): pages · Home, guide, compare, voice, and story pages · `web/src/pages/HomePage.tsx`
- [Marketing site](https://github.com/jeffrey0117/speakslow/blob/main/website/index.html): static site · Static public-facing site and content pages · `website/index.html`

### Packaging and docs

Build, deployment, and architecture documentation

- [Packaging](https://github.com/jeffrey0117/speakslow/blob/main/scripts/build-backend.mjs): build/deploy · Electron packaging, backend build, Docker, and CI workflows · `scripts/build-backend.mjs`
- [Architecture docs](https://github.com/jeffrey0117/speakslow/blob/main/docs/asr-architecture.md): design docs · Specs and research notes for ASR, UI, deployment, and website · `docs/asr-architecture.md`

## Connections

- Electron main → Preload bridge: loads · Main process exposes renderer-safe APIs through preload
- Preload bridge → Desktop UI: bridges · Renderer accesses system capabilities through IPC
- Desktop UI → Feature panels: composes · Main shell renders feature-specific panels
- Desktop UI → Settings & history: opens · Desktop app uses dedicated settings and history views
- Feature panels → Recording hooks: uses · UI controls recording and capture state
- Recording hooks → System helpers: orchestrates · Hooks coordinate hotkeys, windows, tray, and capture feedback
- System helpers → IPC handlers: invokes · System actions are routed through privileged IPC handlers
- IPC handlers → Local store: persists · History and usage data are written to SQLite
- IPC handlers → Sherpa manager: starts transcription · IPC requests trigger local ASR sessions
- Sherpa manager → Model assets: loads · Recognition uses bundled on-device models and VAD assets
- Sherpa manager → Text pipeline: emits text · Raw recognition output is passed to cleanup and enrichment
- Text pipeline → IPC handlers: returns output · Processed text flows back to the privileged runtime for paste or follow-up actions
- IPC handlers → Desktop UI: reports state · UI receives model, permission, and recording updates
- Python ASR → Model assets: tests · Python experiments compare and validate local model sets
- Web app → Web pages: routes · Web runtime renders marketing and demo pages
- Marketing site → Architecture docs: reflects · Public site content aligns with project messaging and specs
- Packaging → Electron main: packages · Build scripts package the Electron desktop app
- Packaging → Web app: deploys · Infrastructure also builds and serves the web experience
- Architecture docs → Sherpa manager: specifies · Docs define the ASR architecture and optimization choices
- Architecture docs → Desktop UI: describes · Docs capture UI, hotkey, and window behavior