jeffrey0117/speakslow

1. Repository shape

This repo is not a single app; it contains a Windows desktop product, a separate web experience/marketing site, and a set of Python ASR experiments and model assets. The desktop app is the core product; the web folders and docs support demos, documentation, and deployment.

2. Primary product: Electron desktop app

The main runtime is an Electron app: main.js boots the Electron process, preload.js exposes safe bridges, and src/ contains the React UI. start-electron.js is the startup entrypoint used during development/package launch. The app is explicitly tuned for Windows and local speech input.

3. UI layer and screens

The desktop UI is built with React + Vite + Tailwind and is split into settings and workflow views:

- src/App.jsx and src/main.jsx drive the main shell

- src/settings.jsx, src/history.jsx, src/history.html, src/settings.html support dedicated windows/pages

- components like HotwordsManager, DictionaryManager, HotkeySettings, HistoryView, and TranscribeModal map directly to product features

The UI is primarily for controlling capture, reviewing history, configuring models, and managing text-processing options.

4. Native/system boundary

Desktop interaction is routed through a clear boundary in src/helpers/ipc/* and src/helpers/ipcHandlers.js. These IPC modules expose system-level capabilities to the UI: window control, hotkeys, transcription, clipboard behavior, AI, database access, and device/system functions. This is the main seam between renderer code and privileged Electron/main-process code.

5. Voice capture and hotkey flow

The central user flow is: press a global hotkey → record audio → transcribe locally → post-process text → paste at the cursor. The hotkey system is implemented around uiohook-napi per the README, with hooks like useRecording, useStreamingRecording, and helpers such as hotkeyManager, windowManager, tray, and typelessManager coordinating capture state and user feedback.

6. Local ASR pipeline

Speech recognition is designed to run fully on-device using sherpa-onnx models managed by src/helpers/sherpaManager.js and Python helpers. The repo includes model download tooling (download_all_models.py) and bundled model directories under poc-sherpa/. The ASR stack in the README is composed of:

- Paraformer for offline recognition

- Silero VAD for speech activity detection

- ct-transformer for punctuation

- Zipformer / streaming recognition for incremental decoding

The architecture emphasizes low-latency transcription and local privacy.

7. Text post-processing pipeline

After recognition, text goes through local cleanup and formatting before insertion. That logic is split between src/helpers/text...-style modules such as aiTextProcessor, commandMode, translate, tts, and the standalone text_processing.py. Features include punctuation cleanup, filler-word removal, list formatting, whitespace/line handling, and optional AI-assisted rewriting. The repo’s docs suggest this is a deliberate rule-based pipeline before any LLM step.

8. Persistence and user history

The app keeps usage/history locally through SQLite via better-sqlite3 and src/helpers/database.js plus IPC wrappers. History-related UI and stats live in HistoryView and related components. This persistence layer stores transcription history, recordings, and dashboard metrics so users can search, re-run recognition, and inspect long-term usage without sending data off-device.

9. Optional AI and integrations

AI features are explicitly optional and isolated: src/helpers/aiPrompts.js, aiTextProcessor.js, translate.js, and tts.js suggest add-on processing for polishing, translation, or speech output. The README says these can target OpenAI-compatible services or local Ollama/LM Studio, so the architecture keeps the ASR path local while allowing an external or local LLM as a separate enrichment step.

10. Python services and experiments

sherpa_server.py, sherpa_web_server.py, requirements-web.txt, and the poc-sherpa/ directory indicate a Python-side experimentation/service layer for ASR model testing and possibly web-serving demos. The poc-sherpa folder contains demo scripts, comparison reports, and model snapshots, which looks like a research sandbox rather than the production UI path.

11. Web app and public site

The web/ directory is a separate TypeScript React/Vite application with pages like HomePage, GuidePage, ComparePage, and VoicePage. It has hooks for useApi, useAudioRecorder, and useWebSocket, so it likely supports an online demo or web-backed comparison flow distinct from the Electron desktop product. Separately, the website/ folder is a static marketing site with plain HTML/CSS/JS.

12. Packaging, deployment, and infra

The repo includes multiple delivery targets:

- Electron packaging/build scripts (build/installer.nsh, scripts/build-backend.mjs, GitHub Actions for Linux/macOS)

- Dockerized backend/frontend and Nginx configs under docker/ and docker-compose*.yml

- model download automation and environment examples (.env.example, download_all_models.py)

This means the architecture spans desktop distribution, containerized deployment, and static web publishing, but the core product remains the local Electron speech-input workflow.

13. Documentation as architecture source

The docs/ directory is unusually comprehensive and appears to function as the project’s design record: ASR architecture, streaming recognition, VAD optimization, deployment, feature specs, roadmap, and website specs. For diagramming purposes, those docs reinforce that the repo is organized around a local-first speech pipeline, with UI, IPC, ASR models, and optional AI as distinct layers.