How moin-ansari-o8/auralyx works
1. Product shape
Auralyx is a Windows desktop dictation app, not a browser-only app. The Python backend runs the real-time voice pipeline and local API, while a React/Vite frontend renders the dashboard inside a PyWebView desktop window.
2. Top-level runtime boundary
launcher.py is the main entry point for the desktop experience. It starts or attaches to the backend runtime, opens the native window, and acts as the bridge between the packaged Python process and the embedded web UI.
3. Backend orchestration layer
The backend is centered around backend/main.py and backend/runtime.py. These coordinate the dictation lifecycle, background service threads, single-instance behavior, and the handoff between audio capture, transcription, refinement, persistence, and UI updates.
4. Audio capture and speech pipeline
backend/audio_recorder.py captures microphone audio, backend/speech_to_text.py performs offline transcription with faster-whisper, and the surrounding logic uses VAD-related filtering to separate speech from silence. This is the core “input to raw text” path.
5. AI refinement boundary
backend/ai_provider_manager.py routes refinement requests to configured providers, and backend/ai_refiner.py shapes the prompt/output based on the selected writing mode. The repo supports multiple AI backends, including local/offline and cloud providers, so this layer is the main external-service integration point.
6. System integration for dictation output
backend/paste_manager.py handles clipboard/writeback behavior and automatic pasting into the active application. Hotkey and global-input concerns are isolated in backend/hotkey_diagnostic.py and the main dictation controller, which is important because the app operates outside the browser sandbox and interacts with the OS.
7. API and real-time UI sync
backend/api/server.py exposes the local FastAPI service. It provides REST endpoints for commands, settings, history, and refinement, plus a WebSocket channel for live status such as recording state, processing state, and audio level. The frontend depends on this channel for real-time dashboards and waveform visualization.
8. Persistence and local state
backend/data_storage.py, backend/settings_store.py, and the data/auralyx.db / JSON files represent local persistence. The app stores transcriptions, app settings, and operational history on disk rather than in a remote database, reinforcing the offline-first design.
9. Frontend structure
The React app lives under frontend/src and is organized into pages, layout components, sidebar/titlebar controls, and reusable UI primitives. frontend/src/hooks/useWebSocket.js and useAudioAnalyser.js connect the UI to backend status and visual audio feedback. The pages (Home, Insights, Dictionary, Snippets, Settings) suggest a dashboard-centric workspace around dictation history and configuration.
10. Desktop UI shell
The frontend is styled as a custom application shell with AppShell, MainWorkspace, Sidebar, Titlebar, modals, and floating panels. This indicates the UI is intended to behave like a native app window rather than a conventional website.
11. Packaging and distribution
build-exe.ps1, Auralyx.spec, and build-installer.ps1 show a two-stage packaging flow: build the React assets, then bundle Python plus the frontend into a single executable, and optionally wrap it in an installer. This affects architecture because the frontend is embedded as static build output, not served by a separate web host in production.
12. Operational safety and single-instance control
The repo includes lock/PID-based single-instance protection, tray behavior, and encoding safeguards in the backend/runtime path. These are important architectural boundaries because the app is designed to live in the system tray and avoid multiple active dictation workers.
13. Tests and validation
The tests/ folder covers backend behavior, provider fallback, hotkeys, icon/logo integration, and faster-whisper integration points. That suggests the most important failure modes are in OS integration, provider routing, and packaging assets rather than in isolated UI logic.
14. How the main data flow works
User action enters through a global hotkey or tray-driven command, audio is captured and transcribed locally, text is optionally refined through a chosen AI provider, the result is saved to local history/settings stores, and the polished text is pasted into the active application. In parallel, the backend pushes state updates to the React dashboard over WebSocket so the UI reflects the live session.
Main entry points: main.py main · AppShell.jsx App shell · HomePage.jsx Home · launcher.py launcher.py
Components 21
Desktop shell
Native entrypoint and window/runtime boundary for the packaged app.
- launcher.py · Starts or attaches to the backend runtime and opens the native PyWebView window.
launcher.py
Backend pipeline
Python services for capture, transcription, refinement, persistence, and OS integration.
- main · Coordinates the dictation app lifecycle and delegates work across backend subsystems.
backend/main.py - runtime · Manages startup, single-instance behavior, and the long-lived backend process.
backend/runtime.py - Recorder · Captures microphone audio for the live dictation pipeline.
backend/audio_recorder.py - Speech STT · Runs offline speech-to-text using faster-whisper and returns raw text.
backend/speech_to_text.py - Refiner · Transforms raw text into polished output based on writing mode.
backend/ai_refiner.py - Provider mgr · Selects and routes refinement requests across local and cloud AI providers.
backend/ai_provider_manager.py - Paste mgr · Writes finalized text back into the active application via clipboard or paste actions.
backend/paste_manager.py - Storage · Persists transcription history and operational records on disk.
backend/data_storage.py - Settings · Loads and saves application settings and user preferences.
backend/settings_store.py - Hotkey diag · Diagnoses global hotkey and OS input behavior for dictation control.
backend/hotkey_diagnostic.py
Local API
FastAPI server and realtime status channel consumed by the desktop UI.
- API server · Exposes local REST commands, settings, history, and refinement endpoints.
backend/api/server.py
Frontend UI
React/Vite dashboard embedded in the desktop window.
- main.jsx · Bootstraps the React application inside the desktop window.
frontend/src/main.jsx - App shell · Provides the native-style application frame, workspace, sidebar, and titlebar.
frontend/src/components/layout/AppShell.jsx - Home · Primary workspace for live dictation status, history, and quick actions.
frontend/src/pages/HomePage.jsx - Insights · Shows usage and analysis views for stored dictation data.
frontend/src/pages/InsightsPage.jsx - Settings · Manages configuration and app preferences from the UI.
frontend/src/pages/SettingsPage.jsx - Dictionary · Manages custom vocabulary used by the dictation workflow.
frontend/src/pages/DictionaryPage.jsx - Snippets · Organizes reusable text snippets for faster insertion.
frontend/src/pages/SnippetsPage.jsx - WS hook · Consumes backend status updates for recording state, processing state, and audio levels.
frontend/src/hooks/useWebSocket.js - Audio hook · Drives waveform and audio level visualization from live input.
frontend/src/hooks/useAudioAnalyser.js
Connections 22
- launcher.py → runtime: starts · Launches or attaches to the backend process.
- launcher.py → App shell: opens · Creates the native desktop window that hosts the UI.
- runtime → main: boots · Hands control to the main backend orchestrator.
- main → Recorder: captures · Initiates microphone capture for dictation.
- main → Speech STT: transcribes · Sends captured audio to offline speech recognition.
- main → Refiner: refines · Applies writing-mode aware refinement to raw text.
- Refiner → Provider mgr: routes via · Delegates provider selection and external model calls.
- main → Paste mgr: writes back · Pastes finalized text into the active application.
- main → Storage: persists · Stores history and session artifacts locally.
- main → Settings: reads/writes · Uses local settings for runtime behavior and preferences.
- main → Hotkey diag: monitors · Tracks global input and hotkey health.
- main → API server: publishes state · Feeds command and status data to the local API.
- API server → WS hook: streams · Pushes live recording and processing updates to the UI.
- main.jsx → App shell: mounts · Bootstraps the desktop-oriented React shell.
- App shell → Home: navigates to · Hosts the main dashboard page.
- App shell → Insights: navigates to · Hosts analytics and history views.
- App shell → Settings: navigates to · Hosts configuration screens.
- App shell → Dictionary: navigates to · Hosts vocabulary management.
- App shell → Snippets: navigates to · Hosts reusable snippets.
- Home → WS hook: subscribes · Consumes live backend status for the dashboard.
- Home → Audio hook: visualizes · Renders audio feedback and waveform state.
- Settings → Settings: configures · Reflects and updates persisted settings through the backend.