misyaguziya/vrct

1. Overall shape

- VRCT is a desktop application for VRChat conversation support, built as a three-part system: a Tauri shell for desktop integration, a React/Vite UI for the user experience, and a Python backend for audio, translation, overlay, and integrations.

- The architecture is centered on the frontend driving user interaction, while the Python process performs most runtime work.

2. Desktop host and process boundary

- src-tauri is the native desktop container. It packages the app, manages the window, permissions/capabilities, and platform-specific build/release behavior.

- The Tauri layer is the boundary between the OS and the app: it launches the Python backend, hosts the web UI, and handles native capabilities such as window control and system integration.

3. Frontend application layer

- src-ui is the main user-facing app, organized around pages (main_page, config_page) and reusable common components.

- It is a React codebase with state/logics separated from presentation: views contains UI, while logics holds hooks and state coordination.

4. UI responsibility split

- The main page is the operational screen for live transcription/translation/chat flow.

- The config page manages settings for devices, transcription, translation providers, hotkeys, plugins, appearance, VR-related behavior, and app metadata.

- The UI also includes splash/loading, updating, modal, snackbar, and error boundary subsystems, which means startup, update, and failure states are first-class parts of the architecture.

5. Frontend state and backend communication

- src-ui/logics contains hooks for backend readiness, network handling, receiving routes, volume, LLM connectivity, software updates, and stdout forwarding from Python.

- This indicates the UI does not just render data; it actively orchestrates configuration, subscribes to backend status, and reacts to asynchronous backend events.

- The UI talks to the backend through message/route-style communication rather than direct shared memory.

6. Python backend as service core

- src-python is the computational core, with modules for mainloop, controller, model, utils, and error handling.

- The backend appears structured as a long-running service that coordinates device access, transcription pipelines, translation pipelines, overlay output, websocket serving, watchdog behavior, and telemetry.

7. Audio and speech pipeline

- The transcription subsystem is split into recorder, transcriber, Whisper-based transcription, and language definitions.

- This suggests an audio flow of: device selection in the UI → backend capture via the recorder → transcription via the transcriber/Whisper layer → delivery of recognized text to downstream consumers.

- device_manager.py and related docs imply explicit management of input/output audio devices and compute device selection.

8. Translation subsystem

- Translation is provider-based, with support modules for OpenAI, OpenRouter, Gemini, Groq, LM Studio, Ollama, and Plamo.

- There is a shared translation layer plus provider-specific adapters and settings/prompt templates under models/translation/translation_settings.

- This makes translation a pluggable integration boundary: the backend normalizes requests, then dispatches to the selected provider.

9. Output and integration channels

- The backend includes modules for overlay, clipboard, OSC, and websocket server output.

- These are distinct delivery paths for translated or transcribed content:

- overlay for on-screen rendering,

- clipboard for copy/paste workflows,

- OSC for external control/integration,

- websocket for local network or UI clients.

- The architecture therefore supports multiple consumers of the same language-processing results.

10. Overlay and presentation support

- models/overlay contains image/font utilities and overlay rendering logic, indicating the app can present text in a dedicated always-on-top style surface.

- The UI also has runtime controls for transparency, corner radius, font family, and window behavior, which means overlay appearance is part of the product’s core behavior, not just a theme setting.

11. Telemetry and update awareness

- Telemetry is explicitly modeled under models/telemetry, with client/core/state modules, and the README states anonymous usage data is collected via Aptabase.

- Updates are also integrated into the app flow through frontend update modals and backend/software-version hooks.

- So operationally, the app includes lifecycle concerns: startup progress, update availability, and anonymous usage tracking.

12. Plugins and extensibility

- The UI has a plugin system in src-ui/plugins and plugin controllers in the app layer.

- The presence of plugin loading/merging/fetching latest plugin data indicates the app supports extension points rather than being a fixed monolith.

- For a diagram, plugins should be shown as an optional externalized extension layer attached mainly to the UI/app orchestration layer.

13. Configuration and localization

- Configuration is a first-class subsystem spanning the config page, backend config modules, and locale files.

- Localization is handled through locales/*.yml and includes multiple languages for the UI and docs, which materially affects the rendering layer but not backend computation.

- Settings appear to drive almost every subsystem: device choice, language selection, translation provider, hotkeys, appearance, and VR behaviors.

14. Build, packaging, and verification

- The repository is a multi-toolchain desktop project:

- React/Vite for frontend build,

- Tauri/Rust for desktop packaging,

- Python for backend runtime,

- Windows batch scripts under bat/ for build/install variants including CUDA support.

- Tests are split between Python backend specs and endpoint/test-client utilities, reflecting that the backend can be validated independently of the UI.

- For an architecture diagram, the important deployment boundary is a packaged desktop app that contains a native shell, a web UI, and a separate Python service process.