# avarayr/yap-for-cursor architecture

> Architecture diagram of the GitHub repository avarayr/yap-for-cursor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/avarayr/yap-for-cursor
- Repository: https://github.com/avarayr/yap-for-cursor

## Overview

##### Project type and purpose
- **Type**: A client-side editor extension (open-source tool) that injects JavaScript into the Cursor editor to provide in-browser, WebGPU-powered voice-to-text (ASR) using Hugging Face’s Whisper model.
- **Purpose**: Seamlessly add a microphone button and hotkey to Cursor’s UI, record audio locally, run a Whisper model in a Web Worker via WebGPU, and insert the transcribed text into the chat input.

##### File structure
- **Top-level files**
- build.ts / bun.lock / package.json / tsconfig.json: build and dependency configuration (Bundler = Bun, TypeScript).
- dist/yap-for-cursor.js: final injected bundle.
- **src/**
- src/main.ts: entry point that initializes the extension, loads UI, wires up hotkeys.
- src/ui/: UI components (mic-button.ts, context-menu.ts, dom-selectors.ts).
- src/asr/: ASR orchestration (manager.ts), instance lifecycle (instance.ts), Worker thread logic (worker.ts).
- src/audio/processing.ts: raw audio capture, preprocessing (e.g., buffering, format conversion).
- src/utils/hotkey.ts: global hotkey registration.
- src/config.ts & src/types.ts: configuration constants and shared type definitions.
- src/styles/: CSS for button styling.
- **Build output** in dist – ready-to-inject JS.

##### Key insights from README
- Local in-browser processing; no external servers except initial model download.
- Uses a VS Code “Custom CSS and JS Loader” to inject the bundle.
- Hotkey support (Cmd+Shift+Y) and click interactions.
- Dependencies: Hugging Face Transformers (Whisper), WebGPU, Web Worker API.

##### Main components & interactions (to represent
a. **Cursor Host Environment**
- The Cursor editor / DOM into which the script is injected via Custom CSS/JS Loader.
b. **Injection Layer (dist/yap-for-cursor.js)**
- The single bundle injected at startup.
c. **UI Layer**
- Microphone Button (src/ui/mic-button.ts)
- Context Menu (src/ui/context-menu.ts)
- DOM Selectors (src/ui/dom-selectors.ts)
- CSS Styles (src/styles/styles.css)
d. **Control & Hotkey**
- Hotkey Manager (src/utils/hotkey.ts)
- Entry Point (src/main.ts) wires UI↔Hotkey.
e. **Audio Pipeline**
- Audio Capture → Processing (src/audio/processing.ts) → PCM chunks
f. **ASR Layer**
- ASR Manager (src/asr/manager.ts): orchestrates instances & Worker
- ASR Instance (src/asr/instance.ts): per-recording state
- Web Worker (src/asr/worker.ts): runs Whisper via WebGPU
- WebGPU & Transformers Model
g. **External Service**
- Model Download from Hugging Face on first run

##### Relationships & data flow
1. **UI Event** (click hotkey) → main.ts
2. **Audio Capture** → audio/processing.ts → PCM stream
3. **PCM** → postMessage → ASR Worker
4. **Worker** loads Whisper via WebGPU → transcribes → returns text
5. **Manager** receives transcript → updates UI component
6. **UI Layer** injects text into Cursor chat input

##### Architectural patterns & principles
- **Separation of Concerns**: UI vs. Audio vs. ASR logic vs. config/types
- **Asynchronous / Worker Offloading**: Heavy ML runs off the main thread
- **Modular TypeScript** organized by feature (ui, asr, audio, utils)
- **In-Browser Edge Computing**: Local processing for privacy

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Cursor Host Environment" 
        style CursorHost fill:#f3f0ff,stroke:#8c7ae6,stroke-width:2px
        CursorHost["Cursor Host Environment"]:::infra

        %% Injection Bundle
        subgraph "Injected Bundle" 
            style Bundle fill:#e2dbf0,stroke:#6c5ce7,stroke-width:2px
            Bundle["Injected Bundle (yap-for-cursor.js)"]:::infra

            %% Entry Point & Control
            EntryPoint["Entry Point & Bootstrap\n(src/main.ts)"]:::infra
            HotkeyManager["Hotkey Manager\n(src/utils/hotkey.ts)"]:::infra
            Config["Configuration Constants\n(src/config.ts)"]:::infra
            Types["Shared Types\n(src/types.ts)"]:::infra

            %% UI Layer
            subgraph "UI Layer"
                style UILayer fill:#dfe6e9,stroke:#0984e3,stroke-width:1px
                MicButton["Microphone Button"]:::ui
                ContextMenu["Context Menu"]:::ui
                DOMSelectors["DOM Selectors"]:::ui
                Styles["Styles/CSS"]:::ui
            end

            %% Audio Pipeline
            subgraph "Audio Pipeline"
                style AudioLayer fill:#fff5e6,stroke:#fdcb6e,stroke-width:1px
                AudioProcessing["Audio Capture & Preprocessing"]:::audio
            end

            %% ASR Layer
            subgraph "ASR Layer"
                style ASRLayer fill:#e8f9fd,stroke:#00b894,stroke-width:1px
                ASRManager["ASR Manager (orchestration)"]:::asr
                ASRInstance["ASR Instance\n(per-recording state)"]:::asr
                subgraph "Worker Thread"
                    style WorkerLayer fill:#e8f9fd,stroke:#00b894,stroke-width:1px,stroke-dasharray: 5 5
                    ASRWorker["Web Worker\n(src/asr/worker.ts)\nWhisper + WebGPU"]:::asr
                end
            end

        end

        ChatInput["Cursor Chat Input"]:::infra

        %% External Model Source
        ExternalModel["Hugging Face Model Download"]:::external
    end

    %% Build & Bundling Configuration
    subgraph "Build & Bundling"
        style BuildLayer fill:#f0f0f0,stroke:#636e72,stroke-width:1px
        BuildScript["Build Script\n(build.ts)"]:::infra
        Lockfile["Dependency Lockfile\n(bun.lock)"]:::infra
        PackageJson["Package Manifest\n(package.json)"]:::infra
        TSConfig["TypeScript Config\n(tsconfig.json)"]:::infra
    end

    %% Connections
    Bundle -->|contains| EntryPoint
    Bundle -->|contains| HotkeyManager
    Bundle -->|contains| Config
    Bundle -->|contains| Types

    EntryPoint -->|"wires UI & hotkey"| MicButton
    EntryPoint -->|wires| HotkeyManager
    HotkeyManager -->|"Cmd+Shift+Y"| MicButton
    MicButton -->|"click"| AudioProcessing
    AudioProcessing -->|"PCM audio frames"| ASRWorker
    ExternalModel -->|"one-time download"| ASRWorker
    ASRWorker -->|"transcript text"| ASRManager
    ASRManager -->|"update"| ContextMenu
    ContextMenu -->|"inject text"| ChatInput

    %% Bundle to Host
    CursorHost -->|"injects"| Bundle
    Bundle -->|"inserts text"| ChatInput

    %% Click Events
    click Bundle "https://github.com/avarayr/yap-for-cursor/blob/master/dist/yap-for-cursor.js"
    click EntryPoint "https://github.com/avarayr/yap-for-cursor/blob/master/src/main.ts"
    click MicButton "https://github.com/avarayr/yap-for-cursor/blob/master/src/ui/mic-button.ts"
    click ContextMenu "https://github.com/avarayr/yap-for-cursor/blob/master/src/ui/context-menu.ts"
    click DOMSelectors "https://github.com/avarayr/yap-for-cursor/blob/master/src/ui/dom-selectors.ts"
    click Styles "https://github.com/avarayr/yap-for-cursor/blob/master/src/styles/styles.css"
    click HotkeyManager "https://github.com/avarayr/yap-for-cursor/blob/master/src/utils/hotkey.ts"
    click Config "https://github.com/avarayr/yap-for-cursor/blob/master/src/config.ts"
    click Types "https://github.com/avarayr/yap-for-cursor/blob/master/src/types.ts"
    click AudioProcessing "https://github.com/avarayr/yap-for-cursor/blob/master/src/audio/processing.ts"
    click ASRManager "https://github.com/avarayr/yap-for-cursor/blob/master/src/asr/manager.ts"
    click ASRInstance "https://github.com/avarayr/yap-for-cursor/blob/master/src/asr/instance.ts"
    click ASRWorker "https://github.com/avarayr/yap-for-cursor/blob/master/src/asr/worker.ts"
    click BuildScript "https://github.com/avarayr/yap-for-cursor/blob/master/build.ts"
    click Lockfile "https://github.com/avarayr/yap-for-cursor/blob/master/bun.lock"
    click PackageJson "https://github.com/avarayr/yap-for-cursor/blob/master/package.json"
    click TSConfig "https://github.com/avarayr/yap-for-cursor/blob/master/tsconfig.json"

    %% Styles
    classDef ui fill:#d0e8ff,stroke:#0984e3,stroke-width:1px
    classDef audio fill:#fff0db,stroke:#fdcb6e,stroke-width:1px
    classDef asr fill:#e0f8f7,stroke:#00b894,stroke-width:1px
    classDef external fill:#dfe6e9,stroke:#b2bec3,stroke-width:1px
    classDef infra fill:#f5f5f5,stroke:#636e72,stroke-width:2px
```
