# subramanyajagadeesh/whispercode architecture

> Architecture diagram of the GitHub repository subramanyajagadeesh/whispercode, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/subramanyajagadeesh/whispercode
- Repository: https://github.com/subramanyajagadeesh/whispercode

## Overview

##### Project Type & Purpose
- This is a VS Code extension designed for speech-to-code conversion. Its purpose is to allow developers to speak their code, which is then transcribed and formatted using OpenAI’s Whisper (for speech-to-text) and GPT-4 (for language-specific formatting).
- The README highlights features like secure API key management, multi-language support, real-time feedback, and integration with external audio tools (e.g., SoX).

##### File Structure
- At the top level, you have configuration files (e.g., tsconfig.json, package.json, eslint.config.mjs) and developer utilities (e.g., .vscode, .gitignore).
- The main logic lives in the /src directory:
- extension.ts and extension_with_py.ts – likely providing the entry points for the extension, one for standard TypeScript code and one integrating Python (as indicated by the presence of speech_to_text.py).
- /utils folder – contains helper modules such as audio.ts (likely handling audio processing details), code.ts (managing code formatting or conversion logic), file.ts (for file system interactions), and keys.ts (for secure API key management).
- /test folder – includes tests (extension.test.ts) ensuring the extension’s functionality.
- The presence of a Python file (speech_to_text.py) suggests that part of the speech transcription might be offloaded to Python code (possibly leveraging external libraries or scripts).

a. Main Components:
- VS Code Extension UI/Activation:
- Represented by extension.ts and extension_with_py.ts.
- Interfaces with VS Code's Command Palette and notifications (real-time status, API validation, etc.).
- Audio Capture & Processing:
- Possibly interfaced with SoX (as noted in README) and handled in src/utils/audio.ts.
- Integration with speech_to_text.py for converting audio input to text.
- AI API Integration:
- Communication with OpenAI's Whisper (for ASR) and GPT-4 (for language-specific code formatting).
- API Key & Configuration Management:
- Managed through src/utils/keys.ts and likely configured securely within VS Code.
- Utility Modules:
- Additional helpers (file handling, code formatting, etc.) from src/utils/code.ts and src/utils/file.ts.
- Testing Framework:
- Test components in src/test ensuring that each module and overall functionality is performing as expected.

b. Relationships & Data Flow:
- The user interacts with the VS Code UI to activate the command.
- Once activated, the extension captures audio input.
- Audio data is processed (possibly with SoX and the Python script for transcription) via the audio module.
- The transcribed text is sent to the GPT-4 API for context-aware code formatting.
- The formatted code is then inserted back into the current open file in VS Code.
- API key management is done securely, ensuring safe access to OpenAI services.
- Logs, configurations, and errors are managed throughout the process.

c. Architectural Patterns and Important Details:
- Modular design: Clear separation between UI/extension activation code, utilities, and external integrations.
- Client interaction with external asynchronous APIs (Whisper and GPT-4).
- Usage of both TypeScript (for the extension) and Python (for speech-to-text processing) might indicate inter-process communication or a hybrid system design.
- Emphasis on real-time feedback and secure key management.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global Entities
    U("User Action: Initiate Voice Code"):::user

    %% VS Code Extension Layer
    subgraph "VS Code Extension Layer"
        EXT_A("Extension Activation (TypeScript)"):::ui
        EXT_B("Extension Activation (Python Integration)"):::ui
    end

    %% Audio Capture & Processing Module
    subgraph "Audio Processing Module"
        AUD("Audio Processing (audio.ts)"):::audio
        SPCH("Speech-to-Text Processing (speech_to_text.py)"):::audio
    end

    %% AI API Integration Module
    subgraph "AI API Integration"
        WHISPER("Whisper ASR Service"):::external
        GPT4("GPT-4 Formatting Service"):::external
    end

    %% Utility Modules
    subgraph "Utility Modules"
        API_KEYS("API Key & Configuration (keys.ts)"):::utility
        CODE_UTIL("Code Formatting Utility (code.ts)"):::utility
        FILE_UTIL("File Handling Utility (file.ts)"):::utility
    end

    %% Testing Framework
    subgraph "Testing Framework"
        TEST("Extension Tests (extension.test.ts)"):::test
    end

    %% Connections - Data Flow
    U -->|"activateExtension"| EXT_A
    U -->|"activateExtension"| EXT_B
    EXT_A -->|"captureAudio"| AUD
    EXT_B -->|"captureAudio"| AUD
    AUD -->|"processAudio"| SPCH
    SPCH -->|"transcribeAudio"| WHISPER
    WHISPER -->|"returnTranscription"| GPT4
    GPT4 -->|"formatCode"| CODE_UTIL
    CODE_UTIL -->|"prepareInsertion"| FILE_UTIL
    FILE_UTIL -->|"updateFile"| EXT_A
    FILE_UTIL -->|"updateFile"| EXT_B
    API_KEYS -.->|"secureAccess"| EXT_A
    API_KEYS -.->|"secureAccess"| EXT_B

    %% Testing interactions
    TEST -->|"validateExtension"| EXT_A
    TEST -->|"validateAudio"| AUD
    TEST -->|"validateConfig"| API_KEYS

    %% Click Events
    click EXT_A "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/extension.ts"
    click EXT_B "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/extension_with_py.ts"
    click AUD "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/utils/audio.ts"
    click SPCH "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/speech_to_text.py"
    click API_KEYS "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/utils/keys.ts"
    click CODE_UTIL "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/utils/code.ts"
    click FILE_UTIL "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/utils/file.ts"
    click TEST "https://github.com/subramanyajagadeesh/whispercode/blob/main/src/test/extension.test.ts"

    %% Styles Definitions
    classDef user fill:#FFE4B5,stroke:#333,stroke-width:2px;
    classDef ui fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef audio fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef external fill:#FFDAB9,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
    classDef utility fill:#F0E68C,stroke:#333,stroke-width:2px;
    classDef test fill:#D3D3D3,stroke:#333,stroke-width:2px;
```
