# harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction architecture

> Architecture diagram of the GitHub repository harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction
- Repository: https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction

## Overview

This is a voice-controlled browser extension with a local Flask speech-recognition backend. The main workflow is: the user toggles listening in the popup, the extension stores that state, a content script polls the backend for recognized speech, and matching commands act on the current page or navigate to a search or site. The README also describes hands-free navigation; sampled content code implements scrolling, typing, clicking, selection, and page reading. An offscreen Web Speech path exists in the tree, but its integration with the polling workflow is not shown.

Main entry points: [`voice-backend/app.py`](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py) (Flask endpoints), [`voice-extension/content.js`](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js) (Page listener), [`voice-extension/popup.js`](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/popup.js) (Listening popup), [`voice-extension/offscreen.js`](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/offscreen.js) (Web Speech capture)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Listening control"]
  node_popup["Listening popup<br/>[popup.js]"]
  node_state[("Listening state<br/>[popup.js]")]
end

subgraph group_recognition["Speech recognition"]
  node_backend["Flask endpoints<br/>[app.py]"]
  node_microphone["Microphone capture<br/>[app.py]"]
  node_speech_service["Google recognition<br/>[app.py]"]
  node_offscreen["Web Speech capture<br/>[offscreen.js]"]
end

subgraph group_commands["Command handling"]
  node_page_script["Page listener<br/>[content.js]"]
  node_command_dispatch["Command dispatch<br/>[content.js]"]
  node_page_actions["Page actions<br/>[content.js]"]
end

subgraph group_extension["Extension runtime"]
  node_background["Background listener<br/>[background.js]"]
end

node_user(("User"))
node_browser{{"Browser APIs"}}
node_web(("Web pages"))

node_user -->|"toggles listening"| node_popup
node_popup -->|"starts or stops"| node_backend
node_popup -->|"updates state"| node_state
node_state -->|"notifies changes"| node_page_script
node_page_script -->|"polls commands"| node_backend
node_backend -->|"captures audio"| node_microphone
node_microphone -->|"recognizes speech"| node_speech_service
node_backend -->|"returns transcript"| node_page_script
node_page_script -->|"dispatches transcript"| node_command_dispatch
node_command_dispatch -->|"executes commands"| node_page_actions
node_page_actions -->|"acts on page"| node_web
node_offscreen -->|"sends transcript"| node_browser
node_background -->|"handles toggle requests"| node_backend

click node_popup "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/popup.js"
click node_state "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/popup.js"
click node_page_script "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js"
click node_command_dispatch "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js"
click node_page_actions "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js"
click node_backend "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py"
click node_microphone "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py"
click node_speech_service "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py"
click node_offscreen "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/offscreen.js"
click node_background "https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/background.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_popup,node_state,node_user,node_browser toneBlue
class node_backend,node_microphone,node_speech_service,node_offscreen toneAmber
class node_page_script,node_command_dispatch,node_page_actions toneMint
class node_background toneRose
class node_web toneIndigo
```

## Components

### Listening control

- [Listening popup](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/popup.js): `voice-extension/popup.js`
- [Listening state](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/popup.js): `voice-extension/popup.js`

### Speech recognition

- [Flask endpoints](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py): `voice-backend/app.py`
- [Microphone capture](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py): `voice-backend/app.py`
- [Google recognition](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-backend/app.py): `voice-backend/app.py`
- [Web Speech capture](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/offscreen.js): `voice-extension/offscreen.js`

### Command handling

- [Page listener](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js): `voice-extension/content.js`
- [Command dispatch](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js): `voice-extension/content.js`
- [Page actions](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/content.js): `voice-extension/content.js`

### Extension runtime

- [Background listener](https://github.com/harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction/blob/main/voice-extension/background.js): `voice-extension/background.js`

### Other components

- **User**
- **Browser APIs**
- **Web pages**

## Connections

- User → Listening popup: toggles listening (evidence: `voice-extension/popup.js`)
- Listening popup → Flask endpoints: starts or stops (evidence: `voice-extension/popup.js`)
- Listening popup → Listening state: updates state (evidence: `voice-extension/popup.js`)
- Listening state → Page listener: notifies changes (evidence: `voice-extension/content.js`)
- Page listener → Flask endpoints: polls commands (evidence: `voice-extension/content.js`)
- Flask endpoints → Microphone capture: captures audio (evidence: `voice-backend/app.py`)
- Microphone capture → Google recognition: recognizes speech (evidence: `voice-backend/app.py`)
- Flask endpoints → Page listener: returns transcript (evidence: `voice-extension/content.js`)
- Page listener → Command dispatch: dispatches transcript (evidence: `voice-extension/content.js`)
- Command dispatch → Page actions: executes commands (evidence: `voice-extension/content.js`)
- Page actions → Web pages: acts on page (evidence: `voice-extension/content.js`)
- Web Speech capture → Browser APIs: sends transcript (evidence: `voice-extension/offscreen.js`)
- Background listener → Flask endpoints: handles toggle requests (evidence: `voice-extension/background.js`)
