How harri03/ai-voice-enabled-browser-extension-for-accessible-web-interaction works
This is a voice-controlled browser extension with a local Flask speech-recognition backend. The main workflow is: the user toggles listening in the popup, the extension stores that state, a content script polls the backend for recognized speech, and matching commands act on the current page or navigate to a search or site. The README also describes hands-free navigation; sampled content code implements scrolling, typing, clicking, selection, and page reading. An offscreen Web Speech path exists in the tree, but its integration with the polling workflow is not shown.
Main entry points: app.py Flask endpoints · content.js Page listener · popup.js Listening popup · offscreen.js Web Speech capture
Components 13
Listening control
- Listening popup
voice-extension/popup.js - Listening state
voice-extension/popup.js
Speech recognition
- Flask endpoints
voice-backend/app.py - Microphone capture
voice-backend/app.py - Google recognition
voice-backend/app.py - Web Speech capture
voice-extension/offscreen.js
Command handling
- Page listener
voice-extension/content.js - Command dispatch
voice-extension/content.js - Page actions
voice-extension/content.js
Extension runtime
- Background listener
voice-extension/background.js
Other components
- User
- Browser APIs
- Web pages
Connections 13
- User → Listening popup: toggles listening
voice-extension/popup.js - Listening popup → Flask endpoints: starts or stops
voice-extension/popup.js - Listening popup → Listening state: updates state
voice-extension/popup.js - Listening state → Page listener: notifies changes
voice-extension/content.js - Page listener → Flask endpoints: polls commands
voice-extension/content.js - Flask endpoints → Microphone capture: captures audio
voice-backend/app.py - Microphone capture → Google recognition: recognizes speech
voice-backend/app.py - Flask endpoints → Page listener: returns transcript
voice-extension/content.js - Page listener → Command dispatch: dispatches transcript
voice-extension/content.js - Command dispatch → Page actions: executes commands
voice-extension/content.js - Page actions → Web pages: acts on page
voice-extension/content.js - Web Speech capture → Browser APIs: sends transcript
voice-extension/offscreen.js - Background listener → Flask endpoints: handles toggle requests
voice-extension/background.js