# anshul-sys32/voice-controlled-system architecture

> Architecture diagram of the GitHub repository anshul-sys32/voice-controlled-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anshul-sys32/voice-controlled-system
- Repository: https://github.com/anshul-sys32/voice-controlled-system

## Overview

This is a voice-controlled desktop automation application. Its main workflow captures microphone audio, detects and buffers speech, transcribes it, parses intent and entities, then plans a desktop action. The planner can locate on-screen text using a screenshot and OCR before clicking, or start/stop scrolling. Source excerpts support this workflow and the transcription/classification internals; other listed modules are not mapped where their runtime wiring was not shown.

Main entry points: [`main.py`](https://github.com/anshul-sys32/voice-controlled-system/blob/main/main.py) (Runtime loop), [`planner/action_planner.py`](https://github.com/anshul-sys32/voice-controlled-system/blob/main/planner/action_planner.py) (Action planner), [`speech_to_text/transcriber.py`](https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/transcriber.py) (Transcriber), [`nlp/command_parser.py`](https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/command_parser.py) (Command parser)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_audio["Audio capture"]
  node_mic["Microphone stream<br/>[mic_input.py]"]
  node_vad["Voice detection<br/>[vad.py]"]
  node_buffer["Speech buffer<br/>[speech_buffer.py]"]
end

subgraph group_speech["Speech recognition"]
  node_transcriber["Transcriber<br/>[transcriber.py]"]
  node_whisper["Whisper STT<br/>[whisper_stt.py]"]
  node_confidence["Confidence filter"]
end

subgraph group_language["Command understanding"]
  node_parser["Command parser<br/>[command_parser.py]"]
  node_intent["Intent classifier"]
  node_entities["Entity extractor"]
end

subgraph group_vision["Screen understanding"]
  node_capture["Screen capture<br/>[screen_capture.py]"]
  node_ocr["OCR reader<br/>[ocr_reader.py]"]
  node_locator["Text locator<br/>[text_locator.py]"]
end

subgraph group_actions["Desktop actions"]
  node_planner["Action planner<br/>[action_planner.py]"]
  node_mouse["Mouse controller"]
end

node_user(("Voice user"))
node_main["Runtime loop<br/>[main.py]"]
node_desktop(("Desktop"))
node_microphone(("Audio input"))

node_user -->|"speaks"| node_microphone
node_microphone -->|"audio input"| node_mic
node_main -->|"reads audio"| node_mic
node_main -->|"checks speech"| node_vad
node_main -->|"buffers chunks"| node_buffer
node_main -->|"transcribes speech"| node_transcriber
node_main -->|"parses text"| node_parser
node_main -->|"executes command"| node_planner
node_transcriber -->|"transcribes"| node_whisper
node_transcriber -->|"checks confidence"| node_confidence
node_parser -->|"classifies intent"| node_intent
node_parser -->|"extracts entities"| node_entities
node_planner -->|"captures screen"| node_capture
node_planner -->|"reads detections"| node_ocr
node_planner -->|"locates text"| node_locator
node_planner -->|"moves and clicks"| node_mouse
node_planner -->|"starts or stops scroll"| node_mouse
node_mic -->|"reads stream"| node_microphone
node_capture -->|"captures display"| node_desktop
node_mouse -->|"controls input"| node_desktop

click node_main "https://github.com/anshul-sys32/voice-controlled-system/blob/main/main.py"
click node_mic "https://github.com/anshul-sys32/voice-controlled-system/blob/main/audio/mic_input.py"
click node_vad "https://github.com/anshul-sys32/voice-controlled-system/blob/main/audio/vad.py"
click node_buffer "https://github.com/anshul-sys32/voice-controlled-system/blob/main/audio/speech_buffer.py"
click node_transcriber "https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/transcriber.py"
click node_whisper "https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/whisper_stt.py"
click node_confidence "https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/confidence_filter.py"
click node_parser "https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/command_parser.py"
click node_intent "https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/intent_classifier.py"
click node_entities "https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/entity_extractor.py"
click node_planner "https://github.com/anshul-sys32/voice-controlled-system/blob/main/planner/action_planner.py"
click node_capture "https://github.com/anshul-sys32/voice-controlled-system/blob/main/vision/screen_capture.py"
click node_ocr "https://github.com/anshul-sys32/voice-controlled-system/blob/main/vision/ocr_reader.py"
click node_locator "https://github.com/anshul-sys32/voice-controlled-system/blob/main/vision/text_locator.py"
click node_mouse "https://github.com/anshul-sys32/voice-controlled-system/blob/main/automation/mouse_controller.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_mic,node_vad,node_buffer,node_user toneBlue
class node_transcriber,node_whisper,node_confidence toneAmber
class node_parser,node_intent,node_entities toneMint
class node_capture,node_ocr,node_locator toneRose
class node_planner,node_mouse,node_desktop,node_microphone toneIndigo
class node_main toneTeal
```

## Components

### Audio capture

- [Microphone stream](https://github.com/anshul-sys32/voice-controlled-system/blob/main/audio/mic_input.py): `audio/mic_input.py`
- [Voice detection](https://github.com/anshul-sys32/voice-controlled-system/blob/main/audio/vad.py): `audio/vad.py`
- [Speech buffer](https://github.com/anshul-sys32/voice-controlled-system/blob/main/audio/speech_buffer.py): `audio/speech_buffer.py`

### Speech recognition

- [Transcriber](https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/transcriber.py): `speech_to_text/transcriber.py`
- [Whisper STT](https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/whisper_stt.py): `speech_to_text/whisper_stt.py`
- [Confidence filter](https://github.com/anshul-sys32/voice-controlled-system/blob/main/speech_to_text/confidence_filter.py): `speech_to_text/confidence_filter.py`

### Command understanding

- [Command parser](https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/command_parser.py): `nlp/command_parser.py`
- [Intent classifier](https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/intent_classifier.py): `nlp/intent_classifier.py`
- [Entity extractor](https://github.com/anshul-sys32/voice-controlled-system/blob/main/nlp/entity_extractor.py): `nlp/entity_extractor.py`

### Screen understanding

- [Screen capture](https://github.com/anshul-sys32/voice-controlled-system/blob/main/vision/screen_capture.py): `vision/screen_capture.py`
- [OCR reader](https://github.com/anshul-sys32/voice-controlled-system/blob/main/vision/ocr_reader.py): `vision/ocr_reader.py`
- [Text locator](https://github.com/anshul-sys32/voice-controlled-system/blob/main/vision/text_locator.py): `vision/text_locator.py`

### Desktop actions

- [Action planner](https://github.com/anshul-sys32/voice-controlled-system/blob/main/planner/action_planner.py): `planner/action_planner.py`
- [Mouse controller](https://github.com/anshul-sys32/voice-controlled-system/blob/main/automation/mouse_controller.py): `automation/mouse_controller.py`

### Other components

- **Voice user**
- [Runtime loop](https://github.com/anshul-sys32/voice-controlled-system/blob/main/main.py): `main.py`
- **Desktop**
- **Audio input**

## Connections

- Voice user → Audio input: speaks
- Audio input → Microphone stream: audio input
- Runtime loop → Microphone stream: reads audio
- Runtime loop → Voice detection: checks speech
- Runtime loop → Speech buffer: buffers chunks
- Runtime loop → Transcriber: transcribes speech
- Runtime loop → Command parser: parses text
- Runtime loop → Action planner: executes command
- Transcriber → Whisper STT: transcribes
- Transcriber → Confidence filter: checks confidence
- Command parser → Intent classifier: classifies intent
- Command parser → Entity extractor: extracts entities
- Action planner → Screen capture: captures screen
- Action planner → OCR reader: reads detections
- Action planner → Text locator: locates text
- Action planner → Mouse controller: moves and clicks
- Action planner → Mouse controller: starts or stops scroll
- Microphone stream → Audio input: reads stream
- Screen capture → Desktop: captures display
- Mouse controller → Desktop: controls input