How anshul-sys32/voice-controlled-system works
This is a voice-controlled desktop automation application. Its main workflow captures microphone audio, detects and buffers speech, transcribes it, parses intent and entities, then plans a desktop action. The planner can locate on-screen text using a screenshot and OCR before clicking, or start/stop scrolling. Source excerpts support this workflow and the transcription/classification internals; other listed modules are not mapped where their runtime wiring was not shown.
Main entry points: main.py Runtime loop · action_planner.py Action planner · transcriber.py Transcriber · command_parser.py Command parser
Components 18
Audio capture
- Microphone stream
audio/mic_input.py - Voice detection
audio/vad.py - Speech buffer
audio/speech_buffer.py
Speech recognition
- Transcriber
speech_to_text/transcriber.py - Whisper STT
speech_to_text/whisper_stt.py - Confidence filter
speech_to_text/confidence_filter.py
Command understanding
- Command parser
nlp/command_parser.py - Intent classifier
nlp/intent_classifier.py - Entity extractor
nlp/entity_extractor.py
Screen understanding
- Screen capture
vision/screen_capture.py - OCR reader
vision/ocr_reader.py - Text locator
vision/text_locator.py
Desktop actions
- Action planner
planner/action_planner.py - Mouse controller
automation/mouse_controller.py
Other components
- Voice user
- Runtime loop
main.py - Desktop
- Audio input
Connections 20
- Voice user → Audio input: speaks
- Audio input → Microphone stream: audio input
- Runtime loop → Microphone stream: reads audio
- Runtime loop → Voice detection: checks speech
- Runtime loop → Speech buffer: buffers chunks
- Runtime loop → Transcriber: transcribes speech
- Runtime loop → Command parser: parses text
- Runtime loop → Action planner: executes command
- Transcriber → Whisper STT: transcribes
- Transcriber → Confidence filter: checks confidence
- Command parser → Intent classifier: classifies intent
- Command parser → Entity extractor: extracts entities
- Action planner → Screen capture: captures screen
- Action planner → OCR reader: reads detections
- Action planner → Text locator: locates text
- Action planner → Mouse controller: moves and clicks
- Action planner → Mouse controller: starts or stops scroll
- Microphone stream → Audio input: reads stream
- Screen capture → Desktop: captures display
- Mouse controller → Desktop: controls input