# ateeb09/nova architecture

> Architecture diagram of the GitHub repository ateeb09/nova, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ateeb09/nova
- Repository: https://github.com/ateeb09/nova

## Overview

1. **What this repo is**
   - This is a **local, offline voice-control agent** for Windows-style desktop automation.
   - The architecture is centered on a **single command pipeline**: wake word → record audio → speech-to-text → intent resolution → desktop action → spoken feedback.

2. **Two entry points, one shared automation core**
   - `voice_agent.py` is the **terminal-first main loop** described in the README.
   - `web_app.py` is the **web dashboard entry point**, with UI assets in `templates/` and `static/`.
   - Both entry points appear to sit on top of the same underlying voice/automation modules rather than duplicating the action logic.

3. **Voice processing subsystem**
   - `record.py` likely handles microphone capture and audio buffering.
   - `voice_loop.py` appears to encapsulate the main listen/process cycle.
   - `mic_test.py`, `list_mics.py`, `speak_test.py`, and `voice_test.py` are diagnostic/support scripts for audio input/output verification.
   - The README indicates speech recognition uses **Whisper**, and text-to-speech uses **pyttsx3** locally.

4. **Intent understanding and command routing**
   - `voice_action.py` is the main **command dispatcher**: it likely maps parsed intent to concrete actions.
   - The README’s “fuzzy intent matching” and “keyword catch” behavior suggests the dispatcher has a **rule-based fallback path** when LLM parsing is unavailable.
   - `voice_agent.py` likely orchestrates the decision order: **Gemini/Ollama LLM first when enabled, then rules/keywords, then execution**.

5. **Desktop automation layer**
   - `voice_mouse.py` is a dedicated module for **mouse operations** such as move, click, scroll, double-click, and right-click.
   - Keyboard, window, system, and application actions are likely grouped in `voice_action.py` and/or helper routines it calls.
   - The repo is built around local system control via libraries such as **pyautogui** and native OS subprocess calls.

6. **Screen understanding / OCR path**
   - `screen_read.py` is the screen-reading/OCR module.
   - `screen_test.py` and `vision_test.py` suggest a separate validation path for screen capture and OCR behavior.
   - The README identifies **Tesseract OCR** as the offline OCR dependency, so this subsystem is distinct from the speech pipeline and used only for screen interpretation tasks.

7. **Web dashboard subsystem**
   - `web_app.py` likely serves the browser UI and bridges the dashboard to the agent runtime.
   - `templates/index.html` is the main HTML template, while `static/css/style.css` and `static/js/app.js` provide the front-end behavior.
   - `index2.html` appears to be an alternate or experimental UI page, likely related to the dashboard/widget mode described in the README.

8. **Runtime boundaries**
   - The repo has a clear split between:
     - **Audio ingestion and STT**
     - **Intent resolution**
     - **OS-level action execution**
     - **UI presentation**
   - That separation is important for a diagram: the web UI is not the automation engine; it is a front-end over the same agent capabilities.

9. **External/local services and optional dependencies**
   - The system is designed to run **fully offline** by default.
   - Optional integrations affect architecture only when enabled:
     - **Ollama** for local LLM intent interpretation
     - **Gemini API** for cloud-backed intent understanding
     - **Tesseract** for OCR
   - The dependency list in `requirements.txt` likely pulls in the libraries for audio capture, STT, GUI automation, OCR, and web serving.

10. **Testing and validation structure**
   - The `*_test.py` scripts are not unit tests in the strict framework sense; they look like **manual or smoke-test utilities** for individual subsystems.
   - `automation_test.py` likely exercises the action layer end-to-end, while the other test files validate mic, speech, screen, and vision behavior independently.

11. **Likely data flow diagram to draw**
   - A useful architecture diagram would show:
     - **Microphone input** feeding the recording/preprocessing layer
     - **Speech recognition** converting audio to text
     - **Intent engine** choosing between LLM, rules, and keyword catch
     - **Action executor** dispatching mouse/keyboard/window/system/screen actions
     - **Speaker/TTS** returning status to the user
     - **Web dashboard** as an alternate control surface connected to the same backend capabilities

12. **Repository shape**
   - This is primarily a **single-machine automation application**, not a distributed system.
   - The major architectural boundary is between **frontend/dashboard** and **local agent runtime**, with all critical work happening on the local host.

Main entry points: [`voice_agent.py`](https://github.com/ateeb09/nova/blob/main/voice_agent.py) (Voice agent), [`web_app.py`](https://github.com/ateeb09/nova/blob/main/web_app.py) (Web app), [`voice_action.py`](https://github.com/ateeb09/nova/blob/main/voice_action.py) (Action router), [`record.py`](https://github.com/ateeb09/nova/blob/main/record.py) (Recorder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Local agent"]
  node_voice_agent["Voice agent<br/>orchestrator<br/>[voice_agent.py]"]
  node_voice_loop["Voice loop<br/>listen cycle<br/>[voice_loop.py]"]
  node_voice_action["Action router<br/>dispatcher<br/>[voice_action.py]"]
  node_voice_mouse["Mouse control<br/>pointer ops<br/>[voice_mouse.py]"]
end

subgraph group_ui["Web dashboard"]
  node_web_app["Web app<br/>dashboard server<br/>[web_app.py]"]
  node_templates_index["Template<br/>html view<br/>[index.html]"]
  node_static_js_app["UI script<br/>frontend js<br/>[app.js]"]
  node_static_css_style["Styles<br/>frontend css<br/>[style.css]"]
  node_index2_html["Alt page<br/>html view<br/>[index2.html]"]
end

subgraph group_perception["Perception"]
  node_record["Recorder<br/>audio capture<br/>[record.py]"]
  node_screen_read["Screen read<br/>ocr path<br/>[screen_read.py]"]
end

subgraph group_validation["Smoke tests"]
  node_automation_test["Automation test<br/>smoke script<br/>[automation_test.py]"]
  node_mic_test["Mic test<br/>smoke script<br/>[mic_test.py]"]
  node_list_mics["List mics<br/>utility script<br/>[list_mics.py]"]
  node_speak_test["Speak test<br/>smoke script<br/>[speak_test.py]"]
  node_voice_test["Voice test<br/>smoke script<br/>[voice_test.py]"]
  node_screen_test["Screen test<br/>smoke script<br/>[screen_test.py]"]
  node_vision_test["Vision test<br/>smoke script<br/>[vision_test.py]"]
end

node_voice_agent -->|"drives"| node_voice_loop
node_voice_loop -->|"captures"| node_record
node_voice_agent -->|"dispatches"| node_voice_action
node_voice_agent -.->|"listens"| node_record
node_voice_action -->|"pointer ops"| node_voice_mouse
node_voice_action -->|"reads screen"| node_screen_read
node_voice_agent -.->|"optional vision"| node_screen_read
node_web_app -->|"serves"| node_templates_index
node_web_app -->|"loads"| node_static_js_app
node_web_app -->|"loads"| node_static_css_style
node_web_app -.->|"alternative view"| node_index2_html
node_web_app -->|"bridges"| node_voice_agent
node_automation_test -->|"exercises"| node_voice_action
node_mic_test -->|"checks"| node_record
node_list_mics -.->|"inspects"| node_record
node_speak_test -.->|"verifies"| node_voice_agent
node_voice_test -->|"validates"| node_voice_loop
node_screen_test -->|"validates"| node_screen_read
node_vision_test -.->|"probes"| node_screen_read

click node_voice_agent "https://github.com/ateeb09/nova/blob/main/voice_agent.py"
click node_voice_loop "https://github.com/ateeb09/nova/blob/main/voice_loop.py"
click node_record "https://github.com/ateeb09/nova/blob/main/record.py"
click node_voice_action "https://github.com/ateeb09/nova/blob/main/voice_action.py"
click node_voice_mouse "https://github.com/ateeb09/nova/blob/main/voice_mouse.py"
click node_screen_read "https://github.com/ateeb09/nova/blob/main/screen_read.py"
click node_web_app "https://github.com/ateeb09/nova/blob/main/web_app.py"
click node_templates_index "https://github.com/ateeb09/nova/blob/main/templates/index.html"
click node_static_js_app "https://github.com/ateeb09/nova/blob/main/static/js/app.js"
click node_static_css_style "https://github.com/ateeb09/nova/blob/main/static/css/style.css"
click node_index2_html "https://github.com/ateeb09/nova/blob/main/index2.html"
click node_automation_test "https://github.com/ateeb09/nova/blob/main/automation_test.py"
click node_mic_test "https://github.com/ateeb09/nova/blob/main/mic_test.py"
click node_list_mics "https://github.com/ateeb09/nova/blob/main/list_mics.py"
click node_speak_test "https://github.com/ateeb09/nova/blob/main/speak_test.py"
click node_voice_test "https://github.com/ateeb09/nova/blob/main/voice_test.py"
click node_screen_test "https://github.com/ateeb09/nova/blob/main/screen_test.py"
click node_vision_test "https://github.com/ateeb09/nova/blob/main/vision_test.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_voice_agent,node_voice_loop,node_voice_action,node_voice_mouse toneBlue
class node_web_app,node_templates_index,node_static_js_app,node_static_css_style,node_index2_html toneAmber
class node_record,node_screen_read toneMint
class node_automation_test,node_mic_test,node_list_mics,node_speak_test,node_voice_test,node_screen_test,node_vision_test toneRose
```

## Components

### Local agent

Offline voice-control runtime and desktop automation core.

- [Voice agent](https://github.com/ateeb09/nova/blob/main/voice_agent.py): orchestrator · Main terminal entry point that coordinates listening, intent resolution, execution, and feedback. · `voice_agent.py`
- [Voice loop](https://github.com/ateeb09/nova/blob/main/voice_loop.py): listen cycle · Encapsulates the repeated listen/process loop for the agent. · `voice_loop.py`
- [Action router](https://github.com/ateeb09/nova/blob/main/voice_action.py): dispatcher · Maps recognized intent to concrete desktop actions and fallback rules. · `voice_action.py`
- [Mouse control](https://github.com/ateeb09/nova/blob/main/voice_mouse.py): pointer ops · Dedicated mouse movement, click, scroll, and drag operations. · `voice_mouse.py`

### Web dashboard

Browser-facing control surface and its static assets.

- [Web app](https://github.com/ateeb09/nova/blob/main/web_app.py): dashboard server · Browser entry point that exposes the agent through a web dashboard. · `web_app.py`
- [Template](https://github.com/ateeb09/nova/blob/main/templates/index.html): html view · Main dashboard HTML template. · `templates/index.html`
- [UI script](https://github.com/ateeb09/nova/blob/main/static/js/app.js): frontend js · Dashboard client behavior and browser-side interaction logic. · `static/js/app.js`
- [Styles](https://github.com/ateeb09/nova/blob/main/static/css/style.css): frontend css · Dashboard styling rules. · `static/css/style.css`
- [Alt page](https://github.com/ateeb09/nova/blob/main/index2.html): html view · Alternate or experimental dashboard page. · `index2.html`

### Perception

Audio capture, speech recognition, and screen-reading paths.

- [Recorder](https://github.com/ateeb09/nova/blob/main/record.py): audio capture · Captures microphone input and buffers audio for transcription. · `record.py`
- [Screen read](https://github.com/ateeb09/nova/blob/main/screen_read.py): ocr path · Reads the screen and performs OCR-based interpretation when needed. · `screen_read.py`

### Smoke tests

Manual scripts used to exercise individual subsystems.

- [Automation test](https://github.com/ateeb09/nova/blob/main/automation_test.py): smoke script · End-to-end exercise of the action layer. · `automation_test.py`
- [Mic test](https://github.com/ateeb09/nova/blob/main/mic_test.py): smoke script · Manual microphone capture check. · `mic_test.py`
- [List mics](https://github.com/ateeb09/nova/blob/main/list_mics.py): utility script · Enumerates available audio input devices. · `list_mics.py`
- [Speak test](https://github.com/ateeb09/nova/blob/main/speak_test.py): smoke script · Verifies spoken feedback / TTS output. · `speak_test.py`
- [Voice test](https://github.com/ateeb09/nova/blob/main/voice_test.py): smoke script · Checks the speech pipeline end to end. · `voice_test.py`
- [Screen test](https://github.com/ateeb09/nova/blob/main/screen_test.py): smoke script · Validates screen capture and OCR behavior. · `screen_test.py`
- [Vision test](https://github.com/ateeb09/nova/blob/main/vision_test.py): smoke script · Auxiliary validation for screen interpretation. · `vision_test.py`

## Connections

- Voice agent → Voice loop: drives · Runs the repeated listen/process cycle.
- Voice loop → Recorder: captures · Pulls audio from the microphone for processing.
- Voice agent → Action router: dispatches · Sends parsed intent into the command router.
- Voice agent → Recorder: listens · Coordinates audio intake for the main loop.
- Action router → Mouse control: pointer ops · Delegates mouse-specific desktop actions.
- Action router → Screen read: reads screen · Uses OCR/screen interpretation when needed.
- Voice agent → Screen read: optional vision · Can invoke screen reading as part of command handling.
- Web app → Template: serves · Renders the main dashboard template.
- Web app → UI script: loads · Provides browser-side interaction logic.
- Web app → Styles: loads · Provides dashboard styling.
- Web app → Alt page: alternative view · Supports the alternate dashboard page.
- Web app → Voice agent: bridges · Connects the web dashboard to the local agent runtime.
- Automation test → Action router: exercises · Smoke-tests the desktop action layer.
- Mic test → Recorder: checks · Validates microphone capture.
- List mics → Recorder: inspects · Enumerates devices used by audio capture.
- Speak test → Voice agent: verifies · Checks spoken feedback path.
- Voice test → Voice loop: validates · Exercises the main voice processing loop.
- Screen test → Screen read: validates · Checks OCR/screen capture behavior.
- Vision test → Screen read: probes · Additional screen interpretation smoke test.