# dawoodtouseef/j.ar.v.i.s. architecture

> Architecture diagram of the GitHub repository dawoodtouseef/j.ar.v.i.s., generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dawoodtouseef/j.ar.v.i.s.
- Repository: https://github.com/dawoodtouseef/j.ar.v.i.s.

## Overview

#### Purpose and launch surfaces
This is a Windows-oriented Python desktop automation assistant (“JARVIS”) supporting voice, text, camera, screenshot, and system-status interactions. Primary launch paths are the desktop application (`main.py`, `jarvis.py`, `run_jarvis.bat`), with selectable GPT All Star coding and Open Interpreter modes. The UI is a PyQt-based desktop client rooted in `gui/AssistantGUI.py` and `gui/Home.py`.

#### Desktop interaction and background workers
The GUI coordinates user-facing screens, terminal access, settings, login, and integrations. Background objects handle agent execution, alerts, speech recognition, TTS, wake-word activation, and consciousness/monitoring behavior. Key boundaries:
- Presentation/workers: `gui/AssistantGUI.py`, `gui/Objects/AgentWorker.py`, `gui/Objects/SpeechRecognition.py`
- Voice lifecycle: `gui/Objects/wakeword.py`, `gui/Objects/TTSWorker.py`
- Embedded terminal subsystem: `gui/terminal/terminal_manager.py`, `gui/terminal/virtual_filesystem.py`

#### Assistant brain, agent routing, and memory
The core assistant layer turns requests and sensed context into delegated agent work. It defines conversational, planning, proactive, security, financial, Gmail, vision, and personal-assistant roles; shared agent state provides the control/data boundary between orchestration and specialized actions. Persistent conversational memory is isolated under `core/memory/`.
- Orchestration: `core/brain.py`, `core/agents/state.py`
- Agent roles: `core/agents/ConversationalAgent.py`, `core/agents/planner_agent.py`, `core/agents/vision_agents.py`
- Persistence: `core/memory/memory_agent.py`

#### Automation, sensing, and external capabilities
Tools encapsulate OS-facing and external integrations: system health/sensors, scheduling, routing, web/news access, calculator/charting, home automation, software management, and stop control. This creates a practical boundary: agents decide intent; tools perform side effects and return observations.
- Tool catalog and routing: `core/tools/router.py`, `core/tools/standard_tools.py`
- Monitoring and proactive signals: `core/tools/health_monitor.py`, `core/tools/sensors.py`, `core/tools/consciousness.py`
- Automation integrations: `core/tools/scheduler.py`, `core/tools/home_assistant_tool.py`, `core/tools/news_scraper.py`

#### Computer-use interpreter runtime
The bundled Open Interpreter-style runtime offers an LLM-driven tool loop for desktop control. Its core resolves LLM responses/tool calls, then invokes browser, filesystem, terminal, display, keyboard, mouse, clipboard, contacts, calendar, mail, and vision adapters. It can be used independently through a terminal interface.
- Control loop and LLM dispatch: `interpreter/core/core.py`, `interpreter/core/llm/llm.py`, `interpreter/computer_use/loop.py`
- Computer abstraction: `interpreter/core/computer/computer.py`, `interpreter/core/computer/terminal/terminal.py`
- CLI/profiles: `interpreter/terminal_interface/terminal_interface.py`, `interpreter/terminal_interface/profiles/profiles.py`

#### GPT All Star coding pipeline
A separate multi-agent software-delivery workflow converts a coding request into staged specification, system/UI design, development, QA, healing, execution, and deployment. Role-specific agents collaborate through chains/team state; tools enable files, shell commands, and document/index retrieval. `components/coder/` and `components/gpt_all_star/` are near-parallel implementations and should be treated as duplicated architectural surfaces unless consolidated.
- Entrypoints/orchestration: `components/coder/main.py`, `components/coder/core/gpt_all_star.py`
- Roles and collaboration: `components/coder/core/agents/agents.py`, `components/coder/core/team.py`
- Pipeline stages: `components/coder/core/steps/steps.py`, `components/coder/core/steps/development/development.py`, `components/coder/core/steps/quality_assurance/quality_assurance.py`
- Execution boundary: `components/coder/core/execution/execution.py`, `components/coder/core/tools/shell_tool.py`

#### Infrastructure and persistence
The application is Python 3.12+ and Windows 10/11-focused, using PyQt5 and OpenCV per project documentation. Audio providers include OpenAI and local Whisper STT plus multiple TTS implementations. The integration layer contains database models and Alembic migrations, indicating persisted users, preferences, command history, contacts, agents, tools, sensors, and environment variables.
- Audio provider boundary: `jarvis_integration/audio/stt_providers/openai.py`, `jarvis_integration/audio/tts_providers/BaseTTS.py`
- Data models/database: `jarvis_integration/internals/db.py`, `jarvis_integration/models/command_history.py`
- Schema evolution: `alembic.ini`, `migrations/env.py`

Main entry points: [`core/brain.py`](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/brain.py) (Assistant brain), [`gui/AssistantGUI.py`](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/AssistantGUI.py) (PyQt assistant UI), [`main.py`](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/main.py) (Desktop launchers), [`gui/Objects/SpeechRecognition.py`](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/SpeechRecognition.py) (Speech input)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_desktop["Desktop client"]
  node_launch["Desktop launchers<br/>entrypoints<br/>[main.py]"]
  node_gui["PyQt assistant UI<br/>desktop UI<br/>[AssistantGUI.py]"]
  node_agent_worker["Agent worker<br/>background worker<br/>[AgentWorker.py]"]
  node_speech_input["Speech input<br/>voice worker"]
  node_tts["Speech output<br/>TTS worker<br/>[TTSWorker.py]"]
  node_terminal["Embedded terminal<br/>terminal subsystem"]
end

subgraph group_assistant["Assistant core"]
  node_brain{{"Assistant brain<br/>orchestrator<br/>[brain.py]"}}
  node_agent_state["Shared agent state<br/>state boundary<br/>[state.py]"]
  node_agent_roles["Specialist agents<br/>agent roles"]
  node_memory[("Conversation memory<br/>persistent memory<br/>[memory_agent.py]")]
  node_tool_router["Tool router<br/>capability boundary<br/>[router.py]"]
  node_monitoring["Monitoring signals<br/>proactive sensing<br/>[health_monitor.py]"]
  node_automation["Automation integrations<br/>side-effect tools<br/>[scheduler.py]"]
end

subgraph group_interpreter["Computer-use runtime"]
  node_interpreter_loop{{"Interpreter control loop<br/>LLM tool loop<br/>[core.py]"}}
  node_computer["Computer adapters<br/>desktop control<br/>[computer.py]"]
  node_interpreter_cli["Interpreter CLI<br/>standalone interface"]
end

subgraph group_coding["Coding pipeline"]
  node_coding_orchestrator{{"GPT All Star orchestrator<br/>delivery workflow<br/>[gpt_all_star.py]"}}
  node_coding_team["Coding agent team<br/>multi-agent state<br/>[team.py]"]
  node_coding_stages["Delivery stages<br/>pipeline stages<br/>[steps.py]"]
  node_coding_execution["Code execution<br/>execution boundary<br/>[execution.py]"]
  node_duplicate_pipeline["Parallel GPT All Star surface<br/>duplicated implementation"]
end

subgraph group_platform["Integration platform"]
  node_audio_providers["Audio providers<br/>STT/TTS adapters<br/>[openai.py]"]
  node_database[("Application database<br/>persistence<br/>[db.py]")]
  node_migrations["Schema migrations<br/>database evolution<br/>[env.py]"]
end

node_launch -->|"starts"| node_gui
node_gui -->|"submits requests"| node_agent_worker
node_gui -->|"controls"| node_speech_input
node_gui -->|"controls"| node_tts
node_gui -->|"hosts"| node_terminal
node_speech_input -->|"voice requests"| node_agent_worker
node_agent_worker -->|"invokes"| node_brain
node_brain -->|"reads and updates"| node_agent_state
node_brain -->|"delegates"| node_agent_roles
node_brain -->|"retrieves and stores"| node_memory
node_brain -->|"selects capabilities"| node_tool_router
node_monitoring -->|"proactive context"| node_brain
node_tool_router -->|"performs side effects"| node_automation
node_tts -->|"synthesizes through"| node_audio_providers
node_speech_input -->|"transcribes through"| node_audio_providers
node_brain -->|"persists application data"| node_database
node_migrations -->|"evolves schema"| node_database
node_interpreter_cli -->|"runs"| node_interpreter_loop
node_interpreter_loop -->|"executes tool calls"| node_computer
node_terminal -.->|"can launch"| node_interpreter_cli
node_coding_orchestrator -->|"coordinates"| node_coding_team
node_coding_team -->|"drives"| node_coding_stages
node_coding_stages -->|"hands off execution"| node_coding_execution
node_duplicate_pipeline -.->|"parallel surface"| node_coding_orchestrator
node_launch -.->|"selectable mode"| node_coding_orchestrator
node_launch -.->|"selectable mode"| node_interpreter_cli

click node_launch "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/main.py"
click node_gui "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/AssistantGUI.py"
click node_agent_worker "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/AgentWorker.py"
click node_speech_input "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/SpeechRecognition.py"
click node_tts "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/TTSWorker.py"
click node_terminal "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/terminal/terminal_manager.py"
click node_brain "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/brain.py"
click node_agent_state "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/agents/state.py"
click node_agent_roles "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/agents/ConversationalAgent.py"
click node_memory "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/memory/memory_agent.py"
click node_tool_router "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/tools/router.py"
click node_monitoring "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/tools/health_monitor.py"
click node_automation "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/tools/scheduler.py"
click node_interpreter_loop "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/interpreter/core/core.py"
click node_computer "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/interpreter/core/computer/computer.py"
click node_interpreter_cli "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/interpreter/terminal_interface/terminal_interface.py"
click node_coding_orchestrator "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/gpt_all_star.py"
click node_coding_team "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/team.py"
click node_coding_stages "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/steps/steps.py"
click node_coding_execution "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/execution/execution.py"
click node_audio_providers "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/jarvis_integration/audio/stt_providers/openai.py"
click node_database "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/jarvis_integration/internals/db.py"
click node_migrations "https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/migrations/env.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_launch,node_gui,node_agent_worker,node_speech_input,node_tts,node_terminal toneBlue
class node_brain,node_agent_state,node_agent_roles,node_memory,node_tool_router,node_monitoring,node_automation toneAmber
class node_interpreter_loop,node_computer,node_interpreter_cli toneMint
class node_coding_orchestrator,node_coding_team,node_coding_stages,node_coding_execution,node_duplicate_pipeline toneRose
class node_audio_providers,node_database,node_migrations toneIndigo
```

## Components

### Desktop client

- [Desktop launchers](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/main.py): entrypoints · Windows launch surfaces and mode selection. · `main.py`
- [PyQt assistant UI](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/AssistantGUI.py): desktop UI · `gui/AssistantGUI.py`
- [Agent worker](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/AgentWorker.py): background worker · `gui/Objects/AgentWorker.py`
- [Speech input](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/SpeechRecognition.py): voice worker · Recognition and wake-word activation. · `gui/Objects/SpeechRecognition.py`
- [Speech output](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/Objects/TTSWorker.py): TTS worker · `gui/Objects/TTSWorker.py`
- [Embedded terminal](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/gui/terminal/terminal_manager.py): terminal subsystem · Virtual filesystem-backed terminal access. · `gui/terminal/terminal_manager.py`

### Assistant core

- [Assistant brain](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/brain.py): orchestrator · Routes requests and sensed context to roles and tools. · `core/brain.py`
- [Shared agent state](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/agents/state.py): state boundary · `core/agents/state.py`
- [Specialist agents](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/agents/ConversationalAgent.py): agent roles · Conversational, planning, vision, and domain agents. · `core/agents/ConversationalAgent.py`
- [Conversation memory](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/memory/memory_agent.py): persistent memory · `core/memory/memory_agent.py`
- [Tool router](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/tools/router.py): capability boundary · `core/tools/router.py`
- [Monitoring signals](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/tools/health_monitor.py): proactive sensing · Health, sensor, and consciousness signals. · `core/tools/health_monitor.py`
- [Automation integrations](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/core/tools/scheduler.py): side-effect tools · Scheduling, home automation, news, and OS capabilities. · `core/tools/scheduler.py`

### Computer-use runtime

- [Interpreter control loop](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/interpreter/core/core.py): LLM tool loop · `interpreter/core/core.py`
- [Computer adapters](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/interpreter/core/computer/computer.py): desktop control · Terminal, display, input, filesystem, browser, and vision adapters. · `interpreter/core/computer/computer.py`
- [Interpreter CLI](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/interpreter/terminal_interface/terminal_interface.py): standalone interface · `interpreter/terminal_interface/terminal_interface.py`

### Coding pipeline

- [GPT All Star orchestrator](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/gpt_all_star.py): delivery workflow · `components/coder/core/gpt_all_star.py`
- [Coding agent team](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/team.py): multi-agent state · Role collaboration and shared team state. · `components/coder/core/team.py`
- [Delivery stages](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/steps/steps.py): pipeline stages · Specification through development, QA, healing, and deployment. · `components/coder/core/steps/steps.py`
- [Code execution](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/components/coder/core/execution/execution.py): execution boundary · Runs shell and file operations. · `components/coder/core/execution/execution.py`
- **Parallel GPT All Star surface**: duplicated implementation · components/gpt_all_star mirrors the coder workflow surface.

### Integration platform

- [Audio providers](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/jarvis_integration/audio/stt_providers/openai.py): STT/TTS adapters · OpenAI/local STT and multiple TTS implementations. · `jarvis_integration/audio/stt_providers/openai.py`
- [Application database](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/jarvis_integration/internals/db.py): persistence · Users, preferences, history, contacts, agents, tools, and sensors. · `jarvis_integration/internals/db.py`
- [Schema migrations](https://github.com/dawoodtouseef/j.ar.v.i.s./blob/main/migrations/env.py): database evolution · `migrations/env.py`

## Connections

- Desktop launchers → PyQt assistant UI: starts
- PyQt assistant UI → Agent worker: submits requests
- PyQt assistant UI → Speech input: controls
- PyQt assistant UI → Speech output: controls
- PyQt assistant UI → Embedded terminal: hosts
- Speech input → Agent worker: voice requests
- Agent worker → Assistant brain: invokes
- Assistant brain → Shared agent state: reads and updates
- Assistant brain → Specialist agents: delegates
- Assistant brain → Conversation memory: retrieves and stores
- Assistant brain → Tool router: selects capabilities
- Monitoring signals → Assistant brain: proactive context
- Tool router → Automation integrations: performs side effects
- Speech output → Audio providers: synthesizes through
- Speech input → Audio providers: transcribes through
- Assistant brain → Application database: persists application data
- Schema migrations → Application database: evolves schema
- Interpreter CLI → Interpreter control loop: runs
- Interpreter control loop → Computer adapters: executes tool calls
- Embedded terminal → Interpreter CLI: can launch
- GPT All Star orchestrator → Coding agent team: coordinates
- Coding agent team → Delivery stages: drives
- Delivery stages → Code execution: hands off execution
- Parallel GPT All Star surface → GPT All Star orchestrator: parallel surface
- Desktop launchers → GPT All Star orchestrator: selectable mode
- Desktop launchers → Interpreter CLI: selectable mode