# itz-vk-dude/pygent architecture

> Architecture diagram of the GitHub repository itz-vk-dude/pygent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/itz-vk-dude/pygent
- Repository: https://github.com/itz-vk-dude/pygent

## Overview

PHYGENT is a cyber-physical AI agent that monitors a workspace, predicts object motion, reasons about possible tasks, simulates and safety-checks actions, then may command a robot and learn from observed outcomes. Its documented loop is coordinated by the agent core; the web interface is launched separately by the application entry point. Edges from agent_core.py rely on its SOURCE INDEX references, while several detailed subsystem links follow the README workflow; unsampled implementation details are not inferred.

Main entry points: [`agent/agent_core.py`](https://github.com/itz-vk-dude/pygent/blob/main/agent/agent_core.py) (Agent loop), [`interface/web_app.py`](https://github.com/itz-vk-dude/pygent/blob/main/interface/web_app.py) (Web interface), [`main.py`](https://github.com/itz-vk-dude/pygent/blob/main/main.py) (Application entry), [`perception/perception_engine.py`](https://github.com/itz-vk-dude/pygent/blob/main/perception/perception_engine.py) (Perception)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime and state"]
  node_main["Application entry<br/>[main.py]"]
  node_agent["Agent loop<br/>[agent_core.py]"]
  node_autonomy["Autonomy policy"]
  node_state["World state<br/>[state_manager.py]"]
end

subgraph group_sensing["Sensing and data"]
  node_camera["Camera input<br/>[camera.py]"]
  node_perception["Perception"]
  node_quality["Data quality<br/>[quality.py]"]
  node_database[("Audit database<br/>[database.py]")]
end

subgraph group_planning["Prediction and planning"]
  node_prediction["Hybrid prediction"]
  node_reasoning["Agent reasoning"]
  node_simulation["Action simulation"]
end

subgraph group_execution["Safety and execution"]
  node_decision["Action selection<br/>[decision_engine.py]"]
  node_safety["Safety gate<br/>[safety.py]"]
  node_kinematics["Arm kinematics<br/>[kinematics.py]"]
  node_executor["Robot executor<br/>[executor.py]"]
  node_serial["Serial controller"]
end

subgraph group_learning_ui["Feedback and interface"]
  node_feedback["Outcome feedback<br/>[feedback_engine.py]"]
  node_learning["Continual learning"]
  node_web["Web interface<br/>[web_app.py]"]
  node_api["REST API<br/>[api.py]"]
end

node_user(("User"))
node_grok["Grok service"]
node_robot["Robot hardware"]

node_main -->|"starts loop"| node_agent
node_main -->|"starts web app"| node_web
node_web -->|"uses API"| node_api
node_user -.->|"interacts"| node_web
node_agent -->|"reads frames"| node_camera
node_agent -->|"processes frames"| node_perception
node_perception -.->|"quality-checks data"| node_quality
node_agent -->|"checks quality"| node_quality
node_agent -->|"records data"| node_database
node_agent -->|"updates state"| node_state
node_agent -->|"predicts motion"| node_prediction
node_agent -->|"requests reasoning"| node_reasoning
node_reasoning -->|"queries"| node_grok
node_agent -->|"generates candidates"| node_simulation
node_simulation -->|"uses kinematics"| node_kinematics
node_agent -->|"selects action"| node_decision
node_decision -->|"validates targets"| node_safety
node_safety -->|"checks reachability"| node_kinematics
node_agent -.->|"evaluates autonomy"| node_autonomy
node_agent -->|"executes action"| node_executor
node_executor -.->|"sends commands"| node_serial
node_serial -->|"sends serial commands"| node_robot
node_agent -->|"evaluates outcome"| node_feedback
node_feedback -.->|"informs learning"| node_learning
node_agent -->|"updates models"| node_learning

click node_main "https://github.com/itz-vk-dude/pygent/blob/main/main.py"
click node_agent "https://github.com/itz-vk-dude/pygent/blob/main/agent/agent_core.py"
click node_autonomy "https://github.com/itz-vk-dude/pygent/blob/main/agent/autonomy_manager.py"
click node_state "https://github.com/itz-vk-dude/pygent/blob/main/world_model/state_manager.py"
click node_camera "https://github.com/itz-vk-dude/pygent/blob/main/physical_world/camera.py"
click node_perception "https://github.com/itz-vk-dude/pygent/blob/main/perception/perception_engine.py"
click node_quality "https://github.com/itz-vk-dude/pygent/blob/main/data/quality.py"
click node_database "https://github.com/itz-vk-dude/pygent/blob/main/data/database.py"
click node_prediction "https://github.com/itz-vk-dude/pygent/blob/main/prediction/hybrid_predictor.py"
click node_reasoning "https://github.com/itz-vk-dude/pygent/blob/main/reasoning/reasoning_engine.py"
click node_simulation "https://github.com/itz-vk-dude/pygent/blob/main/simulation/candidate_actions.py"
click node_decision "https://github.com/itz-vk-dude/pygent/blob/main/decision/decision_engine.py"
click node_safety "https://github.com/itz-vk-dude/pygent/blob/main/decision/safety.py"
click node_kinematics "https://github.com/itz-vk-dude/pygent/blob/main/robotics/kinematics.py"
click node_executor "https://github.com/itz-vk-dude/pygent/blob/main/robotics/executor.py"
click node_serial "https://github.com/itz-vk-dude/pygent/blob/main/communication/serial_controller.py"
click node_feedback "https://github.com/itz-vk-dude/pygent/blob/main/feedback/feedback_engine.py"
click node_learning "https://github.com/itz-vk-dude/pygent/blob/main/learning/continual_learning.py"
click node_web "https://github.com/itz-vk-dude/pygent/blob/main/interface/web_app.py"
click node_api "https://github.com/itz-vk-dude/pygent/blob/main/interface/api.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_agent,node_autonomy,node_state,node_user toneBlue
class node_camera,node_perception,node_quality,node_database toneAmber
class node_prediction,node_reasoning,node_simulation toneMint
class node_decision,node_safety,node_kinematics,node_executor,node_serial toneRose
class node_feedback,node_learning,node_web,node_api,node_grok,node_robot toneIndigo
```

## Components

### Runtime and state

- [Application entry](https://github.com/itz-vk-dude/pygent/blob/main/main.py): `main.py`
- [Agent loop](https://github.com/itz-vk-dude/pygent/blob/main/agent/agent_core.py): `agent/agent_core.py`
- [Autonomy policy](https://github.com/itz-vk-dude/pygent/blob/main/agent/autonomy_manager.py): `agent/autonomy_manager.py`
- [World state](https://github.com/itz-vk-dude/pygent/blob/main/world_model/state_manager.py): `world_model/state_manager.py`

### Sensing and data

- [Camera input](https://github.com/itz-vk-dude/pygent/blob/main/physical_world/camera.py): `physical_world/camera.py`
- [Perception](https://github.com/itz-vk-dude/pygent/blob/main/perception/perception_engine.py): `perception/perception_engine.py`
- [Data quality](https://github.com/itz-vk-dude/pygent/blob/main/data/quality.py): `data/quality.py`
- [Audit database](https://github.com/itz-vk-dude/pygent/blob/main/data/database.py): `data/database.py`

### Prediction and planning

- [Hybrid prediction](https://github.com/itz-vk-dude/pygent/blob/main/prediction/hybrid_predictor.py): `prediction/hybrid_predictor.py`
- [Agent reasoning](https://github.com/itz-vk-dude/pygent/blob/main/reasoning/reasoning_engine.py): `reasoning/reasoning_engine.py`
- [Action simulation](https://github.com/itz-vk-dude/pygent/blob/main/simulation/candidate_actions.py): `simulation/candidate_actions.py`

### Safety and execution

- [Action selection](https://github.com/itz-vk-dude/pygent/blob/main/decision/decision_engine.py): `decision/decision_engine.py`
- [Safety gate](https://github.com/itz-vk-dude/pygent/blob/main/decision/safety.py): `decision/safety.py`
- [Arm kinematics](https://github.com/itz-vk-dude/pygent/blob/main/robotics/kinematics.py): `robotics/kinematics.py`
- [Robot executor](https://github.com/itz-vk-dude/pygent/blob/main/robotics/executor.py): `robotics/executor.py`
- [Serial controller](https://github.com/itz-vk-dude/pygent/blob/main/communication/serial_controller.py): `communication/serial_controller.py`

### Feedback and interface

- [Outcome feedback](https://github.com/itz-vk-dude/pygent/blob/main/feedback/feedback_engine.py): `feedback/feedback_engine.py`
- [Continual learning](https://github.com/itz-vk-dude/pygent/blob/main/learning/continual_learning.py): `learning/continual_learning.py`
- [Web interface](https://github.com/itz-vk-dude/pygent/blob/main/interface/web_app.py): `interface/web_app.py`
- [REST API](https://github.com/itz-vk-dude/pygent/blob/main/interface/api.py): `interface/api.py`

### Other components

- **User**
- **Grok service**
- **Robot hardware**

## Connections

- Application entry → Agent loop: starts loop (evidence: `main.py`)
- Application entry → Web interface: starts web app (evidence: `main.py`)
- Web interface → REST API: uses API (evidence: `interface/web_app.py`)
- User → Web interface: interacts (evidence: `README.md`)
- Agent loop → Camera input: reads frames (evidence: `agent/agent_core.py`)
- Agent loop → Perception: processes frames (evidence: `agent/agent_core.py`)
- Perception → Data quality: quality-checks data (evidence: `README.md`)
- Agent loop → Data quality: checks quality (evidence: `agent/agent_core.py`)
- Agent loop → Audit database: records data (evidence: `agent/agent_core.py`)
- Agent loop → World state: updates state (evidence: `agent/agent_core.py`)
- Agent loop → Hybrid prediction: predicts motion (evidence: `agent/agent_core.py`)
- Agent loop → Agent reasoning: requests reasoning (evidence: `agent/agent_core.py`)
- Agent reasoning → Grok service: queries (evidence: `reasoning/reasoning_engine.py`)
- Agent loop → Action simulation: generates candidates (evidence: `agent/agent_core.py`)
- Action simulation → Arm kinematics: uses kinematics (evidence: `simulation/candidate_actions.py`)
- Agent loop → Action selection: selects action (evidence: `agent/agent_core.py`)
- Action selection → Safety gate: validates targets (evidence: `decision/decision_engine.py`)
- Safety gate → Arm kinematics: checks reachability (evidence: `decision/safety.py`)
- Agent loop → Autonomy policy: evaluates autonomy (evidence: `agent/agent_core.py`)
- Agent loop → Robot executor: executes action (evidence: `agent/agent_core.py`)
- Robot executor → Serial controller: sends commands
- Serial controller → Robot hardware: sends serial commands (evidence: `communication/serial_controller.py`)
- Agent loop → Outcome feedback: evaluates outcome (evidence: `agent/agent_core.py`)
- Outcome feedback → Continual learning: informs learning (evidence: `README.md`)
- Agent loop → Continual learning: updates models (evidence: `agent/agent_core.py`)
