# ucd-193ab-ws24/minecapstone architecture

> Architecture diagram of the GitHub repository ucd-193ab-ws24/minecapstone, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ucd-193ab-ws24/minecapstone
- Repository: https://github.com/ucd-193ab-ws24/minecapstone

## Overview

##### Project Type and Purpose
- This is a full-stack benchmarking platform for embodied LLM agents, implemented as a 3D sandbox game in Godot (GDScript + C#) with a Python backend that manages LLM-driven agent control via WebSockets.
- Purpose: Generate scripted “goals,” send them to an LLM via a prompt architecture, receive a sequence of high-level actions, and execute them in the Godot environment to quantitatively benchmark agent performance.

##### Key Components (to include
a. Godot Game Environment
- Scene Graph: world/world.tscn, prefabs/player/agent.tscn
- Autoload Singletons: globals/agent_controller.gd, globals/message_broker.gd, globals/agent_manager.gd
b. Agent Node
- agent.gd (prefabs/player/agent.gd) handles action dispatch and receives commands
c. Prompting Architecture (Python)
- prompts.py: builds the goal-script prompt based on scenario definitions
- llm_service_factory.py + service adapters (openai_service_adapter.py, gemini_service_adapter.py): select and invoke the LLM
- llm_service.py: common interface for sending prompts, receiving responses
d. WebSocket Server
- python/websocket.py: listens for JSON RPC from Godot, routes requests to the LLM service, and pushes back agent action sequences
e. Available Agent Actions
- High-level actions defined in Godot (move_to, attack, pick_up, drop, build, etc.), encoded by the prompt and executed by agent_controller.gd
f. Environment Feedback
- State snapshots (e.g., agent position, world state) sent by Godot to backend to inform chain-of-thought or infinite decision-making

##### Relationships & Data Flow
1. Godot Agent Initialization
- Agent node starts scenario via ScenarioManager → sets a “goal” (e.g., collect wood).
2. Goal → Prompt Construction
- AgentController calls WebSocket client (godot/websocket.gd) with a “generate_prompt” RPC, including goal and current state.
3. Backend Prompting
- websocket.py receives RPC → prompts.py assembles the goal-script prompt → llm_service_factory picks adapter → sends to LLM API → obtains action list.
4. Action Sequence → Godot
- Backend returns JSON with ordered actions → websocket.gd in Godot parses and forwards to agent_controller.gd.
5. Action Execution
- agent_controller.gd iterates actions → invokes Agent methods (move_to, attack, pick_up, etc.) → environment updates (navigation, physics, animations).
6. Loop / Feedback
- After each action or at checkpoints, Godot can send updated state back to backend for further reasoning or next prompt (infinite decision-making).

##### Architectural Patterns & Principles
- Client-Server: Godot (client) ↔ Python (server) over WebSocket.
- Autoload Singletons (Godot): centralize agent management and messaging.
- Strategy Pattern: llm_service_factory selects between multiple LLM adapters.
- Modular Scenes: Scenarios, Prefabs, World Generation separated in folders.

##### Focusing on Agent ↔ Goal-Script Prompting Interaction
- Sequence:
[AgentController.gd] → send “generate_goal_script” RPC →
[websocket.py] → prompts.py.build_goal_prompt() →
[LLM Adapter] → LLM API →
[websocket.py] → return action list JSON →
[agent_controller.gd] → execute actions in world.
- Input: Goal definition (from ScenarioManager)
- Processing: Prompt Template + Scenario context
- Output: JSON array of actions.

##### Final Checklist
- Essential modules: Godot Client, WebSocket Client, Python WebSocket Server, Prompt Engine, LLM Adapter, LLM Services, Game Environment.
- Available agent actions as an action list block.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Godot Client Layer
    subgraph "Godot Client" 
        direction TB
        SM["ScenarioManager"]:::godot
        AC["AgentController (godot/globals/agent_controller.gd)"]:::godot
        AG["Agent Node (godot/prefabs/player/agent.gd)"]:::godot
        WSClient["WebSocket Client (godot/websocket/websocket.gd)"]:::godot
        ACTIONS["Available Actions\n(move_to, attack, pick_up, drop, build)"]:::godot
        ENV["Game Environment\n(godot/world/world.tscn\nprefabs/player/agent.tscn)"]:::godot
    end

    %% Python Backend Layer
    subgraph "Goal-Script Prompting (Python Backend)"
        direction TB
        WS["WebSocket Server\n(python/websocket.py)"]:::python
        PB["Prompt Builder\n(python/prompts.py)"]:::python
        LSF["LLM Service Factory\n(python/llm_service_factory.py)"]:::python
        LSI["LLM Service Interface\n(python/llm_service.py)"]:::python
        OA["OpenAI Adapter\n(python/openai_service_adapter.py)"]:::python
        GA["Gemini Adapter\n(python/gemini_service_adapter.py)"]:::python
    end

    %% External LLM Services
    LLMCloud["LLM Services"]:::external

    %% Data Flows
    SM -->|"goal"| AC
    AC -->|"generate_goal_script RPC\n(goal + state)"| WSClient
    WSClient -->|"RPC request"| WS
    WS -->|"assemble prompt"| PB
    PB -->|"Prompt Template"| PB
    PB -->|"request"| LSF
    LSF -->|"""select adapter"""| OA
    LSF -->|"""select adapter"""| GA
    OA -->|"LLM API call"| LLMCloud
    GA -->|"LLM API call"| LLMCloud
    LLMCloud -->|"response JSON"| OA
    LLMCloud -->|"response JSON"| GA
    OA -->|"parsed actions"| LSI
    GA -->|"parsed actions"| LSI
    LSI -->|"action_list JSON"| WS
    WS -->|"RPC response\n(action_list)"| WSClient
    WSClient -->|"action_list"| AC
    AC -->|"invoke"| ACTIONS
    ACTIONS -->|"execute"| ENV
    ENV -.->|"state update"| WSClient

    %% Click Events
    click SM "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/godot/benchmarking/prefabs/scenario_manager.gd"
    click AC "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/godot/globals/agent_controller.gd"
    click AG "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/godot/prefabs/player/agent.gd"
    click WSClient "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/godot/websocket/websocket.gd"
    click WS "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/python/websocket.py"
    click PB "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/python/prompts.py"
    click LSF "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/python/llm_service_factory.py"
    click LSI "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/python/llm_service.py"
    click OA "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/python/openai_service_adapter.py"
    click GA "https://github.com/ucd-193ab-ws24/minecapstone/blob/main/python/gemini_service_adapter.py"

    %% Styles
    classDef godot fill:#D0E8FF,stroke:#0366D6,color:#0366D6,stroke-width:2px;
    classDef python fill:#DFF7DF,stroke:#2E8B57,color:#2E8B57,stroke-width:2px,stroke-dasharray: 5 5;
    classDef external fill:#FFE8B0,stroke:#D2691E,color:#D2691E,stroke-width:2px;
    class ACTIONS godot
    class ENV godot
```
