# openrobotlab/grutopia architecture

> Architecture diagram of the GitHub repository openrobotlab/grutopia, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openrobotlab/grutopia
- Repository: https://github.com/openrobotlab/grutopia

## Overview

##### Project Type and Purpose
- GRUtopia is a full-stack embodied-AI research platform and simulation framework.
- It provides:
- A Python core library (“grutopia/core”) that wraps NVIDIA Omniverse Isaac Sim.
- A modular config system for sensors, robots, scenes, tasks, and metrics.
- A Gym-compatible environment interface (“grutopia/core/gym_env.py”).
- A DataHub for loading assets (GRScenes) and models.
- Runtime managers for single-node or distributed task execution.
- Extension packages (“grutopia_extension”) for LLM-driven NPCs, specialized agents, controllers, benchmarks, and teleoperation servers.
- Toolkits for scene generation and mocap utilities.

##### File Structure
- Top‐level:
- Dockerfile / CI / pre-commit / setup scripts
- README / docs/ — contains tutorials, API guides, architecture overview, and benchmarks
- grutopia/: core simulation and API
- grutopia_extension/: plug-in modules—agents, controllers, sensors, tasks, interactions
- toolkits/: auxiliary scripts for scene generation, data processing, and mocap
- Key patterns:
- Registry/factory (“grutopia/core/register”) for extensibility
- Clear separation: config / datahub / scene / robot / task / runtime / util
- Gym wrapper around the “core” for RL interoperability
- Demo scripts under grutopia/demo that drive the system end to end

##### Insights from the README
- Purpose: general-purpose research platform for embodied AGI
- Features: GRScenes dataset, GRResidents (LLM-based NPCs), GRBench for benchmarks
- Dependencies: Isaac Sim, rsl_rl, various vision & LLM libraries, OmniGibson patterns

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core System
    subgraph "Core System"
        direction TB
        DataHub["DataHub\n(grutopia/core/datahub)"]:::datastore
        SceneLoader["Scene Loader\n(grutopia/core/scene)"]:::core
        ObjectRegistry["Object Registry\n(grutopia/core/register)"]:::core
        PhysicsEngine["Physics & Rendering Engine\n(Isaac Sim wrapper)"]:::core
        RobotModel["Robot Model\n(grutopia/core/robot)"]:::core
        ControllerModule["Controller Module\n(grutopia/core/robot)"]:::core
        SensorModules["Sensor Modules\n(grutopia/core/robot)"]:::core
        RuntimeManager["Runtime Manager\n(Local & Distributed)"]:::core
        LocalRuntime["LocalTaskRuntimeManager"]:::core
        DistributedRuntime["DistributedTaskRuntimeManager"]:::core
        TaskModule["Task Module\n(grutopia/core/task)"]:::core
        RewardModule["Reward Module\n(grutopia/core/task)"]:::core
        MetricModule["Metric Module\n(grutopia/core/task)"]:::core
        GymEnv["Gym Environment\n(grutopia/core/gym_env.py)"]:::core
        AgentInterface["Agent Interface\n(grutopia/demo & extension base)"]:::core

        DataHub --> SceneLoader
        SceneLoader --> PhysicsEngine
        SceneLoader --> ObjectRegistry
        PhysicsEngine -->|actuation commands| ControllerModule
        ControllerModule -->|controls| PhysicsEngine
        PhysicsEngine --> SensorModules
        SensorModules --> RuntimeManager
        RuntimeManager --> TaskModule
        TaskModule --> RewardModule
        TaskModule --> MetricModule
        RuntimeManager --> GymEnv
        GymEnv --> AgentInterface
        AgentInterface --> GymEnv
        GymEnv -->|action| RuntimeManager
        RuntimeManager --> ControllerModule

        RuntimeManager --> LocalRuntime
        RuntimeManager --> DistributedRuntime
    end

    %% Extensions
    subgraph "Extension Modules"
        direction TB
        ExtAgents["Extension Agents\n(grutopia_extension/agents)"]:::extension
        TeleopServer["Mocap Teleop Server"]:::extension
        VisionPro["VisionPro Server"]:::extension
        VLMServer["VLM Server Wrapper"]:::extension

        ExtAgents --> AgentInterface
        TeleopServer --> SensorModules
        VisionPro --> SensorModules
        VLMServer --> AgentInterface
    end

    %% External Services
    subgraph "External Services" 
        direction TB
        LLMVLM["LLM/VLM Services"]:::external
    end

    LLMVLM --> ExtAgents
    LLMVLM --> SensorModules

    %% Click Events
    click DataHub "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/datahub/datahub.py"
    click SceneLoader "https://github.com/openrobotlab/grutopia/tree/main/grutopia/core/scene/scene"
    click ObjectRegistry "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/register/register.py"
    click PhysicsEngine "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/runner.py"
    click RobotModel "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/robot/robot.py"
    click ControllerModule "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/robot/controller.py"
    click SensorModules "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/robot/sensor.py"
    click TaskModule "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/task/task.py"
    click RewardModule "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/task/reward.py"
    click MetricModule "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/task/metric.py"
    click LocalRuntime "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/runtime/local_task_runtime_manager.py"
    click DistributedRuntime "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/runtime/distributed_task_runtime_manager.py"
    click GymEnv "https://github.com/openrobotlab/grutopia/blob/main/grutopia/core/gym_env.py"
    click AgentInterface "https://github.com/openrobotlab/grutopia/tree/main/grutopia/demo/"
    click ExtAgents "https://github.com/openrobotlab/grutopia/blob/main/grutopia_extension/agents/core/agent/agent.py"
    click TeleopServer "https://github.com/openrobotlab/grutopia/blob/main/grutopia_extension/interactions/motion_capture.py"
    click VisionPro "https://github.com/openrobotlab/grutopia/blob/main/grutopia_extension/interactions/visionpro/TeleVision.py"
    click VLMServer "https://github.com/openrobotlab/grutopia/blob/main/grutopia_extension/agents/common/modules/vlm/server_wrapper.py"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef extension fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
    classDef datastore fill:#D3D3D3,stroke:#000,stroke-width:1px
```
