# manifoldai-research/worldscape-policy architecture

> Architecture diagram of the GitHub repository manifoldai-research/worldscape-policy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/manifoldai-research/worldscape-policy
- Repository: https://github.com/manifoldai-research/worldscape-policy

## Overview

WorldScape Policy 2.0 is a controllable robot World Action Model for planning and manipulation from language, goal images, and demonstration videos. The principal runtime loop accepts robot observations and prompts, maintains per-episode visual and event memory, routes auto-planning or interactive conditioning, then samples predicted video and actions for execution. Separate branches cover multi-source training data and training, checkpoint loading, and benchmark/real-robot evaluation. The README documents broader capabilities; exact call wiring beyond the sampled excerpts is shown only where supported, and evaluation subsystems use their existing enclosing paths.

Main entry points: [`src/worldscape_policy/policy.py`](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/policy.py) (World Policy), [`src/worldscape_policy/rollout/session.py`](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/rollout/session.py) (Episode Runtime), [`src/worldscape_policy/conditioning/router.py`](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/router.py) (Condition Router), [`src/worldscape_policy/conditioning/auto_conditioner.py`](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/auto_conditioner.py) (Auto Planner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Policy Runtime"]
  node_session["Episode Runtime<br/>[session.py]"]
  node_policy["World Policy<br/>[policy.py]"]
  node_router["Condition Router<br/>[router.py]"]
  node_auto["Auto Planner"]
  node_interactive["Interactive Conditioning"]
  node_action["Predicted Actions<br/>[types.py]"]
end

subgraph group_model["Memory And Modeling"]
  node_text["Text Encoding<br/>[t5.py]"]
  node_vlm["VLM Planning<br/>[qwen3vl.py]"]
  node_eventmem["Event Memory<br/>[memory.py]"]
  node_retriever["History Retrieval<br/>[retriever.py]"]
  node_visualmem["Visual Memory<br/>[state.py]"]
  node_wam["Wan World Model<br/>[model.py]"]
  node_sampler["Flow Sampler<br/>[flow_matching.py]"]
end

subgraph group_learning["Data And Training"]
  node_datasets["Dataset Adapters"]
  node_sampling["Temporal Sampling"]
  node_train["Training Orchestrator<br/>[trainer.py]"]
  node_objective["Training Objective<br/>[objective.py]"]
  node_checkpoint["Checkpoint Builder<br/>[native_builder.py]"]
end

subgraph group_deployment["Evaluation Deployment"]
  node_eval["Evaluation Harnesses"]
  node_robotserver["AgileX Transport<br/>[server.py]"]
  node_robotwin["RoboTwin Manager"]
  node_simulator["Simulation Benchmarks"]
end

node_robot(("Robot"))
node_prompts["Task Prompts"]
node_external(("Robot Services"))

node_robot -->|"sends observations"| node_session
node_prompts -->|"provides prompts"| node_session
node_session -->|"requests prediction"| node_policy
node_policy -->|"routes conditioning"| node_router
node_router -->|"selects auto"| node_auto
node_router -->|"selects interactive"| node_interactive
node_auto -->|"generates plans"| node_vlm
node_auto -->|"retrieves history"| node_retriever
node_retriever -->|"reads history"| node_eventmem
node_interactive -->|"encodes instruction"| node_text
node_policy -->|"uses visual state"| node_visualmem
node_policy -->|"conditions model"| node_wam
node_wam -->|"samples dynamics"| node_sampler
node_sampler -->|"returns actions"| node_action
node_session -->|"commits prediction"| node_action
node_datasets -->|"provides episodes"| node_sampling
node_sampling -->|"batches examples"| node_train
node_train -->|"computes loss"| node_objective
node_checkpoint -->|"constructs policy"| node_policy
node_eval -->|"evaluates policy"| node_policy
node_robotserver -->|"exchanges robot data"| node_external
node_robotserver -->|"adapts observations"| node_session
node_robotwin -->|"schedules evaluations"| node_eval
node_simulator -->|"runs benchmarks"| node_eval

click node_session "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/rollout/session.py"
click node_policy "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/policy.py"
click node_router "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/router.py"
click node_auto "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/auto_conditioner.py"
click node_interactive "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/interactive_conditioner.py"
click node_text "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/text/t5.py"
click node_vlm "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/vlm/qwen3vl.py"
click node_eventmem "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/memory/event/memory.py"
click node_retriever "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/memory/event/retriever.py"
click node_visualmem "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/memory/visual/state.py"
click node_wam "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/wam/wan22/model.py"
click node_sampler "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/wam/flow_matching.py"
click node_action "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/types.py"
click node_datasets "https://github.com/manifoldai-research/worldscape-policy/tree/main/src/worldscape_policy/data/adapters"
click node_sampling "https://github.com/manifoldai-research/worldscape-policy/tree/main/src/worldscape_policy/data/sampling"
click node_train "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/training/trainer.py"
click node_objective "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/training/objective.py"
click node_checkpoint "https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/native_builder.py"
click node_eval "https://github.com/manifoldai-research/worldscape-policy/tree/main/evals"
click node_robotserver "https://github.com/manifoldai-research/worldscape-policy/blob/main/evals/agilex/server.py"
click node_robotwin "https://github.com/manifoldai-research/worldscape-policy/blob/main/experiments/robotwin/run_robotwin_manager.py"
click node_simulator "https://github.com/manifoldai-research/worldscape-policy/tree/main/evals"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_session,node_policy,node_router,node_auto,node_interactive,node_action toneBlue
class node_text,node_vlm,node_eventmem,node_retriever,node_visualmem,node_wam,node_sampler toneAmber
class node_datasets,node_sampling,node_train,node_objective,node_checkpoint toneMint
class node_eval,node_robotserver,node_robotwin,node_simulator,node_prompts toneRose
class node_robot,node_external toneIndigo
```

## Components

### Policy Runtime

- [Episode Runtime](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/rollout/session.py): `src/worldscape_policy/rollout/session.py`
- [World Policy](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/policy.py): `src/worldscape_policy/policy.py`
- [Condition Router](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/router.py): `src/worldscape_policy/conditioning/router.py`
- [Auto Planner](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/auto_conditioner.py): `src/worldscape_policy/conditioning/auto_conditioner.py`
- [Interactive Conditioning](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/interactive_conditioner.py): `src/worldscape_policy/conditioning/interactive_conditioner.py`
- [Predicted Actions](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/types.py): `src/worldscape_policy/types.py`

### Memory And Modeling

- [Text Encoding](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/text/t5.py): `src/worldscape_policy/conditioning/text/t5.py`
- [VLM Planning](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/conditioning/vlm/qwen3vl.py): `src/worldscape_policy/conditioning/vlm/qwen3vl.py`
- [Event Memory](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/memory/event/memory.py): `src/worldscape_policy/memory/event/memory.py`
- [History Retrieval](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/memory/event/retriever.py): `src/worldscape_policy/memory/event/retriever.py`
- [Visual Memory](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/memory/visual/state.py): `src/worldscape_policy/memory/visual/state.py`
- [Wan World Model](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/wam/wan22/model.py): `src/worldscape_policy/wam/wan22/model.py`
- [Flow Sampler](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/wam/flow_matching.py): `src/worldscape_policy/wam/flow_matching.py`

### Data And Training

- [Dataset Adapters](https://github.com/manifoldai-research/worldscape-policy/tree/main/src/worldscape_policy/data/adapters): `src/worldscape_policy/data/adapters`
- [Temporal Sampling](https://github.com/manifoldai-research/worldscape-policy/tree/main/src/worldscape_policy/data/sampling): `src/worldscape_policy/data/sampling`
- [Training Orchestrator](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/training/trainer.py): `src/worldscape_policy/training/trainer.py`
- [Training Objective](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/training/objective.py): `src/worldscape_policy/training/objective.py`
- [Checkpoint Builder](https://github.com/manifoldai-research/worldscape-policy/blob/main/src/worldscape_policy/native_builder.py): `src/worldscape_policy/native_builder.py`

### Evaluation Deployment

- [Evaluation Harnesses](https://github.com/manifoldai-research/worldscape-policy/tree/main/evals): `evals`
- [AgileX Transport](https://github.com/manifoldai-research/worldscape-policy/blob/main/evals/agilex/server.py): `evals/agilex/server.py`
- [RoboTwin Manager](https://github.com/manifoldai-research/worldscape-policy/blob/main/experiments/robotwin/run_robotwin_manager.py): `experiments/robotwin/run_robotwin_manager.py`
- [Simulation Benchmarks](https://github.com/manifoldai-research/worldscape-policy/tree/main/evals): `evals`

### Other components

- **Robot**
- **Task Prompts**
- **Robot Services**

## Connections

- Robot → Episode Runtime: sends observations
- Task Prompts → Episode Runtime: provides prompts
- Episode Runtime → World Policy: requests prediction
- World Policy → Condition Router: routes conditioning
- Condition Router → Auto Planner: selects auto
- Condition Router → Interactive Conditioning: selects interactive
- Auto Planner → VLM Planning: generates plans
- Auto Planner → History Retrieval: retrieves history
- History Retrieval → Event Memory: reads history
- Interactive Conditioning → Text Encoding: encodes instruction
- World Policy → Visual Memory: uses visual state
- World Policy → Wan World Model: conditions model
- Wan World Model → Flow Sampler: samples dynamics
- Flow Sampler → Predicted Actions: returns actions
- Episode Runtime → Predicted Actions: commits prediction
- Dataset Adapters → Temporal Sampling: provides episodes
- Temporal Sampling → Training Orchestrator: batches examples
- Training Orchestrator → Training Objective: computes loss
- Checkpoint Builder → World Policy: constructs policy
- Evaluation Harnesses → World Policy: evaluates policy
- AgileX Transport → Robot Services: exchanges robot data
- AgileX Transport → Episode Runtime: adapts observations
- RoboTwin Manager → Evaluation Harnesses: schedules evaluations
- Simulation Benchmarks → Evaluation Harnesses: runs benchmarks