How manifoldai-research/worldscape-policy works
WorldScape Policy 2.0 is a controllable robot World Action Model for planning and manipulation from language, goal images, and demonstration videos. The principal runtime loop accepts robot observations and prompts, maintains per-episode visual and event memory, routes auto-planning or interactive conditioning, then samples predicted video and actions for execution. Separate branches cover multi-source training data and training, checkpoint loading, and benchmark/real-robot evaluation. The README documents broader capabilities; exact call wiring beyond the sampled excerpts is shown only where supported, and evaluation subsystems use their existing enclosing paths.
Main entry points: policy.py World Policy · session.py Episode Runtime · router.py Condition Router · auto_conditioner.py Auto Planner
Components 25
Policy Runtime
- Episode Runtime
src/worldscape_policy/rollout/session.py - World Policy
src/worldscape_policy/policy.py - Condition Router
src/worldscape_policy/conditioning/router.py - Auto Planner
src/worldscape_policy/conditioning/auto_conditioner.py - Interactive Conditioning
src/worldscape_policy/conditioning/interactive_conditioner.py - Predicted Actions
src/worldscape_policy/types.py
Memory And Modeling
- Text Encoding
src/worldscape_policy/conditioning/text/t5.py - VLM Planning
src/worldscape_policy/conditioning/vlm/qwen3vl.py - Event Memory
src/worldscape_policy/memory/event/memory.py - History Retrieval
src/worldscape_policy/memory/event/retriever.py - Visual Memory
src/worldscape_policy/memory/visual/state.py - Wan World Model
src/worldscape_policy/wam/wan22/model.py - Flow Sampler
src/worldscape_policy/wam/flow_matching.py
Data And Training
- Dataset Adapters
src/worldscape_policy/data/adapters - Temporal Sampling
src/worldscape_policy/data/sampling - Training Orchestrator
src/worldscape_policy/training/trainer.py - Training Objective
src/worldscape_policy/training/objective.py - Checkpoint Builder
src/worldscape_policy/native_builder.py
Evaluation Deployment
- Evaluation Harnesses
evals - AgileX Transport
evals/agilex/server.py - RoboTwin Manager
experiments/robotwin/run_robotwin_manager.py - Simulation Benchmarks
evals
Other components
- Robot
- Task Prompts
- Robot Services
Connections 24
- Robot → Episode Runtime: sends observations
- Task Prompts → Episode Runtime: provides prompts
- Episode Runtime → World Policy: requests prediction
- World Policy → Condition Router: routes conditioning
- Condition Router → Auto Planner: selects auto
- Condition Router → Interactive Conditioning: selects interactive
- Auto Planner → VLM Planning: generates plans
- Auto Planner → History Retrieval: retrieves history
- History Retrieval → Event Memory: reads history
- Interactive Conditioning → Text Encoding: encodes instruction
- World Policy → Visual Memory: uses visual state
- World Policy → Wan World Model: conditions model
- Wan World Model → Flow Sampler: samples dynamics
- Flow Sampler → Predicted Actions: returns actions
- Episode Runtime → Predicted Actions: commits prediction
- Dataset Adapters → Temporal Sampling: provides episodes
- Temporal Sampling → Training Orchestrator: batches examples
- Training Orchestrator → Training Objective: computes loss
- Checkpoint Builder → World Policy: constructs policy
- Evaluation Harnesses → World Policy: evaluates policy
- AgileX Transport → Robot Services: exchanges robot data
- AgileX Transport → Episode Runtime: adapts observations
- RoboTwin Manager → Evaluation Harnesses: schedules evaluations
- Simulation Benchmarks → Evaluation Harnesses: runs benchmarks