How allenai/molmospaces works
MolmoSpaces is a robotics simulation ecosystem for manipulation and navigation, supporting benchmark evaluation, policy execution, and trajectory generation. A central workflow configures a data-generation run, samples tasks, rolls policies through simulated environments, and prepares episode data for saving. The package also includes benchmark evaluation, robot control and planning, grasp generation, house generation, and rendering. README claims guide capability coverage; sampled source confirms parts of data generation, task behavior, camera setup, and remote policy clients, but not every subsystem’s wiring.
Main entry points: env.py Simulation environment · pipeline.py Rollout runner · main.py Data generation · opening_tasks.py Task implementations
Components 24
Runs and benchmarks
- Data generation
molmo_spaces/data_generation/main.py - Experiment configs
molmo_spaces/data_generation/config_registry.py - Rollout runner
molmo_spaces/data_generation/pipeline.py - Episode data
- Benchmark evaluation
molmo_spaces/evaluation/eval_main.py - Evaluation runner
molmo_spaces/evaluation/json_eval_runner.py
Simulation and tasks
- Task samplers
molmo_spaces/tasks/task_sampler.py - Task implementations
molmo_spaces/tasks/opening_tasks.py - Simulation environment
molmo_spaces/env/env.py - Scene objects
molmo_spaces/env/object_manager.py - Camera setup
molmo_spaces/env/camera_manager.py - Rendering
molmo_spaces/renderer/abstract_renderer.py
Robotics and policies
- Robot models
molmo_spaces/robots/abstract.py - Robot controllers
molmo_spaces/controllers/abstract.py - Policy implementations
molmo_spaces/policy/base_policy.py - Learned policies
molmo_spaces/policy/learned_policy - Motion planning
molmo_spaces/planner/abstract.py - Grasp generation
molmo_spaces/grasp_generation
Assets and tools
- House generation
molmo_spaces/housegen - Simulation assets
molmo_spaces/resources - Isaac assets
molmo_spaces_isaac/src/molmo_spaces_isaac/downloader/main.py - ManiSkill assets
molmo_spaces_maniskill/src/molmo_spaces_maniskill/downloader/main.py
Other components
- Researcher
- Remote policy service
Connections 25
- Researcher → Data generation: starts run
- Data generation → Experiment configs: resolves config
- Data generation → Rollout runner: starts rollouts
- Rollout runner → Task samplers: samples tasks
- Rollout runner → Policy implementations: runs policy
- Task samplers → Task implementations: creates tasks
- Task implementations → Simulation environment: steps simulation
- Task implementations → Scene objects: queries objects
- Simulation environment → Camera setup: sets up cameras
- Simulation environment → Rendering: renders scene
- Simulation environment → Robot models: simulates robot
- Robot models → Robot controllers: uses control
- Policy implementations → Learned policies: includes implementations
- Learned policies → Remote policy service: requests inference
- Policy implementations → Motion planning: can use planning
- Rollout runner → Episode data: prepares episode data
- Researcher → Benchmark evaluation: starts evaluation
- Benchmark evaluation → Evaluation runner: runs evaluation
- Evaluation runner → Task samplers: evaluates tasks
- Grasp generation → Scene objects: finds objects
- Researcher → Grasp generation: generates grasps
- Researcher → House generation: builds houses
- Researcher → Isaac assets: downloads assets
- Researcher → ManiSkill assets: downloads assets
- Simulation assets → Simulation environment: supplies assets