How ankile/robust-rearrangement works
Robust Rearrangement is a robotics research codebase for learning and refining policies that assemble furniture, from demonstrations through simulation evaluation and real-robot use. Its main workflow collects demonstrations, converts them into training datasets, trains behavior-cloning or reinforcement-learning policies, then evaluates policies in simulated rollouts. A separate sim-to-real tooling branch rerenders demonstrations with randomized scenes. The graph reflects the README and sampled source; some subsystem links are documented at a high level, while unsampled implementation details are intentionally left unwired.
Main entry points: base.py Behavior policies · dataset.py Sequence datasets · residual_ppo_w_bc.py PPO refinement · rollout.py Policy rollout
Components 21
Demonstration data
- Raw demonstrations
- Demo collection
src/data_collection/data_collector_sm.py - Demo processing
src/data_processing/process_pickles.py - Processed demonstrations
src/dataset/zarr.py
Policy learning
- Sequence datasets
src/dataset/dataset.py - Data normalization
src/dataset/normalizer.py - Behavior policies
src/behavior/base.py - Neural models
src/models/transformer.py - BC training
src/train/bc.py - DAgger training
src/train/dagger.py - PPO refinement
src/train/residual_ppo_w_bc.py - Rollout buffer
src/dataset/rollout_buffer.py
Simulation evaluation
- Furniture simulator
src/gym/mj_dual_franka_env.py - Policy rollout
src/eval/rollout.py - Evaluation runner
src/eval/evaluate_model.py
Robot execution
- Real inference
src/real/minimal.py - Real teleoperation
src/real/teleop_sm.py
Simulation transfer
- Demo rerendering
src/sim2real/isaac_lab_rerender.py - Part conversion
src/sim2real/convert_parts_to_usd.py
Other components
- Operator
- Physical robot
Connections 25
- Operator → Demo collection: provides demonstrations
- Demo collection → Raw demonstrations: saves trajectories
- Raw demonstrations → Demo processing: loads pickles
- Demo processing → Processed demonstrations: writes datasets
- Processed demonstrations → Sequence datasets: loads samples
- Sequence datasets → BC training: feeds training
- Sequence datasets → DAgger training: feeds training
- Sequence datasets → PPO refinement: loads demos
- BC training → Behavior policies: trains policy
- DAgger training → Behavior policies: trains policy
- PPO refinement → Behavior policies: refines policy
- Behavior policies → Neural models: uses networks
- Evaluation runner → Furniture simulator: creates environment
- Evaluation runner → Policy rollout: runs evaluation
- Policy rollout → Behavior policies: queries policy
- Policy rollout → Furniture simulator: steps environment
- PPO refinement → Furniture simulator: collects experience
- PPO refinement → Rollout buffer: stores rollouts
- Operator → Real teleoperation: provides commands
- Real teleoperation → Physical robot: controls robot
- Physical robot → Real inference: provides observations
- Real inference → Physical robot: sends actions
- Raw demonstrations → Demo rerendering: reads demonstrations
- Demo rerendering → Processed demonstrations: produces episodes
- Part conversion → Demo rerendering: supplies geometry