How ut-austin-robin/datamil works
DataMIL computes trajectory-level datamodel scores for robot imitation datasets, selects examples, then trains and evaluates policies on the selected data. The principal workflow covers dataset conversion/loading, benchmark-specific selection, and policy training/evaluation; internal metagradient/VJP machinery computes the selection signal. The README documents the end-to-end MetaWorld and LIBERO workflows. Some implementation links are based on SOURCE INDEX references because those files were not excerpted.
Main entry points: do_mw_data_selection_parallel.py MetaWorld selector · do_libero_data_selection_parallel.py LIBERO selector · scripts Dataset conversion · vjp.py VJP estimation
Components 26
Data preparation
- LIBERO RLDS builders
rlds_utils/rlds_dataset_builders - Dataset conversion
scripts - Trajectory transforms
datamil/data/dataset.py - MetaWorld MDS loader
datamil/metagradients/make_mw_loader_mds.py - LIBERO MDS loader
datamil/metagradients/make_libero_loader_mds.py
Datamodel selection
- MetaWorld selector
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO selector
datamil/metagradients/do_libero_data_selection_parallel.py - Selected data scores
Policy training
- MetaWorld policy
datamil/models/mw_model_jax.py - Octo policy
datamil/models/make_octo_model.py - Octo action head
datamil/models/octo_action_head.py - LIBERO cotraining
scripts/cotraining.py - MetaWorld selected training
scripts/selected_training_mw.py - Benchmark evaluation
datamil
Core runtime
- Replay data loading
datamil/metagradients/core/dataloading.py - VJP estimation
datamil/metagradients/core/vjp.py - VJP blocks
datamil/metagradients/core/vjp_blocks.py - RoboDM VJP
datamil/metagradients/core/vjp_robodm.py - Adam optimizer
datamil/metagradients/core/optimizers/adam.py - Optimizer schedules
datamil/metagradients/core/optimizers/interpolation.py - Learning-rate schedule
datamil/metagradients/core/optimizers/schedules.py - JAX and tree utilities
datamil/metagradients/core/utils.py - Device transfer
datamil/metagradients/core/dlpack.py - Selection configuration
datamil/metagradients/flags_config.py
Other components
- Researcher
- Robot datasets
Connections 37
- Researcher → Robot datasets: provides
- Robot datasets → LIBERO RLDS builders: converts to RLDS
- Robot datasets → Dataset conversion: converts to MDS
- LIBERO RLDS builders → Dataset conversion: converts to MDS
- Dataset conversion → Trajectory transforms: prepares trajectories
scripts/libero_rlds_to_mds.py - Trajectory transforms → LIBERO cotraining: prepares batches
scripts/cotraining.py - Dataset conversion → MetaWorld MDS loader: produces MDS input
- Dataset conversion → LIBERO MDS loader: produces MDS input
- MetaWorld MDS loader → MetaWorld selector: loads batches
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO MDS loader → LIBERO selector: loads batches
datamil/metagradients/do_libero_data_selection_parallel.py - MetaWorld selector → Selected data scores: selects trajectories
- LIBERO selector → Selected data scores: selects trajectories
- MetaWorld selector → MetaWorld policy: uses policy
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO selector → Octo policy: uses policy
datamil/metagradients/do_libero_data_selection_parallel.py - MetaWorld selector → RoboDM VJP: computes metagradients
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO selector → RoboDM VJP: computes metagradients
datamil/metagradients/do_libero_data_selection_parallel.py - Selected data scores → MetaWorld selected training: selects training data
- Selected data scores → LIBERO cotraining: selects training data
- MetaWorld selected training → MetaWorld policy: trains policy
scripts/selected_training_mw.py - LIBERO cotraining → Octo policy: trains policy
- MetaWorld selected training → Benchmark evaluation: evaluates policy
scripts/selected_training_mw.py - LIBERO cotraining → Benchmark evaluation: evaluates policy
scripts/cotraining.py - Benchmark evaluation → Researcher: reports results
- MetaWorld selector → Replay data loading: requests replay batches
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO selector → Replay data loading: requests replay batches
datamil/metagradients/do_libero_data_selection_parallel.py - RoboDM VJP → VJP estimation: calls VJP
datamil/metagradients/core/vjp_robodm.py - VJP blocks → VJP estimation: uses VJP
datamil/metagradients/core/vjp_blocks.py - VJP estimation → Replay data loading: consumes batchers
datamil/metagradients/core/vjp.py - VJP estimation → Device transfer: transfers arrays
datamil/metagradients/core/vjp.py - VJP estimation → JAX and tree utilities: uses tree utilities
datamil/metagradients/core/vjp.py - RoboDM VJP → Adam optimizer: uses optimizer
datamil/metagradients/core/vjp_robodm.py - Adam optimizer → Learning-rate schedule: builds schedule
datamil/metagradients/core/optimizers/adam.py - MetaWorld selector → Optimizer schedules: uses schedules
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO selector → Optimizer schedules: uses schedules
datamil/metagradients/do_libero_data_selection_parallel.py - MetaWorld selector → Selection configuration: reads configuration
datamil/metagradients/do_mw_data_selection_parallel.py - LIBERO selector → Selection configuration: reads configuration
datamil/metagradients/do_libero_data_selection_parallel.py - Octo policy → Octo action head: configures action head
datamil/models/make_octo_model.py