How tranxuantuyen/planner-refiner works
Project type and purpose
• This is a research‐oriented PyTorch codebase implementing “Planner-Refiner” for Referring Video Object Segmentation (RVOS) and Video Temporal Grounding (VTG).
• It provides data pipelines, model definitions (backbones, pixel decoder, transformer decoder, matching and loss modules), training/evaluation engines, configuration files, and utility scripts.
File structure
• Top‐level packages:
• ITR/ – core implementation of the Planner-Refiner (“ITR”) model
• mask2former/ – external segmentation library integrated as a submodule
• configs/ – YAML configuration files for different experiments
• scripts/ – shell scripts to train, test, and validate
• tools/ – evaluation & checkpoint utilities
• train_net_itr.py – entry point for launching experiments
• ITR/ subpackages mimic an “ML pipeline” layering:
• data/ – dataset definitions, augmentation, data loaders
• modeling/ – model architectures (transformer decoder, criterion, matcher)
• engine/ – training loop, hooks, distributed launch
• evaluation/ – evaluator interface for metrics
• utils/ – miscellaneous helpers
• mask2former/ itself is organized similarly with separate data mappers, backbones, pixel‐decoder, transformer‐decoder, etc.
README insights
• Relies on MeViS dataset + custom CUDA kernel for MSDeformAttn
• Requires Anaconda environment, Python 3.8, PyTorch
• Uses pre‐trained backbone weights and external downloads for benchmarks
• Entrypoints: scripts/train.sh, scripts/eval.sh, train_net_itr.py