How isee-laboratory/referdino works
ReferDINO is a referring video object segmentation system: users provide a video and referring expression, and the model predicts object masks across frames. Its main runtime branches are dataset-based pretraining/fine-tuning and dataset-specific inference, with a separate video demo. The README documents COCO pretraining and YouTube-VOS, MeViS, DAVIS, A2D-Sentences, and JHMDB-Sentences workflows. Some subsystem links are based on documented capabilities and entry-point imports; unsampled implementation details are not inferred.
Main entry points: eval Dataset inference · trainer.py Fine-tuning · pretrainer.py COCO pretraining · __init__.py Model construction
Components 19
Runtime Entry Points
- Train/eval CLI
main.py - Video demo
demo_video.py - Dataset inference
eval
Video Data
- Referring datasets
datasets - Video transforms
datasets/transform_video.py - Dataset sampler
datasets/sampler.py
Grounding Model
- Model construction
models/__init__.py - GroundingDINO
models/GroundingDINO/groundingdino.py - Swin backbone
models/GroundingDINO/backbone/swin_transformer.py - Temporal modeling
models/GroundingDINO/temporal_modules.py - Mask prediction
models/GroundingDINO/segmentation.py
Training and Evaluation
- Fine-tuning
trainer.py - COCO pretraining
pretrainer.py - Training losses
models/criterion.py
Inference Results
- Mask postprocessing
models/postprocessing.py - Dataset evaluation
eval
Other components
- Researcher
- Video and expression
- Mask annotations
Connections 25
- Researcher → Train/eval CLI: configures run
- Researcher → Video demo: submits video
- Researcher → Dataset inference: starts inference
- Video and expression → Video demo: provides input
- Train/eval CLI → Fine-tuning: starts training
- Train/eval CLI → COCO pretraining: starts pretraining
- Fine-tuning → Referring datasets: loads samples
- COCO pretraining → Referring datasets: loads samples
- Fine-tuning → Dataset sampler: selects batches
- COCO pretraining → Dataset sampler: selects batches
- Referring datasets → Video transforms: augments clips
- Fine-tuning → Model construction: builds model
- COCO pretraining → Model construction: builds model
- Fine-tuning → Training losses: computes losses
- COCO pretraining → Training losses: computes losses
- Model construction → GroundingDINO: constructs
- GroundingDINO → Swin backbone: extracts features
- GroundingDINO → Temporal modeling: models video
- GroundingDINO → Mask prediction: predicts masks
- Dataset inference → Model construction: builds model
- Dataset inference → Mask postprocessing: formats predictions
- Dataset inference → Dataset evaluation: evaluates outputs
- Dataset inference → Mask annotations: writes masks
- Video demo → Model construction: loads model
- Video demo → Mask annotations: returns predictions