How yangchris11/samurai works
SAMURAI is a zero-shot visual object tracking system built on SAM 2, using a bounding-box prompt and motion-aware memory to segment an object across video frames. The main user workflow is to provide a video or frame directory and an initial box, run inference, and obtain tracked masks; the repository also includes VOS inference and evaluation, automatic mask generation, and training code. The graph covers these documented capabilities and sampled model internals. Some script-to-model wiring and unsampled subsystems are not shown, so those connections are omitted.
Main entry points: sam2_base.py Memory tracking core · trainer.py Training loop · sam2_video_predictor.py Video predictor · build_sam.py Model builder
Components 26
Inference interfaces
- Custom video demo
scripts/demo.py - Benchmark inference
scripts/main_inference.py - Chunked inference
scripts/main_inference_chunk.py
SAM 2 tracking model
- Model builder
sam2/sam2/build_sam.py - Video predictor
sam2/sam2/sam2_video_predictor.py - Image predictor
sam2/sam2/sam2_image_predictor.py - Automatic masks
sam2/sam2/automatic_mask_generator.py - Memory tracking core
sam2/sam2/modeling/sam2_base.py - Mask decoder
sam2/sam2/modeling/sam/mask_decoder.py - Prompt transformer
sam2/sam2/modeling/sam/transformer.py - Model utilities
sam2/sam2/modeling/sam2_utils.py - Motion estimation
sam2/sam2/utils/kalman_filter.py
Training pipelines
- SAM 2 training
sam2/training/train.py - Training loop
sam2/training/trainer.py - Training model
sam2/training/model/sam2.py - Optimizer
sam2/training/optimizer.py - Training datasets
sam2/training/dataset - Tracker training entry
lib/train/train_script.py - Tracking data setup
lib/train/base_functions.py - Sequence sampling
lib/train/data/sampler.py - Tracking objectives
lib/train/actors/artrack_seq.py - Tracking trainer
lib/train/trainers/ltr_seq_trainer.py
VOS and evaluation
- VOS inference
sam2/tools/vos_inference.py - SAV evaluation
sam2/sav_dataset/sav_evaluator.py - Video metrics
sam2/sav_dataset/utils/sav_benchmark.py
Other components
- Video and box user
Connections 24
- Video and box user → Custom video demo: provides video and box
README.md - Custom video demo → Video predictor: runs tracking
- Video and box user → Benchmark inference: starts benchmark run
README.md - Benchmark inference → Video predictor: runs tracking
- Chunked inference → Model builder: builds model
scripts/main_inference_chunk.py - VOS inference → Model builder: builds model
sam2/tools/vos_inference.py - Model builder → Video predictor: constructs predictor
sam2/sam2/sam2_video_predictor.py - Video predictor → Memory tracking core: delegates tracking
sam2/sam2/sam2_video_predictor.py - Memory tracking core → Mask decoder: uses decoder
sam2/sam2/modeling/sam2_base.py - Memory tracking core → Prompt transformer: uses transformer
sam2/sam2/modeling/sam2_base.py - Memory tracking core → Model utilities: uses utilities
sam2/sam2/modeling/sam2_base.py - Memory tracking core → Motion estimation: uses motion filter
sam2/sam2/modeling/sam2_base.py - Mask decoder → Model utilities: uses utilities
sam2/sam2/modeling/sam/mask_decoder.py - Prompt transformer → Model utilities: uses utilities
sam2/sam2/modeling/sam/transformer.py - SAM 2 training → Training loop: starts training
- Training loop → Optimizer: configures optimizer
sam2/training/trainer.py - Training loop → Training model: instantiates model
sam2/training/trainer.py - Training loop → Training datasets: loads datasets
- Training model → Memory tracking core: extends model core
sam2/training/model/sam2.py - Tracker training entry → Tracking data setup: configures data
- Tracking data setup → Sequence sampling: uses sampler
lib/train/base_functions.py - Tracker training entry → Tracking trainer: starts training
- Tracking trainer → Tracking objectives: runs actor
- SAV evaluation → Video metrics: uses metrics
sam2/sav_dataset/sav_evaluator.py