# yangchris11/samurai architecture

> Architecture diagram of the GitHub repository yangchris11/samurai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yangchris11/samurai
- Repository: https://github.com/yangchris11/samurai

## Overview

SAMURAI is a zero-shot visual object tracking system built on SAM 2, using a bounding-box prompt and motion-aware memory to segment an object across video frames. The main user workflow is to provide a video or frame directory and an initial box, run inference, and obtain tracked masks; the repository also includes VOS inference and evaluation, automatic mask generation, and training code. The graph covers these documented capabilities and sampled model internals. Some script-to-model wiring and unsampled subsystems are not shown, so those connections are omitted.

Main entry points: [`sam2/sam2/modeling/sam2_base.py`](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam2_base.py) (Memory tracking core), [`sam2/training/trainer.py`](https://github.com/yangchris11/samurai/blob/master/sam2/training/trainer.py) (Training loop), [`sam2/sam2/sam2_video_predictor.py`](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/sam2_video_predictor.py) (Video predictor), [`sam2/sam2/build_sam.py`](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/build_sam.py) (Model builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Inference interfaces"]
  node_demo["Custom video demo<br/>[demo.py]"]
  node_vot["Benchmark inference<br/>[main_inference.py]"]
  node_chunk["Chunked inference"]
end

subgraph group_model["SAM 2 tracking model"]
  node_builder["Model builder<br/>[build_sam.py]"]
  node_video_predictor["Video predictor"]
  node_image_predictor["Image predictor"]
  node_mask_generator["Automatic masks"]
  node_sam2_base["Memory tracking core<br/>[sam2_base.py]"]
  node_mask_decoder["Mask decoder<br/>[mask_decoder.py]"]
  node_transformer["Prompt transformer<br/>[transformer.py]"]
  node_sam2_utils["Model utilities<br/>[sam2_utils.py]"]
  node_kalman["Motion estimation<br/>[kalman_filter.py]"]
end

subgraph group_training["Training pipelines"]
  node_train_entry["SAM 2 training<br/>[train.py]"]
  node_train_loop["Training loop<br/>[trainer.py]"]
  node_train_model["Training model<br/>[sam2.py]"]
  node_optimizer["Optimizer<br/>[optimizer.py]"]
  node_train_data["Training datasets"]
  node_legacy_entry["Tracker training entry<br/>[train_script.py]"]
  node_legacy_data["Tracking data setup<br/>[base_functions.py]"]
  node_legacy_sampler["Sequence sampling<br/>[sampler.py]"]
  node_legacy_actor["Tracking objectives<br/>[artrack_seq.py]"]
  node_legacy_trainer["Tracking trainer<br/>[ltr_seq_trainer.py]"]
end

subgraph group_evaluation["VOS and evaluation"]
  node_vos_infer["VOS inference<br/>[vos_inference.py]"]
  node_sav_eval["SAV evaluation<br/>[sav_evaluator.py]"]
  node_sav_benchmark["Video metrics<br/>[sav_benchmark.py]"]
end

node_user(("Video and box user"))

node_user -->|"provides video and box"| node_demo
node_demo -.->|"runs tracking"| node_video_predictor
node_user -->|"starts benchmark run"| node_vot
node_vot -.->|"runs tracking"| node_video_predictor
node_chunk -->|"builds model"| node_builder
node_vos_infer -->|"builds model"| node_builder
node_builder -->|"constructs predictor"| node_video_predictor
node_video_predictor -->|"delegates tracking"| node_sam2_base
node_sam2_base -->|"uses decoder"| node_mask_decoder
node_sam2_base -->|"uses transformer"| node_transformer
node_sam2_base -->|"uses utilities"| node_sam2_utils
node_sam2_base -->|"uses motion filter"| node_kalman
node_mask_decoder -->|"uses utilities"| node_sam2_utils
node_transformer -->|"uses utilities"| node_sam2_utils
node_train_entry -.->|"starts training"| node_train_loop
node_train_loop -->|"configures optimizer"| node_optimizer
node_train_loop -->|"instantiates model"| node_train_model
node_train_loop -.->|"loads datasets"| node_train_data
node_train_model -->|"extends model core"| node_sam2_base
node_legacy_entry -.->|"configures data"| node_legacy_data
node_legacy_data -->|"uses sampler"| node_legacy_sampler
node_legacy_entry -.->|"starts training"| node_legacy_trainer
node_legacy_trainer -.->|"runs actor"| node_legacy_actor
node_sav_eval -->|"uses metrics"| node_sav_benchmark

click node_demo "https://github.com/yangchris11/samurai/blob/master/scripts/demo.py"
click node_vot "https://github.com/yangchris11/samurai/blob/master/scripts/main_inference.py"
click node_chunk "https://github.com/yangchris11/samurai/blob/master/scripts/main_inference_chunk.py"
click node_vos_infer "https://github.com/yangchris11/samurai/blob/master/sam2/tools/vos_inference.py"
click node_builder "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/build_sam.py"
click node_video_predictor "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/sam2_video_predictor.py"
click node_image_predictor "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/sam2_image_predictor.py"
click node_mask_generator "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/automatic_mask_generator.py"
click node_sam2_base "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam2_base.py"
click node_mask_decoder "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam/mask_decoder.py"
click node_transformer "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam/transformer.py"
click node_sam2_utils "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam2_utils.py"
click node_kalman "https://github.com/yangchris11/samurai/blob/master/sam2/sam2/utils/kalman_filter.py"
click node_train_entry "https://github.com/yangchris11/samurai/blob/master/sam2/training/train.py"
click node_train_loop "https://github.com/yangchris11/samurai/blob/master/sam2/training/trainer.py"
click node_train_model "https://github.com/yangchris11/samurai/blob/master/sam2/training/model/sam2.py"
click node_optimizer "https://github.com/yangchris11/samurai/blob/master/sam2/training/optimizer.py"
click node_train_data "https://github.com/yangchris11/samurai/tree/master/sam2/training/dataset"
click node_legacy_entry "https://github.com/yangchris11/samurai/blob/master/lib/train/train_script.py"
click node_legacy_data "https://github.com/yangchris11/samurai/blob/master/lib/train/base_functions.py"
click node_legacy_sampler "https://github.com/yangchris11/samurai/blob/master/lib/train/data/sampler.py"
click node_legacy_actor "https://github.com/yangchris11/samurai/blob/master/lib/train/actors/artrack_seq.py"
click node_legacy_trainer "https://github.com/yangchris11/samurai/blob/master/lib/train/trainers/ltr_seq_trainer.py"
click node_sav_eval "https://github.com/yangchris11/samurai/blob/master/sam2/sav_dataset/sav_evaluator.py"
click node_sav_benchmark "https://github.com/yangchris11/samurai/blob/master/sam2/sav_dataset/utils/sav_benchmark.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_vot,node_chunk,node_user toneBlue
class node_builder,node_video_predictor,node_image_predictor,node_mask_generator,node_sam2_base,node_mask_decoder,node_transformer,node_sam2_utils,node_kalman toneAmber
class node_train_entry,node_train_loop,node_train_model,node_optimizer,node_train_data,node_legacy_entry,node_legacy_data,node_legacy_sampler,node_legacy_actor,node_legacy_trainer toneMint
class node_vos_infer,node_sav_eval,node_sav_benchmark toneRose
```

## Components

### Inference interfaces

- [Custom video demo](https://github.com/yangchris11/samurai/blob/master/scripts/demo.py): `scripts/demo.py`
- [Benchmark inference](https://github.com/yangchris11/samurai/blob/master/scripts/main_inference.py): `scripts/main_inference.py`
- [Chunked inference](https://github.com/yangchris11/samurai/blob/master/scripts/main_inference_chunk.py): `scripts/main_inference_chunk.py`

### SAM 2 tracking model

- [Model builder](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/build_sam.py): `sam2/sam2/build_sam.py`
- [Video predictor](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/sam2_video_predictor.py): `sam2/sam2/sam2_video_predictor.py`
- [Image predictor](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/sam2_image_predictor.py): `sam2/sam2/sam2_image_predictor.py`
- [Automatic masks](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/automatic_mask_generator.py): `sam2/sam2/automatic_mask_generator.py`
- [Memory tracking core](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam2_base.py): `sam2/sam2/modeling/sam2_base.py`
- [Mask decoder](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam/mask_decoder.py): `sam2/sam2/modeling/sam/mask_decoder.py`
- [Prompt transformer](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam/transformer.py): `sam2/sam2/modeling/sam/transformer.py`
- [Model utilities](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/modeling/sam2_utils.py): `sam2/sam2/modeling/sam2_utils.py`
- [Motion estimation](https://github.com/yangchris11/samurai/blob/master/sam2/sam2/utils/kalman_filter.py): `sam2/sam2/utils/kalman_filter.py`

### Training pipelines

- [SAM 2 training](https://github.com/yangchris11/samurai/blob/master/sam2/training/train.py): `sam2/training/train.py`
- [Training loop](https://github.com/yangchris11/samurai/blob/master/sam2/training/trainer.py): `sam2/training/trainer.py`
- [Training model](https://github.com/yangchris11/samurai/blob/master/sam2/training/model/sam2.py): `sam2/training/model/sam2.py`
- [Optimizer](https://github.com/yangchris11/samurai/blob/master/sam2/training/optimizer.py): `sam2/training/optimizer.py`
- [Training datasets](https://github.com/yangchris11/samurai/tree/master/sam2/training/dataset): `sam2/training/dataset`
- [Tracker training entry](https://github.com/yangchris11/samurai/blob/master/lib/train/train_script.py): `lib/train/train_script.py`
- [Tracking data setup](https://github.com/yangchris11/samurai/blob/master/lib/train/base_functions.py): `lib/train/base_functions.py`
- [Sequence sampling](https://github.com/yangchris11/samurai/blob/master/lib/train/data/sampler.py): `lib/train/data/sampler.py`
- [Tracking objectives](https://github.com/yangchris11/samurai/blob/master/lib/train/actors/artrack_seq.py): `lib/train/actors/artrack_seq.py`
- [Tracking trainer](https://github.com/yangchris11/samurai/blob/master/lib/train/trainers/ltr_seq_trainer.py): `lib/train/trainers/ltr_seq_trainer.py`

### VOS and evaluation

- [VOS inference](https://github.com/yangchris11/samurai/blob/master/sam2/tools/vos_inference.py): `sam2/tools/vos_inference.py`
- [SAV evaluation](https://github.com/yangchris11/samurai/blob/master/sam2/sav_dataset/sav_evaluator.py): `sam2/sav_dataset/sav_evaluator.py`
- [Video metrics](https://github.com/yangchris11/samurai/blob/master/sam2/sav_dataset/utils/sav_benchmark.py): `sam2/sav_dataset/utils/sav_benchmark.py`

### Other components

- **Video and box user**

## Connections

- Video and box user → Custom video demo: provides video and box (evidence: `README.md`)
- Custom video demo → Video predictor: runs tracking
- Video and box user → Benchmark inference: starts benchmark run (evidence: `README.md`)
- Benchmark inference → Video predictor: runs tracking
- Chunked inference → Model builder: builds model (evidence: `scripts/main_inference_chunk.py`)
- VOS inference → Model builder: builds model (evidence: `sam2/tools/vos_inference.py`)
- Model builder → Video predictor: constructs predictor (evidence: `sam2/sam2/sam2_video_predictor.py`)
- Video predictor → Memory tracking core: delegates tracking (evidence: `sam2/sam2/sam2_video_predictor.py`)
- Memory tracking core → Mask decoder: uses decoder (evidence: `sam2/sam2/modeling/sam2_base.py`)
- Memory tracking core → Prompt transformer: uses transformer (evidence: `sam2/sam2/modeling/sam2_base.py`)
- Memory tracking core → Model utilities: uses utilities (evidence: `sam2/sam2/modeling/sam2_base.py`)
- Memory tracking core → Motion estimation: uses motion filter (evidence: `sam2/sam2/modeling/sam2_base.py`)
- Mask decoder → Model utilities: uses utilities (evidence: `sam2/sam2/modeling/sam/mask_decoder.py`)
- Prompt transformer → Model utilities: uses utilities (evidence: `sam2/sam2/modeling/sam/transformer.py`)
- SAM 2 training → Training loop: starts training
- Training loop → Optimizer: configures optimizer (evidence: `sam2/training/trainer.py`)
- Training loop → Training model: instantiates model (evidence: `sam2/training/trainer.py`)
- Training loop → Training datasets: loads datasets
- Training model → Memory tracking core: extends model core (evidence: `sam2/training/model/sam2.py`)
- Tracker training entry → Tracking data setup: configures data
- Tracking data setup → Sequence sampling: uses sampler (evidence: `lib/train/base_functions.py`)
- Tracker training entry → Tracking trainer: starts training
- Tracking trainer → Tracking objectives: runs actor
- SAV evaluation → Video metrics: uses metrics (evidence: `sam2/sav_dataset/sav_evaluator.py`)
