How mkturkcan/dart works
DART adapts SAM3 into a real-time, multi-class open-vocabulary detector. Users provide images or video and class prompts; model construction and prediction produce detections, with optional TensorRT engines accelerating backbone and encoder-decoder inference. Video processing can pipeline backbone work across frames, while tracking and evaluation provide additional workflows. The tree and README support these major capabilities, but most runtime implementation was not sampled, so uncertain internal calls and subsystem wiring are omitted.
Main entry points: sam3_multiclass_fast.py Multiclass Predictor · build_engine.py TRT Engine Builder · agent_core.py Agent Core · sam3_image.py Image Model
Components 25
Detection Inference
- Multiclass Predictor
sam3/model/sam3_multiclass_fast.py - Image Processor
sam3/model/sam3_image_processor.py - Agent Inference
sam3/agent/inference.py - Agent Core
sam3/agent/agent_core.py
Video And Tracking
- Video Pipeline
sam3/video_pipeline.py - Video Inference
sam3/model/sam3_video_inference.py - Object Tracking
sam3/tracking/byte_tracker.py
TensorRT Acceleration
- TRT Backbone
sam3/trt/trt_backbone.py - TRT Encoder-Decoder
sam3/trt/trt_enc_dec.py - Backbone Export
sam3/trt/export_backbone.py - Encoder-Decoder Export
sam3/trt/export_enc_dec.py - TRT Engine Builder
sam3/trt/build_engine.py
SAM3 Model
- Model Builder
sam3/model_builder.py - Image Model
sam3/model/sam3_image.py - Vision Backbones
sam3/backbones - Text Encoding
sam3/model/text_encoder_ve.py
Training And Evaluation
- Training
sam3/train/train.py - Student Distillation
sam3/distillation/distill_trainer.py - Evaluation
sam3/eval/coco_eval.py
Other components
- User
- Image Input
- Video Input
- Class Prompts
- Detections
- LLM Service
Connections 26
- User → Image Input: provides image
- User → Video Input: provides video
- User → Class Prompts: specifies classes
- Model Builder → Image Model: builds
- Image Model → Vision Backbones: uses
- Multiclass Predictor → Image Model: uses model
- Multiclass Predictor → Text Encoding: encodes prompts
- Image Input → Multiclass Predictor: image input
- Class Prompts → Multiclass Predictor: prompt input
- Multiclass Predictor → Image Processor: processes predictions
- Image Processor → Detections: returns results
- Video Input → Video Pipeline: video input
- Video Pipeline → Multiclass Predictor: runs prediction
- Video Inference → Object Tracking: tracks objects
- Backbone Export → TRT Engine Builder: exports ONNX
- Encoder-Decoder Export → TRT Engine Builder: exports ONNX
- TRT Engine Builder → TRT Backbone: builds engine
- TRT Engine Builder → TRT Encoder-Decoder: builds engine
- Multiclass Predictor → TRT Backbone: optional acceleration
- Multiclass Predictor → TRT Encoder-Decoder: optional acceleration
- Training → Student Distillation: trains students
- Evaluation → Detections: evaluates results
- User → Agent Inference: starts session
- Agent Inference → Agent Core: runs
- Agent Core → LLM Service: requests generation
- Agent Core → Multiclass Predictor: requests detection