# mkturkcan/dart architecture

> Architecture diagram of the GitHub repository mkturkcan/dart, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mkturkcan/dart
- Repository: https://github.com/mkturkcan/dart

## Overview

DART adapts SAM3 into a real-time, multi-class open-vocabulary detector. Users provide images or video and class prompts; model construction and prediction produce detections, with optional TensorRT engines accelerating backbone and encoder-decoder inference. Video processing can pipeline backbone work across frames, while tracking and evaluation provide additional workflows. The tree and README support these major capabilities, but most runtime implementation was not sampled, so uncertain internal calls and subsystem wiring are omitted.

Main entry points: [`sam3/model/sam3_multiclass_fast.py`](https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_multiclass_fast.py) (Multiclass Predictor), [`sam3/trt/build_engine.py`](https://github.com/mkturkcan/dart/blob/main/sam3/trt/build_engine.py) (TRT Engine Builder), [`sam3/agent/agent_core.py`](https://github.com/mkturkcan/dart/blob/main/sam3/agent/agent_core.py) (Agent Core), [`sam3/model/sam3_image.py`](https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_image.py) (Image Model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Detection Inference"]
  node_multiclass["Multiclass Predictor"]
  node_processor["Image Processor"]
  node_agent["Agent Inference<br/>[inference.py]"]
  node_agent_core["Agent Core<br/>[agent_core.py]"]
end

subgraph group_video["Video And Tracking"]
  node_video_pipeline["Video Pipeline<br/>[video_pipeline.py]"]
  node_video_inference["Video Inference"]
  node_tracking["Object Tracking<br/>[byte_tracker.py]"]
end

subgraph group_acceleration["TensorRT Acceleration"]
  node_trt_backbone["TRT Backbone<br/>[trt_backbone.py]"]
  node_trt_encdec["TRT Encoder-Decoder<br/>[trt_enc_dec.py]"]
  node_export_backbone["Backbone Export<br/>[export_backbone.py]"]
  node_export_encdec["Encoder-Decoder Export<br/>[export_enc_dec.py]"]
  node_engine_builder["TRT Engine Builder<br/>[build_engine.py]"]
end

subgraph group_model["SAM3 Model"]
  node_builder["Model Builder<br/>[model_builder.py]"]
  node_image_model["Image Model<br/>[sam3_image.py]"]
  node_backbone["Vision Backbones"]
  node_text["Text Encoding<br/>[text_encoder_ve.py]"]
end

subgraph group_development["Training And Evaluation"]
  node_training["Training<br/>[train.py]"]
  node_distillation["Student Distillation<br/>[distill_trainer.py]"]
  node_evaluation["Evaluation<br/>[coco_eval.py]"]
end

node_user(("User"))
node_image["Image Input"]
node_video_input["Video Input"]
node_prompts["Class Prompts"]
node_detections["Detections"]
node_llm(("LLM Service"))

node_user -->|"provides image"| node_image
node_user -->|"provides video"| node_video_input
node_user -->|"specifies classes"| node_prompts
node_builder -->|"builds"| node_image_model
node_image_model -->|"uses"| node_backbone
node_multiclass -->|"uses model"| node_image_model
node_multiclass -->|"encodes prompts"| node_text
node_image -->|"image input"| node_multiclass
node_prompts -->|"prompt input"| node_multiclass
node_multiclass -->|"processes predictions"| node_processor
node_processor -->|"returns results"| node_detections
node_video_input -->|"video input"| node_video_pipeline
node_video_pipeline -->|"runs prediction"| node_multiclass
node_video_inference -->|"tracks objects"| node_tracking
node_export_backbone -->|"exports ONNX"| node_engine_builder
node_export_encdec -->|"exports ONNX"| node_engine_builder
node_engine_builder -->|"builds engine"| node_trt_backbone
node_engine_builder -->|"builds engine"| node_trt_encdec
node_multiclass -.->|"optional acceleration"| node_trt_backbone
node_multiclass -.->|"optional acceleration"| node_trt_encdec
node_training -.->|"trains students"| node_distillation
node_evaluation -.->|"evaluates results"| node_detections
node_user -.->|"starts session"| node_agent
node_agent -->|"runs"| node_agent_core
node_agent_core -.->|"requests generation"| node_llm
node_agent_core -.->|"requests detection"| node_multiclass

click node_builder "https://github.com/mkturkcan/dart/blob/main/sam3/model_builder.py"
click node_image_model "https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_image.py"
click node_backbone "https://github.com/mkturkcan/dart/tree/main/sam3/backbones"
click node_text "https://github.com/mkturkcan/dart/blob/main/sam3/model/text_encoder_ve.py"
click node_multiclass "https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_multiclass_fast.py"
click node_processor "https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_image_processor.py"
click node_video_pipeline "https://github.com/mkturkcan/dart/blob/main/sam3/video_pipeline.py"
click node_video_inference "https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_video_inference.py"
click node_tracking "https://github.com/mkturkcan/dart/blob/main/sam3/tracking/byte_tracker.py"
click node_trt_backbone "https://github.com/mkturkcan/dart/blob/main/sam3/trt/trt_backbone.py"
click node_trt_encdec "https://github.com/mkturkcan/dart/blob/main/sam3/trt/trt_enc_dec.py"
click node_export_backbone "https://github.com/mkturkcan/dart/blob/main/sam3/trt/export_backbone.py"
click node_export_encdec "https://github.com/mkturkcan/dart/blob/main/sam3/trt/export_enc_dec.py"
click node_engine_builder "https://github.com/mkturkcan/dart/blob/main/sam3/trt/build_engine.py"
click node_training "https://github.com/mkturkcan/dart/blob/main/sam3/train/train.py"
click node_distillation "https://github.com/mkturkcan/dart/blob/main/sam3/distillation/distill_trainer.py"
click node_evaluation "https://github.com/mkturkcan/dart/blob/main/sam3/eval/coco_eval.py"
click node_agent "https://github.com/mkturkcan/dart/blob/main/sam3/agent/inference.py"
click node_agent_core "https://github.com/mkturkcan/dart/blob/main/sam3/agent/agent_core.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_multiclass,node_processor,node_agent,node_agent_core,node_user toneBlue
class node_video_pipeline,node_video_inference,node_tracking toneAmber
class node_trt_backbone,node_trt_encdec,node_export_backbone,node_export_encdec,node_engine_builder toneMint
class node_builder,node_image_model,node_backbone,node_text toneRose
class node_training,node_distillation,node_evaluation,node_image,node_video_input,node_prompts,node_detections,node_llm toneIndigo
```

## Components

### Detection Inference

- [Multiclass Predictor](https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_multiclass_fast.py): `sam3/model/sam3_multiclass_fast.py`
- [Image Processor](https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_image_processor.py): `sam3/model/sam3_image_processor.py`
- [Agent Inference](https://github.com/mkturkcan/dart/blob/main/sam3/agent/inference.py): `sam3/agent/inference.py`
- [Agent Core](https://github.com/mkturkcan/dart/blob/main/sam3/agent/agent_core.py): `sam3/agent/agent_core.py`

### Video And Tracking

- [Video Pipeline](https://github.com/mkturkcan/dart/blob/main/sam3/video_pipeline.py): `sam3/video_pipeline.py`
- [Video Inference](https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_video_inference.py): `sam3/model/sam3_video_inference.py`
- [Object Tracking](https://github.com/mkturkcan/dart/blob/main/sam3/tracking/byte_tracker.py): `sam3/tracking/byte_tracker.py`

### TensorRT Acceleration

- [TRT Backbone](https://github.com/mkturkcan/dart/blob/main/sam3/trt/trt_backbone.py): `sam3/trt/trt_backbone.py`
- [TRT Encoder-Decoder](https://github.com/mkturkcan/dart/blob/main/sam3/trt/trt_enc_dec.py): `sam3/trt/trt_enc_dec.py`
- [Backbone Export](https://github.com/mkturkcan/dart/blob/main/sam3/trt/export_backbone.py): `sam3/trt/export_backbone.py`
- [Encoder-Decoder Export](https://github.com/mkturkcan/dart/blob/main/sam3/trt/export_enc_dec.py): `sam3/trt/export_enc_dec.py`
- [TRT Engine Builder](https://github.com/mkturkcan/dart/blob/main/sam3/trt/build_engine.py): `sam3/trt/build_engine.py`

### SAM3 Model

- [Model Builder](https://github.com/mkturkcan/dart/blob/main/sam3/model_builder.py): `sam3/model_builder.py`
- [Image Model](https://github.com/mkturkcan/dart/blob/main/sam3/model/sam3_image.py): `sam3/model/sam3_image.py`
- [Vision Backbones](https://github.com/mkturkcan/dart/tree/main/sam3/backbones): `sam3/backbones`
- [Text Encoding](https://github.com/mkturkcan/dart/blob/main/sam3/model/text_encoder_ve.py): `sam3/model/text_encoder_ve.py`

### Training And Evaluation

- [Training](https://github.com/mkturkcan/dart/blob/main/sam3/train/train.py): `sam3/train/train.py`
- [Student Distillation](https://github.com/mkturkcan/dart/blob/main/sam3/distillation/distill_trainer.py): `sam3/distillation/distill_trainer.py`
- [Evaluation](https://github.com/mkturkcan/dart/blob/main/sam3/eval/coco_eval.py): `sam3/eval/coco_eval.py`

### Other components

- **User**
- **Image Input**
- **Video Input**
- **Class Prompts**
- **Detections**
- **LLM Service**

## Connections

- User → Image Input: provides image
- User → Video Input: provides video
- User → Class Prompts: specifies classes
- Model Builder → Image Model: builds
- Image Model → Vision Backbones: uses
- Multiclass Predictor → Image Model: uses model
- Multiclass Predictor → Text Encoding: encodes prompts
- Image Input → Multiclass Predictor: image input
- Class Prompts → Multiclass Predictor: prompt input
- Multiclass Predictor → Image Processor: processes predictions
- Image Processor → Detections: returns results
- Video Input → Video Pipeline: video input
- Video Pipeline → Multiclass Predictor: runs prediction
- Video Inference → Object Tracking: tracks objects
- Backbone Export → TRT Engine Builder: exports ONNX
- Encoder-Decoder Export → TRT Engine Builder: exports ONNX
- TRT Engine Builder → TRT Backbone: builds engine
- TRT Engine Builder → TRT Encoder-Decoder: builds engine
- Multiclass Predictor → TRT Backbone: optional acceleration
- Multiclass Predictor → TRT Encoder-Decoder: optional acceleration
- Training → Student Distillation: trains students
- Evaluation → Detections: evaluates results
- User → Agent Inference: starts session
- Agent Inference → Agent Core: runs
- Agent Core → LLM Service: requests generation
- Agent Core → Multiclass Predictor: requests detection