vivek02sharma/sam3-triton

Purpose and Entry Points

This repository serves Segment Anything 3 (SAM3) inference through NVIDIA Triton Inference Server, using TensorRT-backed model stages. The primary operational sequence is:

1. Build TensorRT engines with compile.sh.

2. Run Triton via docker-compose.yml.

3. Use the Streamlit client UI in app.py.

The container definition is Dockerfile; Python dependencies are managed through pyproject.toml and uv.lock.

Client Application

app.py is the interactive Streamlit entry point and Triton HTTP client. It submits inference requests to the server, using a pool of 32 HTTP connections to support concurrent interactions. The UI is outside Triton’s model execution boundary: it prepares requests, calls served models, and presents segmentation results.

Relevant path:

• app.py

Triton Model Repository

model_repository/ is Triton’s deployment boundary. It contains separately declared model stages and their config.pbtxt contracts, which define how Triton exposes, schedules, and connects each model.

Core model paths:

• model_repository/sam3_model/config.pbtxt

• model_repository/vision_encoder/config.pbtxt

• model_repository/text_encoder/config.pbtxt

• model_repository/mask_decoder/config.pbtxt

SAM3 Orchestration Layer

The top-level SAM3 Triton Python model coordinates the segmentation pipeline. It receives the high-level inference request and invokes the vision encoder, text encoder, and mask decoder stages. It transfers tensors between those stages using DLPack, avoiding host/PCIe copies and preserving GPU-resident data.

Relevant paths:

• model_repository/sam3_model/1/model.py

• model_repository/sam3_model/config.pbtxt

This is the principal control-flow boundary between client-facing SAM3 inference and the specialized TensorRT model components.

Encoder and Decoder Stages

SAM3 is decomposed into three serving components:

• The vision encoder derives image features.

• The text encoder derives prompt/text features.

• The mask decoder combines encoded inputs to produce segmentation masks.

Their Triton interfaces are defined independently, enabling Triton to schedule and dynamically batch requests at model boundaries.

Relevant paths:

• model_repository/vision_encoder/config.pbtxt

• model_repository/text_encoder/config.pbtxt

• model_repository/mask_decoder/config.pbtxt

Runtime, GPU, and External Assets

The runtime is NVIDIA Triton with TensorRT engines generated before server startup. Docker Compose runs the service and pins it to GPU 0; changing GPU allocation requires editing docker-compose.yml. The server readiness endpoint is exposed on port 9000 (/v2/health/ready).

Model source assets are external SAM3 ONNX artifacts referenced by the README; compilation converts or prepares them for the Triton/TensorRT deployment layout.

Relevant paths:

• compile.sh

• docker-compose.yml

• Dockerfile