# vivek02sharma/sam3-triton architecture

> Architecture diagram of the GitHub repository vivek02sharma/sam3-triton, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vivek02sharma/sam3-triton
- Repository: https://github.com/vivek02sharma/sam3-triton

## Overview

#### Purpose and Entry Points
This repository serves Segment Anything 3 (SAM3) inference through NVIDIA Triton Inference Server, using TensorRT-backed model stages. The primary operational sequence is:
1. Build TensorRT engines with `compile.sh`.
2. Run Triton via `docker-compose.yml`.
3. Use the Streamlit client UI in `app.py`.

The container definition is `Dockerfile`; Python dependencies are managed through `pyproject.toml` and `uv.lock`.

#### Client Application
`app.py` is the interactive Streamlit entry point and Triton HTTP client. It submits inference requests to the server, using a pool of 32 HTTP connections to support concurrent interactions. The UI is outside Triton’s model execution boundary: it prepares requests, calls served models, and presents segmentation results.

Relevant path:
- `app.py`

#### Triton Model Repository
`model_repository/` is Triton’s deployment boundary. It contains separately declared model stages and their `config.pbtxt` contracts, which define how Triton exposes, schedules, and connects each model.

Core model paths:
- `model_repository/sam3_model/config.pbtxt`
- `model_repository/vision_encoder/config.pbtxt`
- `model_repository/text_encoder/config.pbtxt`
- `model_repository/mask_decoder/config.pbtxt`

#### SAM3 Orchestration Layer
The top-level SAM3 Triton Python model coordinates the segmentation pipeline. It receives the high-level inference request and invokes the vision encoder, text encoder, and mask decoder stages. It transfers tensors between those stages using DLPack, avoiding host/PCIe copies and preserving GPU-resident data.

Relevant paths:
- `model_repository/sam3_model/1/model.py`
- `model_repository/sam3_model/config.pbtxt`

This is the principal control-flow boundary between client-facing SAM3 inference and the specialized TensorRT model components.

#### Encoder and Decoder Stages
SAM3 is decomposed into three serving components:
- The vision encoder derives image features.
- The text encoder derives prompt/text features.
- The mask decoder combines encoded inputs to produce segmentation masks.

Their Triton interfaces are defined independently, enabling Triton to schedule and dynamically batch requests at model boundaries.

Relevant paths:
- `model_repository/vision_encoder/config.pbtxt`
- `model_repository/text_encoder/config.pbtxt`
- `model_repository/mask_decoder/config.pbtxt`

#### Runtime, GPU, and External Assets
The runtime is NVIDIA Triton with TensorRT engines generated before server startup. Docker Compose runs the service and pins it to GPU 0; changing GPU allocation requires editing `docker-compose.yml`. The server readiness endpoint is exposed on port 9000 (`/v2/health/ready`).

Model source assets are external SAM3 ONNX artifacts referenced by the README; compilation converts or prepares them for the Triton/TensorRT deployment layout.

Relevant paths:
- `compile.sh`
- `docker-compose.yml`
- `Dockerfile`

Main entry points: [`model_repository/sam3_model/1/model.py`](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/sam3_model/1/model.py) (SAM3 orchestrator), [`model_repository/mask_decoder/config.pbtxt`](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/mask_decoder/config.pbtxt) (Mask decoder), [`compile.sh`](https://github.com/vivek02sharma/sam3-triton/blob/main/compile.sh) (Engine compiler), [`docker-compose.yml`](https://github.com/vivek02sharma/sam3-triton/blob/main/docker-compose.yml) (Triton service launcher)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_build["Build & Deployment"]
  node_onnx_assets["SAM3 ONNX assets<br/>external model assets"]
  node_compiler["Engine compiler<br/>build script<br/>[compile.sh]"]
  node_engines{{"TensorRT engines<br/>GPU engine artifacts"}}
  node_compose["Triton service launcher<br/>Docker Compose<br/>[docker-compose.yml]"]
  node_dockerfile["Runtime image<br/>Dockerfile"]
end

subgraph group_client["Client"]
  node_streamlit["Streamlit UI<br/>Python client<br/>[app.py]"]
end

subgraph group_triton["Triton Model Repository"]
  node_triton_server{{"NVIDIA Triton<br/>inference server"}}
  node_sam3_contract["SAM3 API contract<br/>Triton model config<br/>[config.pbtxt]"]
  node_sam3_orchestrator["SAM3 orchestrator<br/>Triton Python model<br/>[model.py]"]
  node_vision_encoder["Vision encoder<br/>TensorRT Triton model<br/>[config.pbtxt]"]
  node_text_encoder["Text encoder<br/>TensorRT Triton model<br/>[config.pbtxt]"]
  node_mask_decoder["Mask decoder<br/>TensorRT Triton model<br/>[config.pbtxt]"]
end

node_onnx_assets -->|"input artifacts"| node_compiler
node_compiler -->|"builds"| node_engines
node_dockerfile -->|"defines image"| node_compose
node_compose -->|"starts on GPU 0"| node_triton_server
node_engines -->|"loaded at startup"| node_triton_server
node_streamlit -->|"HTTP inference requests"| node_triton_server
node_triton_server -->|"exposes"| node_sam3_contract
node_sam3_contract -->|"routes requests"| node_sam3_orchestrator
node_sam3_orchestrator -->|"image tensors via DLPack"| node_vision_encoder
node_sam3_orchestrator -->|"prompt tensors via DLPack"| node_text_encoder
node_vision_encoder -->|"image features"| node_mask_decoder
node_text_encoder -->|"text features"| node_mask_decoder
node_mask_decoder -->|"segmentation masks"| node_sam3_orchestrator
node_sam3_orchestrator -->|"inference results"| node_streamlit

click node_compiler "https://github.com/vivek02sharma/sam3-triton/blob/main/compile.sh"
click node_compose "https://github.com/vivek02sharma/sam3-triton/blob/main/docker-compose.yml"
click node_dockerfile "https://github.com/vivek02sharma/sam3-triton/blob/main/Dockerfile"
click node_streamlit "https://github.com/vivek02sharma/sam3-triton/blob/main/app.py"
click node_sam3_contract "https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/sam3_model/config.pbtxt"
click node_sam3_orchestrator "https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/sam3_model/1/model.py"
click node_vision_encoder "https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/vision_encoder/config.pbtxt"
click node_text_encoder "https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/text_encoder/config.pbtxt"
click node_mask_decoder "https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/mask_decoder/config.pbtxt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_onnx_assets,node_compiler,node_engines,node_compose,node_dockerfile toneBlue
class node_streamlit toneAmber
class node_triton_server,node_sam3_contract,node_sam3_orchestrator,node_vision_encoder,node_text_encoder,node_mask_decoder toneMint
```

## Components

### Build & Deployment

- **SAM3 ONNX assets**: external model assets · External source artifacts used to build deployable engines.
- [Engine compiler](https://github.com/vivek02sharma/sam3-triton/blob/main/compile.sh): build script · Builds TensorRT engines for the model repository. · `compile.sh`
- **TensorRT engines**: GPU engine artifacts · Prebuilt inference engines loaded by Triton.
- [Triton service launcher](https://github.com/vivek02sharma/sam3-triton/blob/main/docker-compose.yml): Docker Compose · Runs Triton on GPU 0 and exposes readiness on port 9000. · `docker-compose.yml`
- [Runtime image](https://github.com/vivek02sharma/sam3-triton/blob/main/Dockerfile): Dockerfile · `Dockerfile`

### Client

- [Streamlit UI](https://github.com/vivek02sharma/sam3-triton/blob/main/app.py): Python client · Interactive UI and Triton HTTP client with a 32-connection pool. · `app.py`

### Triton Model Repository

- **NVIDIA Triton**: inference server · Hosts and schedules the SAM3 model stages.
- [SAM3 API contract](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/sam3_model/config.pbtxt): Triton model config · Client-facing SAM3 model interface. · `model_repository/sam3_model/config.pbtxt`
- [SAM3 orchestrator](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/sam3_model/1/model.py): Triton Python model · Coordinates stages and transfers tensors through DLPack on GPU. · `model_repository/sam3_model/1/model.py`
- [Vision encoder](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/vision_encoder/config.pbtxt): TensorRT Triton model · Produces image features. · `model_repository/vision_encoder/config.pbtxt`
- [Text encoder](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/text_encoder/config.pbtxt): TensorRT Triton model · Produces text prompt features. · `model_repository/text_encoder/config.pbtxt`
- [Mask decoder](https://github.com/vivek02sharma/sam3-triton/blob/main/model_repository/mask_decoder/config.pbtxt): TensorRT Triton model · Combines image and text features into segmentation masks. · `model_repository/mask_decoder/config.pbtxt`

## Connections

- SAM3 ONNX assets → Engine compiler: input artifacts
- Engine compiler → TensorRT engines: builds
- Runtime image → Triton service launcher: defines image
- Triton service launcher → NVIDIA Triton: starts on GPU 0
- TensorRT engines → NVIDIA Triton: loaded at startup
- Streamlit UI → NVIDIA Triton: HTTP inference requests
- NVIDIA Triton → SAM3 API contract: exposes
- SAM3 API contract → SAM3 orchestrator: routes requests
- SAM3 orchestrator → Vision encoder: image tensors via DLPack
- SAM3 orchestrator → Text encoder: prompt tensors via DLPack
- Vision encoder → Mask decoder: image features
- Text encoder → Mask decoder: text features
- Mask decoder → SAM3 orchestrator: segmentation masks
- SAM3 orchestrator → Streamlit UI: inference results