# facebookresearch/segment-anything architecture

> Architecture diagram of the GitHub repository facebookresearch/segment-anything, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookresearch/segment-anything
- Repository: https://github.com/facebookresearch/segment-anything

## Overview

SAM is a promptable image-segmentation model: users load a checkpoint, provide an image and point/box/mask prompts, and receive masks plus quality scores; automatic generation samples prompts and filters masks across an image. The graph also preserves the documented command-line, ONNX export, and browser-demo capabilities. Core predictor/model relationships are supported by excerpts and the source index; the demo implementation and some utility wiring were not sampled, so those links are documented or left unverified.

Main entry points: [`segment_anything/modeling/sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/sam.py) (SAM orchestration), [`segment_anything/predictor.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py) (Prompt predictor), [`segment_anything/modeling/mask_decoder.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/mask_decoder.py) (Mask decoder), [`segment_anything/automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py) (Automatic generator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_api["User APIs"]
  node_model_factory["Model registry<br/>[build_sam.py]"]
  node_predictor["Prompt predictor<br/>[predictor.py]"]
  node_transforms["Image transforms<br/>[transforms.py]"]
end

subgraph group_g_model["SAM Model"]
  node_sam["SAM orchestration<br/>[sam.py]"]
  node_image_encoder["ViT image encoder<br/>[image_encoder.py]"]
  node_prompt_encoder["Prompt encoder<br/>[prompt_encoder.py]"]
  node_mask_decoder["Mask decoder<br/>[mask_decoder.py]"]
  node_transformer["Two-way transformer<br/>[transformer.py]"]
  node_common["Model layers<br/>[common.py]"]
end

subgraph group_g_auto["Automatic Masks"]
  node_automatic["Automatic generator"]
  node_cli["Mask CLI<br/>[amg.py]"]
  node_amg_utils["Mask utilities<br/>[amg.py]"]
end

subgraph group_g_deploy["Export And Demo"]
  node_export_cli["ONNX export CLI"]
  node_onnx_model["ONNX wrapper<br/>[onnx.py]"]
  node_browser_demo["Browser demo"]
  node_onnx_runtime["ONNX Runtime"]
end

node_user(("Image user"))

node_user -->|"loads checkpoint"| node_model_factory
node_model_factory -->|"builds model"| node_sam
node_user -->|"sets image and prompts"| node_predictor
node_predictor -->|"transforms inputs"| node_transforms
node_predictor -->|"requests masks"| node_sam
node_sam -->|"encodes image"| node_image_encoder
node_sam -->|"encodes prompts"| node_prompt_encoder
node_sam -->|"decodes masks"| node_mask_decoder
node_mask_decoder -.->|"attends"| node_transformer
node_image_encoder -->|"uses layers"| node_common
node_mask_decoder -->|"uses layers"| node_common
node_transformer -->|"uses layers"| node_common
node_user -->|"requests all masks"| node_automatic
node_automatic -->|"predicts sampled prompts"| node_predictor
node_automatic -->|"filters and encodes"| node_amg_utils
node_user -->|"runs image command"| node_cli
node_cli -.->|"generates masks"| node_automatic
node_user -->|"requests export"| node_export_cli
node_export_cli -->|"exports wrapper"| node_onnx_model
node_onnx_model -->|"uses prompt encoding"| node_prompt_encoder
node_onnx_model -->|"decodes masks"| node_mask_decoder
node_browser_demo -.->|"runs exported model"| node_onnx_runtime
node_user -->|"uses web demo"| node_browser_demo

click node_model_factory "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/build_sam.py"
click node_predictor "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py"
click node_automatic "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py"
click node_cli "https://github.com/facebookresearch/segment-anything/blob/main/scripts/amg.py"
click node_amg_utils "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/utils/amg.py"
click node_sam "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/sam.py"
click node_image_encoder "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/image_encoder.py"
click node_prompt_encoder "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/prompt_encoder.py"
click node_mask_decoder "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/mask_decoder.py"
click node_transformer "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/transformer.py"
click node_common "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/common.py"
click node_transforms "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/utils/transforms.py"
click node_export_cli "https://github.com/facebookresearch/segment-anything/blob/main/scripts/export_onnx_model.py"
click node_onnx_model "https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/utils/onnx.py"
click node_browser_demo "https://github.com/facebookresearch/segment-anything/tree/main/demo/src"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_model_factory,node_predictor,node_transforms,node_user toneBlue
class node_sam,node_image_encoder,node_prompt_encoder,node_mask_decoder,node_transformer,node_common toneAmber
class node_automatic,node_cli,node_amg_utils toneMint
class node_export_cli,node_onnx_model,node_browser_demo,node_onnx_runtime toneRose
```

## Components

### User APIs

- [Model registry](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/build_sam.py): `segment_anything/build_sam.py`
- [Prompt predictor](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py): `segment_anything/predictor.py`
- [Image transforms](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/utils/transforms.py): `segment_anything/utils/transforms.py`

### SAM Model

- [SAM orchestration](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/sam.py): `segment_anything/modeling/sam.py`
- [ViT image encoder](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/image_encoder.py): `segment_anything/modeling/image_encoder.py`
- [Prompt encoder](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/prompt_encoder.py): `segment_anything/modeling/prompt_encoder.py`
- [Mask decoder](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/mask_decoder.py): `segment_anything/modeling/mask_decoder.py`
- [Two-way transformer](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/transformer.py): `segment_anything/modeling/transformer.py`
- [Model layers](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/common.py): `segment_anything/modeling/common.py`

### Automatic Masks

- [Automatic generator](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py): `segment_anything/automatic_mask_generator.py`
- [Mask CLI](https://github.com/facebookresearch/segment-anything/blob/main/scripts/amg.py): `scripts/amg.py`
- [Mask utilities](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/utils/amg.py): `segment_anything/utils/amg.py`

### Export And Demo

- [ONNX export CLI](https://github.com/facebookresearch/segment-anything/blob/main/scripts/export_onnx_model.py): `scripts/export_onnx_model.py`
- [ONNX wrapper](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/utils/onnx.py): `segment_anything/utils/onnx.py`
- [Browser demo](https://github.com/facebookresearch/segment-anything/tree/main/demo/src): `demo/src`
- **ONNX Runtime**

### Other components

- **Image user**

## Connections

- Image user → Model registry: loads checkpoint (evidence: `README.md`)
- Model registry → SAM orchestration: builds model (evidence: `segment_anything/build_sam.py`)
- Image user → Prompt predictor: sets image and prompts (evidence: `README.md`)
- Prompt predictor → Image transforms: transforms inputs (evidence: `segment_anything/predictor.py`)
- Prompt predictor → SAM orchestration: requests masks (evidence: `segment_anything/predictor.py`)
- SAM orchestration → ViT image encoder: encodes image (evidence: `segment_anything/modeling/sam.py`)
- SAM orchestration → Prompt encoder: encodes prompts (evidence: `segment_anything/modeling/sam.py`)
- SAM orchestration → Mask decoder: decodes masks (evidence: `segment_anything/modeling/sam.py`)
- Mask decoder → Two-way transformer: attends
- ViT image encoder → Model layers: uses layers (evidence: `segment_anything/modeling/image_encoder.py`)
- Mask decoder → Model layers: uses layers (evidence: `segment_anything/modeling/mask_decoder.py`)
- Two-way transformer → Model layers: uses layers (evidence: `segment_anything/modeling/transformer.py`)
- Image user → Automatic generator: requests all masks (evidence: `README.md`)
- Automatic generator → Prompt predictor: predicts sampled prompts (evidence: `segment_anything/automatic_mask_generator.py`)
- Automatic generator → Mask utilities: filters and encodes (evidence: `segment_anything/automatic_mask_generator.py`)
- Image user → Mask CLI: runs image command (evidence: `README.md`)
- Mask CLI → Automatic generator: generates masks
- Image user → ONNX export CLI: requests export (evidence: `README.md`)
- ONNX export CLI → ONNX wrapper: exports wrapper (evidence: `scripts/export_onnx_model.py`)
- ONNX wrapper → Prompt encoder: uses prompt encoding (evidence: `segment_anything/utils/onnx.py`)
- ONNX wrapper → Mask decoder: decodes masks (evidence: `segment_anything/utils/onnx.py`)
- Browser demo → ONNX Runtime: runs exported model (evidence: `README.md`)
- Image user → Browser demo: uses web demo (evidence: `README.md`)
