# idea-research/groundingdino architecture

> Architecture diagram of the GitHub repository idea-research/groundingdino, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/idea-research/groundingdino
- Repository: https://github.com/idea-research/groundingdino

## Overview

Grounding DINO is a PyTorch open-set object detector: an image and text prompt are prepared, jointly processed by a vision-language detection model, and returned as phrase-labelled boxes that can be visualized. The graph also preserves the repository’s demo and COCO dataset/evaluation branches. Some internal model wiring is inferred from the module structure and README; unsampled details are left out.

Main entry points: [`demo/inference_on_a_image.py`](https://github.com/idea-research/groundingdino/blob/main/demo/inference_on_a_image.py) (Image CLI), [`groundingdino/models/GroundingDINO/groundingdino.py`](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/groundingdino.py) (Grounding DINO), [`demo/gradio_app.py`](https://github.com/idea-research/groundingdino/blob/main/demo/gradio_app.py) (Gradio Demo), [`groundingdino/util/inference.py`](https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/inference.py) (Inference API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User Interfaces"]
  node_demo_cli["Image CLI"]
  node_gradio["Gradio Demo<br/>[gradio_app.py]"]
end

subgraph group_inference["Inference"]
  node_inference_api["Inference API<br/>[inference.py]"]
  node_transforms["Image Transforms<br/>[transforms.py]"]
  node_model_builder["Model Builder<br/>[__init__.py]"]
  node_config["Model Config"]
  node_postprocess["Box And Phrase Processing<br/>[inference.py]"]
  node_visualizer["Detection Visualization<br/>[visualizer.py]"]
end

subgraph group_detector["Detection Model"]
  node_detector["Grounding DINO<br/>[groundingdino.py]"]
  node_backbone["Swin Backbone<br/>[backbone.py]"]
  node_swin["Swin Transformer"]
  node_transformer["Vision-Text Transformer<br/>[transformer.py]"]
  node_text_warp["Text Encoding<br/>[bertwarper.py]"]
  node_deform_attn["Deformable Attention<br/>[ms_deform_attn.py]"]
end

subgraph group_data["Dataset And Evaluation"]
  node_dataset_demo["COCO Dataset Creator"]
  node_coco_eval["COCO Evaluation"]
end

subgraph group_integrations["External Components"]
  node_transformers["Transformers / BERT"]
end

node_user(("User"))
node_image["Image"]
node_prompt["Text Prompt"]
node_coco_data[("COCO Data")]
node_detections["Boxes And Labels"]
node_annotated["Annotated Image"]

node_user -->|"runs"| node_demo_cli
node_user -->|"uses"| node_gradio
node_image -->|"provides"| node_demo_cli
node_prompt -->|"provides"| node_demo_cli
node_image -->|"provides"| node_gradio
node_prompt -->|"provides"| node_gradio
node_demo_cli -->|"preprocesses"| node_transforms
node_gradio -->|"requests predictions"| node_inference_api
node_inference_api -->|"preprocesses"| node_transforms
node_demo_cli -->|"builds model"| node_model_builder
node_config -->|"configures"| node_model_builder
node_model_builder -->|"constructs"| node_detector
node_inference_api -->|"runs image and text"| node_detector
node_demo_cli -->|"runs image and text"| node_detector
node_detector -->|"extracts image features"| node_backbone
node_backbone -->|"uses"| node_swin
node_detector -->|"encodes prompt"| node_text_warp
node_text_warp -.->|"uses BERT"| node_transformers
node_detector -->|"fuses features"| node_transformer
node_transformer -->|"uses"| node_deform_attn
node_inference_api -->|"filters and labels"| node_postprocess
node_demo_cli -->|"filters and labels"| node_postprocess
node_postprocess -->|"returns"| node_detections
node_demo_cli -->|"renders boxes"| node_visualizer
node_gradio -->|"renders detections"| node_visualizer
node_visualizer -->|"produces"| node_annotated
node_user -->|"prepares data"| node_dataset_demo
node_coco_data -->|"reads"| node_dataset_demo
node_dataset_demo -->|"creates evaluation data"| node_coco_eval
node_coco_data -->|"evaluates against"| node_coco_eval

click node_demo_cli "https://github.com/idea-research/groundingdino/blob/main/demo/inference_on_a_image.py"
click node_gradio "https://github.com/idea-research/groundingdino/blob/main/demo/gradio_app.py"
click node_inference_api "https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/inference.py"
click node_transforms "https://github.com/idea-research/groundingdino/blob/main/groundingdino/datasets/transforms.py"
click node_model_builder "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/__init__.py"
click node_config "https://github.com/idea-research/groundingdino/blob/main/groundingdino/config/GroundingDINO_SwinB_cfg.py"
click node_detector "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/groundingdino.py"
click node_backbone "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/backbone/backbone.py"
click node_swin "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/backbone/swin_transformer.py"
click node_transformer "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/transformer.py"
click node_text_warp "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/bertwarper.py"
click node_deform_attn "https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/ms_deform_attn.py"
click node_postprocess "https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/inference.py"
click node_visualizer "https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/visualizer.py"
click node_dataset_demo "https://github.com/idea-research/groundingdino/blob/main/demo/create_coco_dataset.py"
click node_coco_eval "https://github.com/idea-research/groundingdino/blob/main/groundingdino/datasets/cocogrounding_eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo_cli,node_gradio,node_user toneBlue
class node_inference_api,node_transforms,node_model_builder,node_config,node_postprocess,node_visualizer,node_coco_data toneAmber
class node_detector,node_backbone,node_swin,node_transformer,node_text_warp,node_deform_attn toneMint
class node_dataset_demo,node_coco_eval toneRose
class node_transformers,node_image,node_prompt,node_detections,node_annotated toneIndigo
```

## Components

### User Interfaces

- [Image CLI](https://github.com/idea-research/groundingdino/blob/main/demo/inference_on_a_image.py): `demo/inference_on_a_image.py`
- [Gradio Demo](https://github.com/idea-research/groundingdino/blob/main/demo/gradio_app.py): `demo/gradio_app.py`

### Inference

- [Inference API](https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/inference.py): `groundingdino/util/inference.py`
- [Image Transforms](https://github.com/idea-research/groundingdino/blob/main/groundingdino/datasets/transforms.py): `groundingdino/datasets/transforms.py`
- [Model Builder](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/__init__.py): `groundingdino/models/__init__.py`
- [Model Config](https://github.com/idea-research/groundingdino/blob/main/groundingdino/config/GroundingDINO_SwinB_cfg.py): `groundingdino/config/GroundingDINO_SwinB_cfg.py`
- [Box And Phrase Processing](https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/inference.py): `groundingdino/util/inference.py`
- [Detection Visualization](https://github.com/idea-research/groundingdino/blob/main/groundingdino/util/visualizer.py): `groundingdino/util/visualizer.py`

### Detection Model

- [Grounding DINO](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/groundingdino.py): `groundingdino/models/GroundingDINO/groundingdino.py`
- [Swin Backbone](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/backbone/backbone.py): `groundingdino/models/GroundingDINO/backbone/backbone.py`
- [Swin Transformer](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/backbone/swin_transformer.py): `groundingdino/models/GroundingDINO/backbone/swin_transformer.py`
- [Vision-Text Transformer](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/transformer.py): `groundingdino/models/GroundingDINO/transformer.py`
- [Text Encoding](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/bertwarper.py): `groundingdino/models/GroundingDINO/bertwarper.py`
- [Deformable Attention](https://github.com/idea-research/groundingdino/blob/main/groundingdino/models/GroundingDINO/ms_deform_attn.py): `groundingdino/models/GroundingDINO/ms_deform_attn.py`

### Dataset And Evaluation

- [COCO Dataset Creator](https://github.com/idea-research/groundingdino/blob/main/demo/create_coco_dataset.py): `demo/create_coco_dataset.py`
- [COCO Evaluation](https://github.com/idea-research/groundingdino/blob/main/groundingdino/datasets/cocogrounding_eval.py): `groundingdino/datasets/cocogrounding_eval.py`

### External Components

- **Transformers / BERT**

### Other components

- **User**
- **Image**
- **Text Prompt**
- **COCO Data**
- **Boxes And Labels**
- **Annotated Image**

## Connections

- User → Image CLI: runs
- User → Gradio Demo: uses
- Image → Image CLI: provides
- Text Prompt → Image CLI: provides
- Image → Gradio Demo: provides
- Text Prompt → Gradio Demo: provides
- Image CLI → Image Transforms: preprocesses
- Gradio Demo → Inference API: requests predictions
- Inference API → Image Transforms: preprocesses
- Image CLI → Model Builder: builds model
- Model Config → Model Builder: configures
- Model Builder → Grounding DINO: constructs
- Inference API → Grounding DINO: runs image and text
- Image CLI → Grounding DINO: runs image and text
- Grounding DINO → Swin Backbone: extracts image features
- Swin Backbone → Swin Transformer: uses
- Grounding DINO → Text Encoding: encodes prompt
- Text Encoding → Transformers / BERT: uses BERT
- Grounding DINO → Vision-Text Transformer: fuses features
- Vision-Text Transformer → Deformable Attention: uses
- Inference API → Box And Phrase Processing: filters and labels
- Image CLI → Box And Phrase Processing: filters and labels
- Box And Phrase Processing → Boxes And Labels: returns
- Image CLI → Detection Visualization: renders boxes
- Gradio Demo → Detection Visualization: renders detections
- Detection Visualization → Annotated Image: produces
- User → COCO Dataset Creator: prepares data
- COCO Data → COCO Dataset Creator: reads
- COCO Dataset Creator → COCO Evaluation: creates evaluation data
- COCO Data → COCO Evaluation: evaluates against