# gpt4vision/ovsgtr architecture

> Architecture diagram of the GitHub repository gpt4vision/ovsgtr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gpt4vision/ovsgtr
- Repository: https://github.com/gpt4vision/ovsgtr

## Overview

##### Project Type and Purpose
- This is a research‐oriented deep‐learning codebase for **open-vocabulary Scene Graph Generation (OvSGG)**. It integrates the GroundingDINO object detector and implements the OvSGTR model to extend scene graph generation to novel concepts.

##### File-Structure Analysis
- Top-level directories:
- GroundingDINO/… → a subpackage containing source, demo scripts, and setup for the GroundingDINO detector.
- config/… → Python config files for different training/eval modes (SwinT/SwinB, OvD, OvR, joint).
- datasets/… → dataset definitions, transforms, evaluation code for VG, COCO, etc.
- models/… → registry and wrapper for the OvSGTR model.
- engine.py & main.py → training/evaluation entry points.
- scripts/… → bash scripts to launch distributed training/evaluation.
- tools/… → Jupyter notebooks and scripts for data preprocessing or language parsing.
- util/… → generic utilities (logging, I/O, timers, plotting).
- figures/ & paper/ → static assets and publication PDFs.

##### Architectural Insights from README
- **Core Components**:
- **Detector**: GroundingDINO
- **SGG Model**: OvSGTR implemented under groundingdino/models
- **Config-driven**: All training/inference behaviors are parameterized by config/*.py
- **Dataset Pipeline**: load → transform → batch → feed into model → post-process
- **Training Engine**: engine.py orchestrates forward/backward, logging, checkpointing
- **Evaluation**: test scripts compute R@20/50/100 metrics per mode (Closed, OvD, OvR, Joint)

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Ingestion
    subgraph "Data Ingestion"
        DSources["Data Sources\nCOCO, VG150, MegaSG"]:::io
        DatasetModule["Dataset Module\ndatasets/*.py"]:::io
    end

    %% Configuration
    subgraph "Configuration"
        ConfigFiles["Config Files\nconfig/*.py"]:::control
        SLConfigUtil["SLConfig & SLIO\nutil/slconfig.py, util/slio.py"]:::control
        DetSLConfigUtil["Detector SLConfig & SLIO\ngroundingdino/util/slconfig.py, slio.py"]:::control
    end

    %% Detection
    subgraph "GroundingDINO Detector"
        Detector["Detector Core\ngroundingdino package"]:::compute
        DetectorDemo["Demo Scripts\nGroundingDINO/demo/"]:::compute
        DetectorSetup["Setup & Requirements\nGroundingDINO/setup.py,\nrequirements.txt"]:::compute
    end

    %% Model
    subgraph "OvSGTR Model"
        Registry["Registry & Wrappers\nmodels/registry.py"]:::compute
        ModelFolder["Model Implementations\nmodels/GroundingDINO/"]:::compute
    end

    %% Engine
    subgraph "Training Engine"
        Engine["Engine\nengine.py & main.py"]:::control
    end

    %% Inference & Evaluation
    subgraph "Inference & Evaluation"
        BashScripts["Bash Scripts\nscripts/DINO_eval.sh,\nscripts/DINO_eval_dist.sh"]:::compute
        DemoEval["Demo & Eval Scripts\nGroundingDINO/demo/inference_on_a_image.py,\ntest_ap_on_coco.py"]:::compute
    end

    %% Utilities
    subgraph "Utilities"
        UtilProj["Project Utilities\nutil/"]:::util
        UtilDet["Detector Utilities\ngroundingdino/util/"]:::util
    end

    %% Tools
    subgraph "Preprocessing & Parsing Tools"
        Tools["Tools\ntools/"]:::util
    end

    %% Assets
    subgraph "Figures & Paper"
        Figures["Figures\nfigures/"]:::util
        Paper["Paper\npaper/"]:::util
    end

    %% Outputs
    Outputs["Outputs\nCheckpoints, logs/, plots, metrics"]:::io

    %% Data Flow
    DSources -->|load data| DatasetModule
    DatasetModule -->|preprocessed batches| Registry
    Detector -->|proposals + embeddings| ModelFolder
    Registry -->|wrap & register| ModelFolder
    Engine -->|train/infer| Detector
    Engine -->|train/infer| ModelFolder
    Engine -->|save & log| UtilProj
    BashScripts -->|load checkpoint| Detector
    BashScripts -->|run evaluation| ModelFolder
    DemoEval -->|run inference| Detector
    DemoEval -->|run inference| ModelFolder
    DemoEval -->|produce metrics| Outputs

    %% Config injections (dashed)
    ConfigFiles -.->|params| DatasetModule
    ConfigFiles -.->|params| Detector
    ConfigFiles -.->|params| ModelFolder
    ConfigFiles -.->|params| Engine
    ConfigFiles -.->|params| BashScripts
    ConfigFiles -.->|params| DemoEval

    %% Utilities connections
    UtilProj -->|ploting, logging| Outputs
    UtilDet -->|visualize| Detector

    %% Click Events
    click DatasetModule "https://github.com/gpt4vision/ovsgtr/tree/master/datasets/"
    click ConfigFiles "https://github.com/gpt4vision/ovsgtr/tree/master/config/"
    click SLConfigUtil "https://github.com/gpt4vision/ovsgtr/blob/master/util/slconfig.py"
    click SLConfigUtil "https://github.com/gpt4vision/ovsgtr/blob/master/util/slio.py"
    click DetSLConfigUtil "https://github.com/gpt4vision/ovsgtr/blob/master/GroundingDINO/groundingdino/util/slconfig.py"
    click DetSLConfigUtil "https://github.com/gpt4vision/ovsgtr/blob/master/GroundingDINO/groundingdino/util/slio.py"
    click Detector "https://github.com/gpt4vision/ovsgtr/tree/master/GroundingDINO/groundingdino/"
    click DetectorDemo "https://github.com/gpt4vision/ovsgtr/tree/master/GroundingDINO/demo/"
    click DetectorSetup "https://github.com/gpt4vision/ovsgtr/blob/master/GroundingDINO/setup.py"
    click DetectorSetup "https://github.com/gpt4vision/ovsgtr/blob/master/GroundingDINO/requirements.txt"
    click Registry "https://github.com/gpt4vision/ovsgtr/blob/master/models/registry.py"
    click ModelFolder "https://github.com/gpt4vision/ovsgtr/tree/master/models/GroundingDINO/"
    click Engine "https://github.com/gpt4vision/ovsgtr/blob/master/engine.py"
    click Engine "https://github.com/gpt4vision/ovsgtr/blob/master/main.py"
    click BashScripts "https://github.com/gpt4vision/ovsgtr/blob/master/scripts/DINO_eval.sh"
    click BashScripts "https://github.com/gpt4vision/ovsgtr/blob/master/scripts/DINO_eval_dist.sh"
    click DemoEval "https://github.com/gpt4vision/ovsgtr/blob/master/GroundingDINO/demo/inference_on_a_image.py"
    click DemoEval "https://github.com/gpt4vision/ovsgtr/blob/master/test_ap_on_coco.py"
    click UtilProj "https://github.com/gpt4vision/ovsgtr/tree/master/util/"
    click UtilDet "https://github.com/gpt4vision/ovsgtr/tree/master/GroundingDINO/groundingdino/util/"
    click Tools "https://github.com/gpt4vision/ovsgtr/tree/master/tools/"
    click Figures "https://github.com/gpt4vision/ovsgtr/tree/master/figures/"
    click Paper "https://github.com/gpt4vision/ovsgtr/tree/master/paper/"

    %% Styles
    classDef compute fill:#D0E7FF,stroke:#3182CE,color:#1B365D;
    classDef io fill:#DFFFE0,stroke:#2F855A,color:#22543D;
    classDef control fill:#FFF5D1,stroke:#DD6B20,color:#662E1A;
    classDef util fill:#E2E8F0,stroke:#4A5568,color:#2D3748;
```
