# suprosanna/relationformer architecture

> Architecture diagram of the GitHub repository suprosanna/relationformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/suprosanna/relationformer
- Repository: https://github.com/suprosanna/relationformer

## Overview

##### Project Type and Purpose
- Relationformer is an open-source research codebase for image-to-graph generation, implemented as a unified, one-stage transformer framework.
- It provides dataset-specific branches (vessel_graph, road_network, road_network_rgb, scene_graph) alongside a core “master” overview.

##### File Structure Analysis
- Top-level:
- .github/ (CI/workflow definitions)
- README.md, LICENSE, .gitignore
- Under version control, developers maintain separate branches or directories per application:
– master (core model and shared utilities)
– vessel_graph (3D vessel extraction scripts and configs)
– road_network (2D binary roads)
– road_network_rgb (2D RGB roads)
– scene_graph (scene graph generation)

##### Key Architectural Insights from README
- Core model: CNN backbone + Transformer encoder/decoder
- Novel tokens: [obj]-tokens for object set prediction, [rln]-token for relation reasoning
- Data loaders and training/evaluation scripts delivered per dataset branch
- Inherited components: DETR, Deformable-DETR, RTN, GGT

a. Main Components
- Data Input (Images)
- Dataset-Specific Data Loader & Preprocessor
- Backbone CNN (e.g., ResNet)
- Transformer Encoder (shared)
- Transformer Decoder with two token sets:
– [obj]-tokens for node predictions
– [rln]-token for relation predictions
- Loss & Optimization Module (object + relation losses)
- Graph Output (nodes & edges)

b. Relationships & Interactions
– Bidirectional associations between [obj]-tokens and [rln]-token inside Decoder
– Outputs flow into Loss & Optimizer and then loop back for training

c. Architectural Patterns & Principles
– One-stage transformer architecture (no separate object detection + relation module)
– Set-based prediction: unordered object set via [obj]-tokens
– Learnable relation token enabling global and local reasoning
– Monorepo structure with core shared code and dataset extensions

d. Key Technologies & Libraries
– PyTorch, TorchVision
– DETR / Deformable-DETR modules
– Custom dataset utilities per branch
– Inherited graph transformer snippets (RTN, GGT)

- Clear component labels (e.g., “Backbone CNN,” “[rln]-token Module”)
- Blue for core model components
- Green for dataset-specific modules
- Orange for external/inherited libraries

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Repository Metadata
    subgraph "Repository Metadata"
        direction TB
        CI[".github/"]:::external
        DOC["README.md"]:::external
        LIC["LICENSE"]:::external
    end

    %% Dataset-Specific Modules
    subgraph "Dataset Branches"
        direction TB
        VG["Vessel Graph Loader"]:::dataset
        RN["Road Network Loader"]:::dataset
        RNR["Road Network RGB Loader"]:::dataset
        SG["Scene Graph Loader"]:::dataset
    end

    %% Core Relationformer Model
    subgraph "Core Model" 
        direction TB
        DI["Data Input (Images)"]:::core
        BP["Backbone CNN"]:::core
        EN["Transformer Encoder"]:::core
        subgraph "Transformer Decoder"
            direction TB
            OBJ["[obj]-tokens Module"]:::core
            RLN["[rln]-token Module"]:::core
        end
        LO["Loss & Optimization Module"]:::core
        GO["Graph Output (Nodes & Edges)"]:::core
    end

    %% External Libraries
    subgraph "External Libraries"
        direction TB
        PT["PyTorch"]:::external
        TV["TorchVision"]:::external
        DETR["DETR Modules"]:::external
        DDETR["Deformable-DETR"]:::external
        RTNLib["RTN Snippets"]:::external
        GGTLib["GGT Snippets"]:::external
    end

    %% Data Flow from Dataset Loaders into Core
    VG -->|loads images| DI
    RN -->|loads images| DI
    RNR -->|loads images| DI
    SG -->|loads images| DI

    %% Core Pipeline Flow
    DI -->|features| BP
    BP -->|embeddings| EN
    EN -->|encodings| OBJ
    EN -->|encodings| RLN
    OBJ <-->|mutual attention| RLN
    OBJ -->|predictions| LO
    RLN -->|predictions| LO
    LO -->|gradients| BP
    LO -->|graph| GO

    %% External Libraries Integration
    PT --> BP
    TV --> BP
    DETR --> EN
    DDETR --> EN
    RTNLib --> OBJ
    GGTLib --> OBJ

    %% Click Events
    click CI "https://github.com/suprosanna/relationformer/tree/master/.github/"
    click DOC "https://github.com/suprosanna/relationformer/blob/master/README.md"
    click LIC "https://github.com/suprosanna/relationformer/tree/master/LICENSE"

    %% Styling
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef dataset fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
```
