How suprosanna/relationformer works
Project Type and Purpose
• Relationformer is an open-source research codebase for image-to-graph generation, implemented as a unified, one-stage transformer framework.
• It provides dataset-specific branches (vessel_graph, road_network, road_network_rgb, scene_graph) alongside a core “master” overview.
File Structure Analysis
• Top-level:
• .github/ (CI/workflow definitions)
• README.md, LICENSE, .gitignore
• Under version control, developers maintain separate branches or directories per application:
– master (core model and shared utilities)
– vessel_graph (3D vessel extraction scripts and configs)
– road_network (2D binary roads)
– road_network_rgb (2D RGB roads)
– scene_graph (scene graph generation)
Key Architectural Insights from README
• Core model: CNN backbone + Transformer encoder/decoder
• Novel tokens: [obj]-tokens for object set prediction, [rln]-token for relation reasoning
• Data loaders and training/evaluation scripts delivered per dataset branch
• Inherited components: DETR, Deformable-DETR, RTN, GGT
a. Main Components
• Data Input (Images)
• Dataset-Specific Data Loader & Preprocessor
• Backbone CNN (e.g., ResNet)
• Transformer Encoder (shared)
• Transformer Decoder with two token sets:
– [obj]-tokens for node predictions
– [rln]-token for relation predictions
• Loss & Optimization Module (object + relation losses)
• Graph Output (nodes & edges)
b. Relationships & Interactions
– Bidirectional associations between [obj]-tokens and [rln]-token inside Decoder
– Outputs flow into Loss & Optimizer and then loop back for training
c. Architectural Patterns & Principles
– One-stage transformer architecture (no separate object detection + relation module)
– Set-based prediction: unordered object set via [obj]-tokens
– Learnable relation token enabling global and local reasoning
– Monorepo structure with core shared code and dataset extensions
d. Key Technologies & Libraries
– PyTorch, TorchVision
– DETR / Deformable-DETR modules
– Custom dataset utilities per branch
– Inherited graph transformer snippets (RTN, GGT)
• Clear component labels (e.g., “Backbone CNN,” “[rln]-token Module”)
• Blue for core model components
• Green for dataset-specific modules
• Orange for external/inherited libraries