# jingkang50/openpsg architecture

> Architecture diagram of the GitHub repository jingkang50/openpsg, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jingkang50/openpsg
- Repository: https://github.com/jingkang50/openpsg

## Overview

OpenPSG implements panoptic scene-graph generation: images and panoptic annotations are loaded and transformed, a configured detector/relation model predicts mask-grounded objects and predicates, and scene-graph metrics or visualizations present the results. The repository also includes a separate CE7454 classification workflow that trains a ResNet-50 predicate classifier and produces ranked test predictions. The README documents multiple model families and panoptic-mask grounding; the graph shows these as supported alternatives, without asserting unsampled internal wiring.

Main entry points: [`ce7454/main.py`](https://github.com/jingkang50/openpsg/blob/main/ce7454/main.py) (Classification Workflow), [`tools/train.py`](https://github.com/jingkang50/openpsg/blob/main/tools/train.py) (Training Entry), [`openpsg/datasets/psg.py`](https://github.com/jingkang50/openpsg/blob/main/openpsg/datasets/psg.py) (Panoptic Scene Graph Dataset), [`openpsg/models/frameworks/sg_panoptic_fpn.py`](https://github.com/jingkang50/openpsg/blob/main/openpsg/models/frameworks/sg_panoptic_fpn.py) (Panoptic FPN)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and Preparation"]
  node_psg_dataset["Panoptic Scene Graph Dataset<br/>[psg.py]"]
  node_sg_dataset["Scene Graph Dataset<br/>[sg.py]"]
  node_data_pipeline["Load and Format"]
end

subgraph group_generation["Scene Graph Generation"]
  node_train_entry["Training Entry<br/>[train.py]"]
  node_model_variants["Model Configurations"]
  node_panoptic_fpn["Panoptic FPN<br/>[sg_panoptic_fpn.py]"]
  node_transformer_models["Transformer Models<br/>[psgtr.py]"]
  node_relation_heads["Relation Heads"]
  node_roi_features["Visual Spatial Features<br/>[visual_spatial.py]"]
end

subgraph group_assessment["Assessment and Display"]
  node_evaluation["Scene Graph Evaluation<br/>[sgg_eval.py]"]
  node_metrics["Graph Metrics<br/>[sgg_metrics.py]"]
  node_visualization["Result Visualization"]
end

subgraph group_coursework["CE7454 Classification"]
  node_course_data["Classification Data<br/>[dataset.py]"]
  node_course_main["Classification Workflow<br/>[main.py]"]
  node_trainer["Classifier Trainer<br/>[trainer.py]"]
  node_classifier_eval["Classifier Evaluator<br/>[evaluator.py]"]
end

node_researcher(("Researcher"))
node_psg_data["PSG Annotations"]
node_predictions["Scene Graph Predictions"]
node_course_result["Ranked Predictions"]

node_researcher -->|"provides annotations"| node_psg_data
node_psg_dataset -->|"loads"| node_psg_data
node_psg_dataset -->|"applies transforms"| node_data_pipeline
node_researcher -->|"starts training"| node_train_entry
node_train_entry -->|"loads configuration"| node_model_variants
node_train_entry -->|"builds dataset"| node_psg_dataset
node_train_entry -->|"builds configured model"| node_panoptic_fpn
node_train_entry -.->|"builds configured model"| node_transformer_models
node_panoptic_fpn -->|"calls relation head"| node_relation_heads
node_transformer_models -.->|"uses relation head"| node_relation_heads
node_relation_heads -.->|"extracts pair features"| node_roi_features
node_panoptic_fpn -->|"produces predictions"| node_predictions
node_transformer_models -.->|"produces predictions"| node_predictions
node_researcher -->|"requests evaluation"| node_evaluation
node_evaluation -->|"computes metrics"| node_metrics
node_evaluation -->|"evaluates against annotations"| node_psg_dataset
node_researcher -->|"requests display"| node_visualization
node_visualization -->|"renders results"| node_predictions
node_researcher -->|"starts classification"| node_course_main
node_course_main -->|"loads splits"| node_course_data
node_course_main -->|"trains classifier"| node_trainer
node_course_main -->|"evaluates classifier"| node_classifier_eval
node_classifier_eval -->|"scores validation data"| node_course_data
node_course_main -->|"submits test data"| node_classifier_eval
node_course_main -->|"writes ranked predictions"| node_course_result

click node_psg_dataset "https://github.com/jingkang50/openpsg/blob/main/openpsg/datasets/psg.py"
click node_sg_dataset "https://github.com/jingkang50/openpsg/blob/main/openpsg/datasets/sg.py"
click node_data_pipeline "https://github.com/jingkang50/openpsg/tree/main/openpsg/datasets/pipelines"
click node_train_entry "https://github.com/jingkang50/openpsg/blob/main/tools/train.py"
click node_model_variants "https://github.com/jingkang50/openpsg/tree/main/configs"
click node_panoptic_fpn "https://github.com/jingkang50/openpsg/blob/main/openpsg/models/frameworks/sg_panoptic_fpn.py"
click node_transformer_models "https://github.com/jingkang50/openpsg/blob/main/openpsg/models/frameworks/psgtr.py"
click node_relation_heads "https://github.com/jingkang50/openpsg/tree/main/openpsg/models/relation_heads"
click node_roi_features "https://github.com/jingkang50/openpsg/blob/main/openpsg/models/roi_extractors/visual_spatial.py"
click node_evaluation "https://github.com/jingkang50/openpsg/blob/main/openpsg/evaluation/sgg_eval.py"
click node_metrics "https://github.com/jingkang50/openpsg/blob/main/openpsg/evaluation/sgg_metrics.py"
click node_visualization "https://github.com/jingkang50/openpsg/tree/main/openpsg/utils/vis_tools"
click node_course_data "https://github.com/jingkang50/openpsg/blob/main/ce7454/dataset.py"
click node_course_main "https://github.com/jingkang50/openpsg/blob/main/ce7454/main.py"
click node_trainer "https://github.com/jingkang50/openpsg/blob/main/ce7454/trainer.py"
click node_classifier_eval "https://github.com/jingkang50/openpsg/blob/main/ce7454/evaluator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_psg_dataset,node_sg_dataset,node_data_pipeline toneBlue
class node_train_entry,node_model_variants,node_panoptic_fpn,node_transformer_models,node_relation_heads,node_roi_features toneAmber
class node_evaluation,node_metrics,node_visualization toneMint
class node_course_data,node_course_main,node_trainer,node_classifier_eval toneRose
class node_researcher,node_psg_data,node_predictions,node_course_result toneIndigo
```

## Components

### Data and Preparation

- [Panoptic Scene Graph Dataset](https://github.com/jingkang50/openpsg/blob/main/openpsg/datasets/psg.py): `openpsg/datasets/psg.py`
- [Scene Graph Dataset](https://github.com/jingkang50/openpsg/blob/main/openpsg/datasets/sg.py): `openpsg/datasets/sg.py`
- [Load and Format](https://github.com/jingkang50/openpsg/tree/main/openpsg/datasets/pipelines): `openpsg/datasets/pipelines`

### Scene Graph Generation

- [Training Entry](https://github.com/jingkang50/openpsg/blob/main/tools/train.py): `tools/train.py`
- [Model Configurations](https://github.com/jingkang50/openpsg/tree/main/configs): `configs`
- [Panoptic FPN](https://github.com/jingkang50/openpsg/blob/main/openpsg/models/frameworks/sg_panoptic_fpn.py): `openpsg/models/frameworks/sg_panoptic_fpn.py`
- [Transformer Models](https://github.com/jingkang50/openpsg/blob/main/openpsg/models/frameworks/psgtr.py): `openpsg/models/frameworks/psgtr.py`
- [Relation Heads](https://github.com/jingkang50/openpsg/tree/main/openpsg/models/relation_heads): `openpsg/models/relation_heads`
- [Visual Spatial Features](https://github.com/jingkang50/openpsg/blob/main/openpsg/models/roi_extractors/visual_spatial.py): `openpsg/models/roi_extractors/visual_spatial.py`

### Assessment and Display

- [Scene Graph Evaluation](https://github.com/jingkang50/openpsg/blob/main/openpsg/evaluation/sgg_eval.py): `openpsg/evaluation/sgg_eval.py`
- [Graph Metrics](https://github.com/jingkang50/openpsg/blob/main/openpsg/evaluation/sgg_metrics.py): `openpsg/evaluation/sgg_metrics.py`
- [Result Visualization](https://github.com/jingkang50/openpsg/tree/main/openpsg/utils/vis_tools): `openpsg/utils/vis_tools`

### CE7454 Classification

- [Classification Data](https://github.com/jingkang50/openpsg/blob/main/ce7454/dataset.py): `ce7454/dataset.py`
- [Classification Workflow](https://github.com/jingkang50/openpsg/blob/main/ce7454/main.py): `ce7454/main.py`
- [Classifier Trainer](https://github.com/jingkang50/openpsg/blob/main/ce7454/trainer.py): `ce7454/trainer.py`
- [Classifier Evaluator](https://github.com/jingkang50/openpsg/blob/main/ce7454/evaluator.py): `ce7454/evaluator.py`

### Other components

- **Researcher**
- **PSG Annotations**
- **Scene Graph Predictions**
- **Ranked Predictions**

## Connections

- Researcher → PSG Annotations: provides annotations
- Panoptic Scene Graph Dataset → PSG Annotations: loads
- Panoptic Scene Graph Dataset → Load and Format: applies transforms
- Researcher → Training Entry: starts training
- Training Entry → Model Configurations: loads configuration
- Training Entry → Panoptic Scene Graph Dataset: builds dataset
- Training Entry → Panoptic FPN: builds configured model
- Training Entry → Transformer Models: builds configured model
- Panoptic FPN → Relation Heads: calls relation head
- Transformer Models → Relation Heads: uses relation head
- Relation Heads → Visual Spatial Features: extracts pair features
- Panoptic FPN → Scene Graph Predictions: produces predictions
- Transformer Models → Scene Graph Predictions: produces predictions
- Researcher → Scene Graph Evaluation: requests evaluation
- Scene Graph Evaluation → Graph Metrics: computes metrics
- Scene Graph Evaluation → Panoptic Scene Graph Dataset: evaluates against annotations
- Researcher → Result Visualization: requests display
- Result Visualization → Scene Graph Predictions: renders results
- Researcher → Classification Workflow: starts classification
- Classification Workflow → Classification Data: loads splits
- Classification Workflow → Classifier Trainer: trains classifier
- Classification Workflow → Classifier Evaluator: evaluates classifier
- Classifier Evaluator → Classification Data: scores validation data
- Classification Workflow → Classifier Evaluator: submits test data
- Classification Workflow → Ranked Predictions: writes ranked predictions