How jingkang50/openpsg works
OpenPSG implements panoptic scene-graph generation: images and panoptic annotations are loaded and transformed, a configured detector/relation model predicts mask-grounded objects and predicates, and scene-graph metrics or visualizations present the results. The repository also includes a separate CE7454 classification workflow that trains a ResNet-50 predicate classifier and produces ranked test predictions. The README documents multiple model families and panoptic-mask grounding; the graph shows these as supported alternatives, without asserting unsampled internal wiring.
Main entry points: main.py Classification Workflow · train.py Training Entry · psg.py Panoptic Scene Graph Dataset · sg_panoptic_fpn.py Panoptic FPN
Components 20
Data and Preparation
- Panoptic Scene Graph Dataset
openpsg/datasets/psg.py - Scene Graph Dataset
openpsg/datasets/sg.py - Load and Format
openpsg/datasets/pipelines
Scene Graph Generation
- Training Entry
tools/train.py - Model Configurations
configs - Panoptic FPN
openpsg/models/frameworks/sg_panoptic_fpn.py - Transformer Models
openpsg/models/frameworks/psgtr.py - Relation Heads
openpsg/models/relation_heads - Visual Spatial Features
openpsg/models/roi_extractors/visual_spatial.py
Assessment and Display
- Scene Graph Evaluation
openpsg/evaluation/sgg_eval.py - Graph Metrics
openpsg/evaluation/sgg_metrics.py - Result Visualization
openpsg/utils/vis_tools
CE7454 Classification
- Classification Data
ce7454/dataset.py - Classification Workflow
ce7454/main.py - Classifier Trainer
ce7454/trainer.py - Classifier Evaluator
ce7454/evaluator.py
Other components
- Researcher
- PSG Annotations
- Scene Graph Predictions
- Ranked Predictions
Connections 25
- Researcher → PSG Annotations: provides annotations
- Panoptic Scene Graph Dataset → PSG Annotations: loads
- Panoptic Scene Graph Dataset → Load and Format: applies transforms
- Researcher → Training Entry: starts training
- Training Entry → Model Configurations: loads configuration
- Training Entry → Panoptic Scene Graph Dataset: builds dataset
- Training Entry → Panoptic FPN: builds configured model
- Training Entry → Transformer Models: builds configured model
- Panoptic FPN → Relation Heads: calls relation head
- Transformer Models → Relation Heads: uses relation head
- Relation Heads → Visual Spatial Features: extracts pair features
- Panoptic FPN → Scene Graph Predictions: produces predictions
- Transformer Models → Scene Graph Predictions: produces predictions
- Researcher → Scene Graph Evaluation: requests evaluation
- Scene Graph Evaluation → Graph Metrics: computes metrics
- Scene Graph Evaluation → Panoptic Scene Graph Dataset: evaluates against annotations
- Researcher → Result Visualization: requests display
- Result Visualization → Scene Graph Predictions: renders results
- Researcher → Classification Workflow: starts classification
- Classification Workflow → Classification Data: loads splits
- Classification Workflow → Classifier Trainer: trains classifier
- Classification Workflow → Classifier Evaluator: evaluates classifier
- Classifier Evaluator → Classification Data: scores validation data
- Classification Workflow → Classifier Evaluator: submits test data
- Classification Workflow → Ranked Predictions: writes ranked predictions