How rohithpeddi/worldsgg works
This repository is a research codebase for monocular-video World Scene Graph Generation (WSGG), with ActionGenome4D data support and detector/model training and evaluation. The principal workflow is to prepare frame and 3D annotations, train or evaluate a monocular 3D detector, then train a WSGG method on world-scene-graph samples and assess relationship predictions. It also includes 2D detection, several WSGG baselines, and the WorldWise architecture. The README documents the model stages; source excerpts are partial, so some subsystem links are shown as unverified rather than asserting unseen calls.
Main entry points: train_wsgg_base.py WSGG Training · train_wsgg_methods.py Method Selection · trainer.py 3D Detector Trainer · worldwise.py WorldWise
Components 17
Datasets
- ActionGenome Data
dataloader/base_ag_dataset.py - World Graph Data
dataloader/world_ag_dataset.py - 3D Detection Data
lib/detector/monocular3d/datasets/ag_dataset_3d.py
Detection
- 2D Detector
lib/detector/detector2d/dinov2_torch.py - 2D Detector Training
lib/detector/detector2d/train.py - Monocular 3D Model
lib/detector/monocular3d/models/dino_mono_3d.py - 3D Detector Trainer
lib/detector/monocular3d/trainer.py
Scene Graph Models
- WSGG Training
train_wsgg_base.py - WSGG Runtime
wsgg_base.py - WorldWise
lib/supervised/worldwise/worldwise.py - W-DSGDetr++
lib/supervised/baselines/w_dsgdetr/w_dsgdetr_pp.py - W-STTran
lib/supervised/baselines/w_sttran/w_sttran.py - Method Selection
train_wsgg_methods.py
Evaluation
- 2D/3D Detection Metrics
lib/detector/monocular3d/evaluation/evaluate_2d.py - Graph Recall Metrics
lib/supervised/evaluation_recall.py - World Graph Evaluation
lib/supervised/evaluation_recall.py
Other components
- Researcher
Connections 22
- Researcher → 2D Detector Training: trains detector
lib/detector/detector2d/train.py - Researcher → 3D Detector Trainer: trains detector
lib/detector/monocular3d/trainer.py - Researcher → WSGG Training: trains WSGG
train_wsgg_base.py - 2D Detector Training → 2D Detector: uses model
lib/detector/detector2d/train.py - 2D Detector Training → ActionGenome Data: uses data
lib/detector/detector2d/train.py - 3D Detector Trainer → 3D Detection Data: uses data
lib/detector/monocular3d/trainer.py - 3D Detector Trainer → Monocular 3D Model: trains model
lib/detector/monocular3d/trainer.py - 3D Detector Trainer → 2D/3D Detection Metrics: evaluates model
lib/detector/monocular3d/trainer.py - 2D/3D Detection Metrics → 3D Detection Data: evaluates samples
lib/detector/monocular3d/evaluation/evaluate_2d.py - WSGG Training → World Graph Data: loads samples
train_wsgg_base.py - WSGG Training → WSGG Runtime: uses runtime
train_wsgg_base.py - WSGG Training → Graph Recall Metrics: evaluates graphs
train_wsgg_base.py - Method Selection → WSGG Training: configures training
train_wsgg_methods.py - Method Selection → WorldWise: selects method
train_wsgg_methods.py - Method Selection → W-DSGDetr++: selects method
train_wsgg_methods.py - Method Selection → W-STTran: selects method
train_wsgg_methods.py - WSGG Runtime → World Graph Data: uses samples
wsgg_base.py - WorldWise → World Graph Evaluation: produces predictions for
- W-DSGDetr++ → World Graph Evaluation: produces predictions for
- W-STTran → World Graph Evaluation: produces predictions for
- Monocular 3D Model → WorldWise: provides 3D stage
README.md - World Graph Data → WorldWise: feeds graph stage
README.md