# rohithpeddi/worldsgg architecture

> Architecture diagram of the GitHub repository rohithpeddi/worldsgg, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rohithpeddi/worldsgg
- Repository: https://github.com/rohithpeddi/worldsgg

## Overview

This repository is a research codebase for monocular-video World Scene Graph Generation (WSGG), with ActionGenome4D data support and detector/model training and evaluation. The principal workflow is to prepare frame and 3D annotations, train or evaluate a monocular 3D detector, then train a WSGG method on world-scene-graph samples and assess relationship predictions. It also includes 2D detection, several WSGG baselines, and the WorldWise architecture. The README documents the model stages; source excerpts are partial, so some subsystem links are shown as unverified rather than asserting unseen calls.

Main entry points: [`train_wsgg_base.py`](https://github.com/rohithpeddi/worldsgg/blob/main/train_wsgg_base.py) (WSGG Training), [`train_wsgg_methods.py`](https://github.com/rohithpeddi/worldsgg/blob/main/train_wsgg_methods.py) (Method Selection), [`lib/detector/monocular3d/trainer.py`](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/trainer.py) (3D Detector Trainer), [`lib/supervised/worldwise/worldwise.py`](https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/worldwise/worldwise.py) (WorldWise)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Datasets"]
  node_ag_data["ActionGenome Data<br/>[base_ag_dataset.py]"]
  node_world_data["World Graph Data"]
  node_mono_data["3D Detection Data<br/>[ag_dataset_3d.py]"]
end

subgraph group_detection["Detection"]
  node_det2d["2D Detector<br/>[dinov2_torch.py]"]
  node_det2d_train["2D Detector Training<br/>[train.py]"]
  node_mono_model["Monocular 3D Model<br/>[dino_mono_3d.py]"]
  node_mono_train["3D Detector Trainer<br/>[trainer.py]"]
end

subgraph group_wsgg["Scene Graph Models"]
  node_wsgg_entry["WSGG Training<br/>[train_wsgg_base.py]"]
  node_wsgg_base["WSGG Runtime<br/>[wsgg_base.py]"]
  node_worldwise["WorldWise<br/>[worldwise.py]"]
  node_dsgdetr["W-DSGDetr++<br/>[w_dsgdetr_pp.py]"]
  node_sttran["W-STTran<br/>[w_sttran.py]"]
  node_method_select["Method Selection"]
end

subgraph group_evaluation["Evaluation"]
  node_mono_eval["2D/3D Detection Metrics<br/>[evaluate_2d.py]"]
  node_recall_eval["Graph Recall Metrics"]
  node_world_eval["World Graph Evaluation"]
end

node_researcher(("Researcher"))

node_researcher -->|"trains detector"| node_det2d_train
node_researcher -->|"trains detector"| node_mono_train
node_researcher -->|"trains WSGG"| node_wsgg_entry
node_det2d_train -->|"uses model"| node_det2d
node_det2d_train -->|"uses data"| node_ag_data
node_mono_train -->|"uses data"| node_mono_data
node_mono_train -->|"trains model"| node_mono_model
node_mono_train -->|"evaluates model"| node_mono_eval
node_mono_eval -->|"evaluates samples"| node_mono_data
node_wsgg_entry -->|"loads samples"| node_world_data
node_wsgg_entry -->|"uses runtime"| node_wsgg_base
node_wsgg_entry -->|"evaluates graphs"| node_recall_eval
node_method_select -->|"configures training"| node_wsgg_entry
node_method_select -->|"selects method"| node_worldwise
node_method_select -->|"selects method"| node_dsgdetr
node_method_select -->|"selects method"| node_sttran
node_wsgg_base -->|"uses samples"| node_world_data
node_worldwise -.->|"produces predictions for"| node_world_eval
node_dsgdetr -.->|"produces predictions for"| node_world_eval
node_sttran -.->|"produces predictions for"| node_world_eval
node_mono_model -.->|"provides 3D stage"| node_worldwise
node_world_data -.->|"feeds graph stage"| node_worldwise

click node_ag_data "https://github.com/rohithpeddi/worldsgg/blob/main/dataloader/base_ag_dataset.py"
click node_world_data "https://github.com/rohithpeddi/worldsgg/blob/main/dataloader/world_ag_dataset.py"
click node_mono_data "https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/datasets/ag_dataset_3d.py"
click node_det2d "https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/detector2d/dinov2_torch.py"
click node_det2d_train "https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/detector2d/train.py"
click node_mono_model "https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/models/dino_mono_3d.py"
click node_mono_train "https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/trainer.py"
click node_mono_eval "https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/evaluation/evaluate_2d.py"
click node_wsgg_entry "https://github.com/rohithpeddi/worldsgg/blob/main/train_wsgg_base.py"
click node_wsgg_base "https://github.com/rohithpeddi/worldsgg/blob/main/wsgg_base.py"
click node_worldwise "https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/worldwise/worldwise.py"
click node_dsgdetr "https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/baselines/w_dsgdetr/w_dsgdetr_pp.py"
click node_sttran "https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/baselines/w_sttran/w_sttran.py"
click node_method_select "https://github.com/rohithpeddi/worldsgg/blob/main/train_wsgg_methods.py"
click node_recall_eval "https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/evaluation_recall.py"
click node_world_eval "https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/evaluation_recall.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ag_data,node_world_data,node_mono_data toneBlue
class node_det2d,node_det2d_train,node_mono_model,node_mono_train toneAmber
class node_wsgg_entry,node_wsgg_base,node_worldwise,node_dsgdetr,node_sttran,node_method_select toneMint
class node_mono_eval,node_recall_eval,node_world_eval toneRose
class node_researcher toneIndigo
```

## Components

### Datasets

- [ActionGenome Data](https://github.com/rohithpeddi/worldsgg/blob/main/dataloader/base_ag_dataset.py): `dataloader/base_ag_dataset.py`
- [World Graph Data](https://github.com/rohithpeddi/worldsgg/blob/main/dataloader/world_ag_dataset.py): `dataloader/world_ag_dataset.py`
- [3D Detection Data](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/datasets/ag_dataset_3d.py): `lib/detector/monocular3d/datasets/ag_dataset_3d.py`

### Detection

- [2D Detector](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/detector2d/dinov2_torch.py): `lib/detector/detector2d/dinov2_torch.py`
- [2D Detector Training](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/detector2d/train.py): `lib/detector/detector2d/train.py`
- [Monocular 3D Model](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/models/dino_mono_3d.py): `lib/detector/monocular3d/models/dino_mono_3d.py`
- [3D Detector Trainer](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/trainer.py): `lib/detector/monocular3d/trainer.py`

### Scene Graph Models

- [WSGG Training](https://github.com/rohithpeddi/worldsgg/blob/main/train_wsgg_base.py): `train_wsgg_base.py`
- [WSGG Runtime](https://github.com/rohithpeddi/worldsgg/blob/main/wsgg_base.py): `wsgg_base.py`
- [WorldWise](https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/worldwise/worldwise.py): `lib/supervised/worldwise/worldwise.py`
- [W-DSGDetr++](https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/baselines/w_dsgdetr/w_dsgdetr_pp.py): `lib/supervised/baselines/w_dsgdetr/w_dsgdetr_pp.py`
- [W-STTran](https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/baselines/w_sttran/w_sttran.py): `lib/supervised/baselines/w_sttran/w_sttran.py`
- [Method Selection](https://github.com/rohithpeddi/worldsgg/blob/main/train_wsgg_methods.py): `train_wsgg_methods.py`

### Evaluation

- [2D/3D Detection Metrics](https://github.com/rohithpeddi/worldsgg/blob/main/lib/detector/monocular3d/evaluation/evaluate_2d.py): `lib/detector/monocular3d/evaluation/evaluate_2d.py`
- [Graph Recall Metrics](https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/evaluation_recall.py): `lib/supervised/evaluation_recall.py`
- [World Graph Evaluation](https://github.com/rohithpeddi/worldsgg/blob/main/lib/supervised/evaluation_recall.py): `lib/supervised/evaluation_recall.py`

### Other components

- **Researcher**

## Connections

- Researcher → 2D Detector Training: trains detector (evidence: `lib/detector/detector2d/train.py`)
- Researcher → 3D Detector Trainer: trains detector (evidence: `lib/detector/monocular3d/trainer.py`)
- Researcher → WSGG Training: trains WSGG (evidence: `train_wsgg_base.py`)
- 2D Detector Training → 2D Detector: uses model (evidence: `lib/detector/detector2d/train.py`)
- 2D Detector Training → ActionGenome Data: uses data (evidence: `lib/detector/detector2d/train.py`)
- 3D Detector Trainer → 3D Detection Data: uses data (evidence: `lib/detector/monocular3d/trainer.py`)
- 3D Detector Trainer → Monocular 3D Model: trains model (evidence: `lib/detector/monocular3d/trainer.py`)
- 3D Detector Trainer → 2D/3D Detection Metrics: evaluates model (evidence: `lib/detector/monocular3d/trainer.py`)
- 2D/3D Detection Metrics → 3D Detection Data: evaluates samples (evidence: `lib/detector/monocular3d/evaluation/evaluate_2d.py`)
- WSGG Training → World Graph Data: loads samples (evidence: `train_wsgg_base.py`)
- WSGG Training → WSGG Runtime: uses runtime (evidence: `train_wsgg_base.py`)
- WSGG Training → Graph Recall Metrics: evaluates graphs (evidence: `train_wsgg_base.py`)
- Method Selection → WSGG Training: configures training (evidence: `train_wsgg_methods.py`)
- Method Selection → WorldWise: selects method (evidence: `train_wsgg_methods.py`)
- Method Selection → W-DSGDetr++: selects method (evidence: `train_wsgg_methods.py`)
- Method Selection → W-STTran: selects method (evidence: `train_wsgg_methods.py`)
- WSGG Runtime → World Graph Data: uses samples (evidence: `wsgg_base.py`)
- WorldWise → World Graph Evaluation: produces predictions for
- W-DSGDetr++ → World Graph Evaluation: produces predictions for
- W-STTran → World Graph Evaluation: produces predictions for
- Monocular 3D Model → WorldWise: provides 3D stage (evidence: `README.md`)
- World Graph Data → WorldWise: feeds graph stage (evidence: `README.md`)
