How rohithpeddi/scenesayer works
SceneSayer is a research system for anticipating future Action Genome scene graphs. Its main workflow prepares video-frame annotations, loads scene data, extracts object detections and features, then trains or evaluates scene-graph and anticipation models in sgdet, sgcls, or predcls settings. Model families include STTran/DSG-DETR variants and ODE/SDE approaches; evaluation reports recall metrics, while a separate results workflow can retrieve and format experiment results. The graph follows documented capabilities and indexed references; some unsampled model and preprocessing internals are represented at their exact enclosing directories.
Main entry points: train_sga_base.py Training Entry · object_detector.py Object Detection · blocks.py Transformer Blocks · prepare_paper_results.py Result Analysis
Components 20
Action Genome Data
- Frame Preparation
datasets/action_genome/tools - Feature Preprocessing
datasets/action_genome/preprocess - Scene Dataset
dataloader/action_genome/ag_dataset.py
Visual Perception
- Object Detection
lib/object_detector.py - Faster R-CNN
fasterRCNN/lib/model/faster_rcnn/resnet.py - ROI and NMS
fasterRCNN/lib/model/roi_layers/nms.py - Box Operations
lib/fpn/box_utils.py
Scene Graph Models
- Training Entry
train_sga_base.py - Object Embeddings
lib/word_vectors.py - Transformer Blocks
lib/supervised/sga/blocks.py - Baseline Anticipation
0_dump/baseline_anticipation.py - Anticipation Methods
lib/supervised/sga - DSG-DETR
lib/supervised/sgg/dsgdetr/dsgdetr.py
Evaluation Results
- Recall Evaluation
lib/supervised/evaluation_recall.py - Result Analysis
analysis/eccv/prepare_paper_results.py - Result Records
analysis/results/Result.py - Firebase Service
analysis/results/FirebaseService.py
Other components
- Researcher
- Action Genome
- Firebase Database
Connections 19
- Researcher → Training Entry: runs training
README.md - Action Genome → Frame Preparation: provides videos
README.md - Frame Preparation → Scene Dataset: prepares frames
- Action Genome → Feature Preprocessing: provides annotations
- Training Entry → Scene Dataset: loads scenes
train_sga_base.py - Training Entry → Object Detection: uses detector
train_sga_base.py - Object Detection → Faster R-CNN: runs backbone
lib/object_detector.py - Object Detection → ROI and NMS: filters boxes
lib/object_detector.py - Training Entry → Anticipation Methods: selects method
README.md - Training Entry → DSG-DETR: trains variants
- Training Entry → Baseline Anticipation: trains baseline
- Baseline Anticipation → Transformer Blocks: uses blocks
0_dump/baseline_anticipation.py - Transformer Blocks → Object Embeddings: embeds objects
lib/supervised/sga/blocks.py - Transformer Blocks → Box Operations: uses geometry
lib/supervised/sga/blocks.py - Training Entry → Recall Evaluation: evaluates predictions
- Researcher → Result Analysis: reviews results
- Result Analysis → Firebase Service: fetches results
analysis/eccv/prepare_paper_results.py - Result Analysis → Result Records: parses records
analysis/eccv/prepare_paper_results.py - Firebase Service → Firebase Database: reads and writes
analysis/results/FirebaseService.py