How cvpr25paper9858/ges3vig works
Project type and purpose
• From the README and file tree, Ges3ViG is a research‐oriented deep learning codebase for 3D visual grounding that incorporates human pointing gestures.
• It is built on PyTorch + PyTorch Lightning, uses Hydra for config management, and relies on CUDA extensions (MinkowskiEngine, custom C++/CUDA ops) plus external tools (ScanNet v2, ImputeRefer dataset, pre-trained PointGroup detector, WandB for logging).
File structure
• Top‐level scripts:
• train.py, test.py, evaluate.py – entry points for training, inference, evaluation
• setup.py – package installer
• Imputer/ – standalone CUDA C++/Python package to generate human‐pose imputations
• Configuration: config/ holds Hydra YAML files for datasets, models, global settings
• Data pipelines: dataset/ contains subfolders for ScanNet v2, ScanRefer, ImputeRefer, each with preprocessing & conversion scripts
• Core library: ges3vig/ package
• data/ – PyTorch Lightning DataModule & datasets
• model/ – vision, language, cross‐modal modules, the main Ges3ViG class
• loss/, evaluation/, util/ – supporting components
• common_ops/ – custom CUDA kernels for clustering, ROI‐pool, etc., compiled via its own setup.py
README for additional insights
• The README details data preparation workflows, external dependencies (ScanNet, human models, pre-trained detector), and how Hydra config keys map to scripts.