How esw0116/odgs works
Project Type and Purpose
• ODGS is a research‐oriented full-stack application for per‐scene optimization of 3D Gaussian Splatting models from omnidirectional images.
• It ingests a sequence of 360° images plus camera parameters (from SfM), builds a 3D Gaussian representation of the scene, renders it differentiably via a custom CUDA rasterizer, and optimizes the model end‐to‐end with PyTorch.
Top‐Level Architecture Components
a. Data Ingestion & Preprocessing
• scene/ and scene_perspective/: Python modules for loading images, poses, intrinsic/extrinsic parameters, Gaussian initialization.
• arguments/: CLI parsing for datasets, hyperparameters.
• submodules/simple-knn: Nearest-neighbor queries used in Gaussian parameter updates.
b. Core 3D Gaussian Engine
• gaussian_renderer/: Python wrapper classes (network_gui.py, pinhole_renderer.py) that assemble and dispatch rendering jobs to CUDA.
• submodules/odgs-gaussian-rasterization/: C++/CUDA rasterizer for forward & backward splatting operations; built via setup.py and CMake.
c. Neural Loss & Optimization
• lpipsPyTorch/: Perceptual loss modules (LPIPS) for photometric quality.
• utils/: Utility functions for camera transforms, graphics math (via glm), loss definitions, system wrappers.
• train.py: Top‐level PyTorch training loop—builds scene model, invokes forward renders, computes losses (MSE, LPIPS), and updates Gaussian parameters.
d. Evaluation & Rendering
• render.py & render_perspective.py: Standalone scripts to produce novel‐view renders after training.
e. Environment & Dependencies
• environment.yaml: Conda specification (Python, PyTorch, CUDA).
• submodules/: External libraries (GLM, STBI image I/O, simple‐knn).
Relationships & Data Flow
1. User invokes train.py with dataset path →
2. arguments parses flags → scene loaders read images & poses →
3. GaussianModel initialized (gaussian_model.py) →
4. train loop:
• gaussian_renderer calls CUDA rasterizer forward to produce synthesized images;
• loss_utils computes photometric & perceptual losses (LPIPSPyTorch);
• backward pass flows through CUDA ops → gradients back to Gaussian parameters;
5. Periodic evaluation: render_perspective.py uses trained model to render views.
Architectural Patterns and Principles
• Modularity: clear separation between data loaders, model definition, renderer, losses, and training logic.
• Hybrid CPU/GPU pipeline: Python controls high-level flow; heavy lifting (splatting) in C++/CUDA.
• Extensibility: external modules (LPIPS, simple-knn, GLM) plugged in via Python interfaces.
• MVC‐style:
• Model = 3D Gaussian parameters;
• View = Rendering engine;
• Controller = train.py orchestrating optimization.