How ruyi-zha/r2_gaussian works
Based on our review of the file tree and the README, this project is a specialized tomographic reconstruction system that uses a variation of 3D Gaussian Splatting (hence “r2_gaussian”) for CT reconstruction. It integrates components for data generation, point cloud initialization, training a radiative Gaussian model, and evaluation – with significant parts running on GPU using CUDA-based submodules for rasterization and voxelization.
• Data Generation & Preprocessing:
• “data_generator” directory as a module responsible for preparing synthetic, real, or NAF-format datasets.
• That this component converts raw CT data (or volumes) into a format with projection images, metadata JSON, or pickle files. Denote that it also supports different scanner configurations (cone beam/parallel beam) via YAML configuration files.
• Point Cloud Initialization:
• “initialize_pcd.py” and corresponding scripts in the “data_generator” and root level. This module generates initial point clouds (stored as *.npy) for the Gaussian splatting model using either the FDK method or random sampling.
• Dependency on external tools (e.g., TIGRE) used during initialization.
• The Training Module (Core Algorithm):
• Break out the “r2_gaussian” directory with its subcomponents:
▪ “r2_gaussian/gaussian”: Contains the main implementations of the Gaussian model (including model initialization, updating, and rendering queries).
▪ “r2_gaussian/dataset”: Implements dataset loading and camera handling – essential for matching the projection data with the reconstruction process.
▪ “r2_gaussian/arguments” and “r2_gaussian/utils”: Hold supporting functions for configuration, experiment logging, and various mathematics/graphics utilities.
• GPU-Accelerated Rendering:
• “r2_gaussian/submodules/xray-gaussian-rasterization-voxelization” folder that provides CUDA-based routines for rasterizing and voxelizing the Gaussians for fast 3D rendering and evaluation. This is critical for the speed of both training and inference.
• Evaluation & Visualization:
• “test.py” and “scripts/visualize_scene.py” components to denote post-training evaluation and demo visualization (rendering 2D/3D outputs).
• Integrate details from the README about logging (tensorboard events) and evaluation scripts that allow quick inspection of rendering quality.
Relationships and Data Flow
• How raw input data flows from the Data Generation component (data_generator) into the Point Cloud Initialization step.
• How the output (initial point clouds / metadata) is then consumed by the Training Module.
• How the training process utilizes utility functions and submodules (rasterization/voxelization) during the iterative optimization.
• How the trained model is evaluated by the Testing/Evaluation module.
Architectural Patterns and Design Principles
• GPU/CUDA Acceleration: Mark the specialized submodules that enable parallel processing in rendering or rasterization.
Relevant Technologies and Frameworks
• Conda Environment: Note that installation is managed via Conda (environment.yml).
• CUDA and CMake: Indicate the use of CUDA routines (e.g., in submodules) with build support via CMake.
• Python-based CLI: Express that the overall orchestration is driven by Python scripts (train.py, test.py, etc.) with command-line arguments for configuration.
• External Libraries: Reference TIGRE and the dependencies on modules like “Naive-neRF” and “Gaussian Splatting” forks.