How rongliu-leo/atomgs works
Based on our analysis of the file tree and README, AtomGS is a high‐performance research codebase for training and rendering radiance fields via “atomized Gaussian splatting.” In effect, it is a specialized full‐pipeline system with clear sections for preparing input data, training a deep model (including unique proliferation and splitting operations on Gaussians), and then rendering or exporting results (including mesh extraction). The project also integrates GPU–accelerated submodules (in CUDA) and perceptual loss modules like LPIPS; it builds on several other repositories and includes a custom viewer for interactive inspection.
Data Input & Preprocessing
• Component: “Data Converter” (convert.py)
• Responsibilities: Reads raw datasets from COLMAP or NeRF Synthetic sources; converts image data, camera parameters, and scene info into a standardized internal format.
• Dependencies: “scene” (with cameras.py, colmap_loader.py, dataset_readers.py, and gaussian_model.py) and some utilities (from utils).
Training Pipeline
• Component: “Training Module” (train.py)
• Responsibilities: Uses the preprocessed dataset to iteratively train an AtomGS model. It incorporates the “Atom Proliferation” routine (threshold‐driven cloning, pruning, and splitting of Gaussian atoms), a warm-up phase (controlled by parameters like warm_up_until), and loss scaling (with lambda_normal, lambda_ms_ssim and other loss terms).
• Dependencies:
– Normalization and loss-related utilities (in utils/loss_utils.py)
– LPIPS module for perceptual loss (lpipsPyTorch folder)
– CUDA–accelerated routines for differentiable Gaussian rasterization (submodules/diff-gaussian-rasterization)
– Possibly simple-knn for spatial queries in high–dimensional space from submodules/simple-knn
Rendering & Visualization
• Components: “View Module” (view.py), “Render Module” (render.py), and “Gaussian Renderer” module (gaussian_renderer/network_gui.py)
• Responsibilities:
– view.py is responsible for interactive visualization of the trained model.
– render.py produces a rendered output (e.g., RGB maps) from the model.
– The included GUI (from gaussian_renderer) may be used for live inspection.
• Data Flow: These modules retrieve the trained model’s checkpoints and the original dataset to reconstruct the radiance field and present results.
Mesh Exportation
• Component: “Export Module” (export.py)
• Responsibilities: Uses the trained model to compute and extract a Poisson mesh. It fuses RGB, depth, and normal maps for mesh generation and applies thresholds (e.g., depth_threshold) supported by export parameters.
• Interaction: Reads the same standardized dataset and the model produced during training.
Auxiliary & Utility Modules
• Components: “Utils” (e.g., camera_utils.py, graphics_utils.py, image_utils.py, etc.)
• Responsibilities: Provide general and system–level functionality (data I/O, camera calibration, graphics helper routines, shell utilities) which serve multiple components in the pipeline.
• Functions provided here support both training and rendering processes.
External Dependencies and Submodules
• diff-gaussian-rasterization (located in submodules): Implements the CUDA–based routines critical for fast rasterizing of the Gaussian atoms. This is a core GPU–accelerated submodule called during training and rendering.
• lpipsPyTorch: Implements the perceptual loss metric required during training.
• simple-knn: Likely an auxiliary library for fast nearest–neighbor (KNN) search assisting in operations such as Gaussian clustering or spatial queries.
– Another for rendering/visualization and export components.
– Optionally, group external submodules together.
• The pipeline architecture: raw data → conversion → training → model output.
• The integration of CUDA–accelerated Gaussian rasterization and KNN operations as performance-critical submodules.
• The interaction between the deep learning training (with specialized loss functions and atom management routines) and the visualization modules.