How yihangchen-ee/hac works
Project Type and Purpose
• This project is a research-oriented Pytorch implementation for compressing 3D Gaussian Splatting representations. Its goal is to achieve significant compression (up to 75X) without rendering fidelity loss by using a hash‑grid assisted context model.
• The README emphasizes components such as training, encoding, decoding, and testing – and the experimental results are organized by different datasets/training scripts (e.g., Tanks&Temples, MipNeRF360, BungeeNeRF, etc.).
File Structure for Architectural Insights
• Top-level directories and files:
• "arguments": likely a module for handling command-line parameters.
• "assets": contains images (teasers and performance figures) for documentation and visualization.
• "gaussian_renderer": includes Python modules for the rendering and GUI aspects, showing the project’s capacity to display or interact with the 3D representations.
• "lpipsPyTorch": holds the modules for LPIPS perceptual loss computation, suggesting its role in maintaining visual fidelity.
• "scene": contains data loaders, camera handling, Colmap integration, and the Gaussian model – these underpin data preprocessing and scene construction.
• "utils": a comprehensive collection of helper functions for camera processing, encodings (both CPU and CUDA), general graphics, image processing, and system utilities.
• "submodules": contains zipped external libraries for arithmetic coding, diff‑gaussian rasterization, grid encoding, and simple k‑nearest neighbors which are essential for the compression pipeline.
• “results”: stores evaluation CSVs for different datasets and experimental runs.
• "run_shell_*.py" scripts: separate entry points for training/testing with different datasets.
• "train.py": the central script orchestrating the training process.
• The well‑segmented structure (modules for data preparation, core computations, rendering, evaluations, and utilities) is a clear indicator of a modular design where each component is responsible for discrete functions in the pipeline.
Components
• Main Components:
• Data Ingestion/Preprocessing: Represented by the “scene” module (cameras, colmap_loader, dataset_readers) and the external “data” folder (created by users according to prescribed structure).
• Training & Compression Pipeline: Driven by the training scripts (train.py & run_shell_xxx.py) that handle the stages of training, encoding, decoding, and evaluation.
• Core Compression and Rendering Modules:
• HAC Compression Core – integrates the innovative hash-grid aided context model, adaptive quantization, and context-based entropy coding.
• Gaussian Renderer (in gaussian_renderer) – visualizes the results.
• Perceptual Loss Module (lpipsPyTorch) – ensures fidelity through LPIPS metrics.
• Utility modules: “utils” for system support (graphics, camera, and encoding utilities) and “arguments” for command-line interfaces.
• External Dependencies: “submodules” containing zipped libraries (for diff‑gaussian rasterization, grid encoding, arithmetic operations, etc.) that perform specialized tasks.
• Result Handling: “results” directory for storing CSV logs and evaluation metrics.
Relationships and Interactions
• Data Flow:
• Data originates from custom or public datasets stored under “data/” (not in the repository but referenced in the README).
• The “scene” module ingests and processes the data (images, sparse point clouds, camera parameters), providing input for training.
• Training Workflow:
• The training scripts (run_shell_*.py) call “train.py”, which in turn orchestrates the flow: it retrieves scene data, invokes the Gaussian model for representing 3D structures, and passes data to the compression modules.
• The compression modules leverage the external “submodules” for specialized processing (such as rasterization and grid encoding) and use “lpipsPyTorch” to calculate perceptual losses if required.
• Rendering & Evaluation:
• The “gaussian_renderer” module generates visual outputs.
• The results from training (logs, encoded bitstreams, rendered test images) are organized and saved in the output directories and “results/” folder.
• Utility Layer:
• “utils” functions support data transformation, visualization, system calls, and GPU integrations.
• Configuration & Execution:
• “arguments” encapsulates command-line input handling while environment requirements are defined in “environment.yml” and installation instructions in the README.