How linjohnss/frugalnerf works
Project Type & Purpose
• This is a machine-learning research codebase (PyTorch) for few-shot novel-view synthesis using NeRF.
• It provides data-loading pipelines, model definitions, training/rendering routines, utility scripts, and a custom CUDA loss extension.
File Structure
Top-level directories/files and their roles:
• configs/
– Text files encoding hyperparameters for different datasets (DTU, LLFF, RealEstate10K, custom).
• dataLoader/
– Dataset-specific loading and preprocessing modules (colmap2nerf, dtu, llff, realestate10k, your_own_data) and ray tracing utilities.
• extra/
– Helper scripts for batch experiments (auto_run_paramsets), metric computation, Colmap preprocessing.
• models/
– Radiance-field architectures: tensoRF.py, sh.py, base classes.
• torch_efficient_distloss/
– Custom PyTorch CUDA extension implementing an efficient distance-based loss.
• train.py & renderer.py
– Entry point for training and rendering loops.
• opt.py & utils.py
– Configuration parsing and shared utilities.
• scripts/
– Shell scripts for launching full experiments.
• assets/, README, LICENSE, .gitignore – ancillary.
Pull Architecture Insights from README
• Core workflow:
– train.py reads a config, loads images (few views), constructs a NeRF model, runs fast training, optionally renders.
• Dependencies: PyTorch, CUDA, configargparse, image I/O libs, tensorboard, custom torch-efficient-distloss.
• Supports multiple datasets via dataLoader modules.
• Uses shell scripts to automate batch runs.
Main Components & Their Interactions
a. CLI & Configuration Layer
– Components: train.py, opt.py, config files (configs/*.txt), shell scripts (scripts/*.sh).
– Responsibility: parse arguments, select hyperparameters, invoke training or rendering.
b. Data Ingestion & Preprocessing
– Components: dataLoader/* and colmap2nerf.py.
c. Model Definition
– Components: models/tensorBase.py, models/tensoRF.py, models/sh.py.
– Encapsulates scene representation (tensor decomposition, spherical harmonics).
d. Renderer & Training Loop
– Components: renderer.py, part of train.py.
e. Loss Computation
– Components: torch_efficient_distloss/, utils.py.
– Computes photometric loss, efficient distance loss via custom CUDA kernels.
f. Optimization & Logging
– Components: inside train.py and utils.py.
– Runs backprop, updates model, logs metrics to TensorBoard, saves checkpoints.
g. Evaluation & Rendering Only Mode
– Triggered via flags (--render_only, --render_test/train/path), reuses renderer to generate images.
h. Experiment Automation
– Components: extra/, scripts/.
– Batch-runs multiple configs, computes evaluation metrics, aggregates results.
Architectural Patterns & Principles
• Modular layering: clear separation of CLI, data layer, model layer, training/rendering, extensions.
• Plugin design: any new dataset or model variant can be added to dataLoader/ or models/.
• Custom extension for performance: CUDA kernels in torch_efficient_distloss.
• Config-driven reproducibility: all hyperparams in text files, shell scripts orchestrate runs.