How geoai4cities/lid-lab-nerf works
Main Components
• Data Ingestion & Preprocessing
– Raw KITTI-360 LiDAR point clouds and semantics (.ply)
– Bash script: preprocess_data.sh
– Python modules under data/preprocess/
• Dataset & Data Loader
– data/base_dataset.py, kitti360_dataset.py, kitti360_dataset_with_image.py
– Generates PyTorch Dataset objects & JSON transforms
• Configuration
– configs/kitti360_*.txt (hyperparameters, model settings)
– workspace/args.txt for record keeping
• Model Definition & Core Library
– model/*.py: NeRF fields (hash_field, planes_field, flow_field), renderer, darknet, unet, vgg, labelnerf
– model/runner.py: orchestrates training and evaluation loops
– model/simulator.py: LiDAR simulation after training
• External Dependencies & Extensions
– PyTorch & CUDA
– tiny-cuda-nn (binds into the renderer)
– utils/chamfer3D (custom CUDA kernels for Chamfer distance)
– Other utils: utils/metrics.py, utils/misc.py, utils/convert.py
• Training & Logging
– main_labelnerf.py: entry point for training
– run_kitti_labelnerf.sh: bash wrapper setting CUDA_VISIBLE_DEVICES, config, workspace
– workspace/run/lidar4d/events*.tfevents: TensorBoard logs
• Simulation & Visualization
– main_labelnerf_sim.py: entry point for dynamic scene replay & novel scan generation
– run_kitti_labelnerf_sim.sh: wrapper for simulation parameters
– Outputs: point clouds, intensity, depth, semantic labels in workspace
Data Flow & Relationships
• Preprocessing → Dataset
• Dataset → Model Runner
• Config Files → Runner & Model
• Model Runner → Workspace (Checkpoint + Logs)
• Saved Checkpoint + Simulator → Simulation Outputs
Architectural Patterns & Design Principles
• Modular Separation:
– Data layer, Model layer, Training orchestration, Simulation layer, Utilities
• Plugin-style fields (hash_field, planes_field, flow_field) that can be swapped via config
• Clear I/O boundary: raw data → preprocessing → standard JSON format → Dataset → NeRF training → checkpoint → simulation
• GPU-accelerated kernels (tiny-cuda-nn, chamfer3D) abstracted in utils
Technologies & Frameworks
• Core: Python, PyTorch (with torch.nn, CUDA backend)
• GPU kernels: tiny-cuda-nn, chamfer3D (written in CUDA/C++)
• Scripting: Bash for orchestrating preprocess, train, simulate
• Logging: TensorBoard (events.out.tfevents…)
• Optional Notebook: main.ipynb for interactive exploration