# bmild/nerf architecture

> Architecture diagram of the GitHub repository bmild/nerf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bmild/nerf
- Repository: https://github.com/bmild/nerf

## Overview

##### Project Type and Purpose
- This is an open-source research codebase for “NeRF: Neural Radiance Fields,” a framework to train a fully connected neural network per scene that learns a volumetric 5D→4D mapping and uses differentiable volume rendering to synthesize novel views.
- It is not a web or microservice architecture, but a single-process, local training+inference tool built on TensorFlow 1.x, supplemented by data download scripts and Jupyter notebooks for exploration and mesh extraction.

##### File Structure Analysis
Top-level files and directories:
- run_nerf.py, run_nerf_helpers.py – core training and rendering driver
- load_*.py – data-loading modules for different scene formats (blender, deepvoxels, llff)
- *.txt in root and paper_configs/ – configuration files for experiment hyperparameters and dataset paths
- download_example_data.sh / download_example_weights.sh – shell scripts to fetch datasets and pretrained checkpoints
- environment.yml – conda environment spec listing TensorFlow, numpy, matplotlib, etc.
- notebooks (tiny_nerf.ipynb, render_demo.ipynb, extract_mesh.ipynb) – interactive demos for toy model, rendering, mesh extraction
Patterns and components:
- Config-driven execution: a single script (run_nerf.py) parses a config file and dispatches data loading, network construction, training loop, and rendering
- Modular data loaders: each loader returns images, poses, intrinsics, near/far bounds
- Core network: multilayer perceptron with positional encoding (in run_nerf_helpers.py)
- Training engine: batching, ray sampling, loss computation, optimizer, checkpoints, TensorBoard logging
- Rendering pipeline: volume rendering integration along camera rays to produce RGB images and depth maps
- Post-processing: optional mesh extraction via marching cubes in extract_mesh.ipynb

##### Key Insights from README
- The pipeline figure in imgs/pipeline.jpg shows data → network → rendered views → loss → backprop
- Dependencies include TensorFlow 1.15, numpy, matplotlib, imageio, configargparse; optional external tools: COLMAP/LLFF code for pose generation, PyMCubes/trimesh/pyrender for mesh extraction
- Usage revolves around a single command: “python run_nerf.py --config <file>”

##### Components
a. Main Components
1. Configuration Parser
2. Dataset Loader Module
3. Positional Encoding + MLP Network (NeRF)
4. Ray Generator & Sampler
5. Volume Renderer
6. Loss & Optimizer (Training Loop)
7. Checkpoint & TensorBoard Logger
8. Rendering Interface (inference mode)
9. Jupyter Notebooks (tiny NeRF demo, render_demo, extract_mesh)
10. External Data Sources & Scripts (download_example_data.sh, download_example_weights.sh)
b. Relationships and Data Flows
- Config → Dataset Loader → Data Batches (rays+poses)
- Rays → Network (via positional encoding) → Raw Output (RGB+σ)
- Raw Output + Rays → Volume Renderer → Rendered Pixels
- Rendered Pixels vs. Ground Truth → Loss Function → Backprop through Network
- Checkpoint & TensorBoard ← Metrics/Model Weights
- Inference Path: Checkpoint + run_nerf.py (with render flags) or render_demo.ipynb → New Views
- extract_mesh.ipynb loads a saved NeRF model → Queries grid of 3D points → Marching Cubes → Mesh
c. Architectural Patterns & Principles
- Modular separation: data loaders, model definition, training loop, rendering, logging
- Config-first design to allow easy swapping of datasets and hyperparameters
- Batch processing and pipeline style—single process but staged operations
- Extensibility: add new data loader or modify network architecture in helper files
d. Key Technologies
- TensorFlow 1.15 for model definition and training
- NumPy, Matplotlib, ImageIO for data processing and visualization
- ConfigArgParse for command-line config management
- Shell scripts for orchestrating data/weight downloads
- Optional: PyMCubes, Trimesh, PyRender for mesh extraction

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources and Config
    subgraph "Data & Config" 
        direction TB
        Config["Configuration Parser\n(run_nerf.py)"]:::io
        ConfigFiles["Config Files\n(config_*.txt, paper_configs/)"]:::io
        EnvSpec["Environment Spec\n(environment.yml)"]:::external
        DataScripts["Download Scripts\n(download_example_data.sh,\ndownload_example_weights.sh)"]:::external
        PipelineAsset["Pipeline Diagram\n(imgs/pipeline.jpg)"]:::external
    end

    %% Dataset Loaders
    subgraph "Dataset Loader Modules" 
        direction TB
        LoaderBlender["load_blender.py"]:::io
        LoaderDeepVoxels["load_deepvoxels.py"]:::io
        LoaderLLFF["load_llff.py"]:::io
    end

    %% Core NeRF Engine
    subgraph "Core NeRF Pipeline" 
        direction TB
        RaySampler["Ray Generator & Sampler\n(run_nerf_helpers.py)"]:::core
        PosEncMLP["Positional Encoding + MLP\n(run_nerf_helpers.py)"]:::core
        VolumeRenderer["Volume Renderer\n(run_nerf_helpers.py)"]:::core
        LossOptim["Loss & Optimizer\n(run_nerf.py,\nrun_nerf_helpers.py)"]:::core
        CheckpointLogger["Checkpoint & TensorBoard Logger\n(run_nerf.py)"]:::core
    end

    %% Inference & Notebooks
    subgraph "Inference & Notebooks"
        direction TB
        RenderingInterface["Rendering Interface\n(run_nerf.py,\nrender_demo.ipynb)"]:::core
        NotebookTiny["tiny_nerf.ipynb"]:::notebook
        NotebookDemo["render_demo.ipynb"]:::notebook
        NotebookMesh["extract_mesh.ipynb"]:::notebook
    end

    %% Connections
    Config -->|provides settings| LoaderBlender
    Config -->|provides settings| LoaderDeepVoxels
    Config -->|provides settings| LoaderLLFF
    DataScripts -->|downloads data| LoaderBlender
    DataScripts -->|downloads data| LoaderDeepVoxels
    DataScripts -->|downloads data| LoaderLLFF

    LoaderBlender -->|outputs rays+poses| RaySampler
    LoaderDeepVoxels -->|outputs rays+poses| RaySampler
    LoaderLLFF -->|outputs rays+poses| RaySampler

    RaySampler -->|rays| PosEncMLP
    PosEncMLP -->|raw RGB+σ| VolumeRenderer
    VolumeRenderer -->|pixels| LossOptim
    LossOptim -->|gradients| PosEncMLP
    LossOptim -->|metrics & weights| CheckpointLogger

    CheckpointLogger -->|saved model| RenderingInterface
    Config -->|render flags| RenderingInterface
    RenderingInterface -->|new views| VolumeRenderer

    %% Notebooks sidecar
    NotebookTiny -.->|demo workflow| PosEncMLP
    NotebookDemo -.->|inference demo| RenderingInterface
    NotebookMesh -.->|mesh extraction| CheckpointLogger

    %% Click Events
    click Config "https://github.com/bmild/nerf/blob/master/run_nerf.py"
    click LoaderBlender "https://github.com/bmild/nerf/blob/master/load_blender.py"
    click LoaderDeepVoxels "https://github.com/bmild/nerf/blob/master/load_deepvoxels.py"
    click LoaderLLFF "https://github.com/bmild/nerf/blob/master/load_llff.py"
    click RaySampler "https://github.com/bmild/nerf/blob/master/run_nerf_helpers.py"
    click PosEncMLP "https://github.com/bmild/nerf/blob/master/run_nerf_helpers.py"
    click VolumeRenderer "https://github.com/bmild/nerf/blob/master/run_nerf_helpers.py"
    click LossOptim "https://github.com/bmild/nerf/blob/master/run_nerf.py"
    click CheckpointLogger "https://github.com/bmild/nerf/blob/master/run_nerf.py"
    click RenderingInterface "https://github.com/bmild/nerf/blob/master/render_demo.ipynb"
    click NotebookTiny "https://github.com/bmild/nerf/blob/master/tiny_nerf.ipynb"
    click NotebookDemo "https://github.com/bmild/nerf/blob/master/render_demo.ipynb"
    click NotebookMesh "https://github.com/bmild/nerf/blob/master/extract_mesh.ipynb"
    click DataScripts "https://github.com/bmild/nerf/blob/master/download_example_data.sh"
    click DataScripts "https://github.com/bmild/nerf/blob/master/download_example_weights.sh"
    click ConfigFiles "https://github.com/bmild/nerf/tree/master/paper_configs/"
    click EnvSpec "https://github.com/bmild/nerf/blob/master/environment.yml"
    click PipelineAsset "https://github.com/bmild/nerf/blob/master/imgs/pipeline.jpg"

    %% Styles
    classDef core fill:#BBDEFB,stroke:#0D47A1,stroke-width:1px
    classDef io fill:#C8E6C9,stroke:#1B5E20,stroke-width:1px
    classDef external fill:#F0F4C3,stroke:#827717,stroke-width:1px
    classDef notebook fill:#FFE0B2,stroke:#E65100,stroke-width:1px
```
