# lzzcd001/meshdiffusion architecture

> Architecture diagram of the GitHub repository lzzcd001/meshdiffusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lzzcd001/meshdiffusion
- Repository: https://github.com/lzzcd001/meshdiffusion

## Overview

##### Project Type and Purpose
- MeshDiffusion is a research-oriented full-stack ML pipeline for 3D mesh generation.
- It implements a score-based diffusion model that directly parameterizes meshes via Deep Marching Tetrahedra (DMTet).
- The codebase splits into:
- Data preparation (raw mesh → tetrahedral grid → cubic volumes)
- Diffusion model (training, sampling, evaluation)
- DMTet fitting & rendering (via the nvdiffrec submodule)
- Visualization (Blender scripts)

##### File Structure & Architectural Patterns
- Top-level modules:
- configs/                — model and diffusion hyperparameters
- data/                   — scripts to generate tet masks & convert to 3D grids
- lib/diffusion/          — core diffusion code: models, trainer, sampler, losses
- main_diffusion.py       — orchestrates train, uncond_gen, cond_gen modes
- metadata/               — builds JSON meta lists of dataset samples
- nvdiffrec/              — DMTet fitting, rendering, evaluation pipeline
- blender_viz/            — Blender Python scripts for final mesh visualization
- Architectural pattern: a modular pipeline with clear separation of concerns—data → model → reconstruction → visualization.

##### Key Components
a. Data Preparation
- Raw ShapeNet meshes → `nvdiffrec/fit_dmtets.py` → DMTet dictionaries → `data/tets_to_3dgrid.py` → 3D grid volumes.
- Metadata builder (`metadata/save_meta.py`) → JSON index files.
b. Diffusion Model
- Configuration loader (`configs/*.py`) → injects hyperparameters via ml_collections.
- Trainer (`lib/diffusion/trainer.py`) → iterates over grid dataset, computes losses (`lib/diffusion/losses.py`).
- Sampler/Inference (`lib/diffusion/sampling.py`, `lib/diffusion/evaler.py`) → unconditional (`--mode=uncond_gen`) or conditional (`--mode=cond_gen`).
- Entry point: `main_diffusion.py`.
c. DMTet Reconstruction & Rendering
- Conditional workflow: use partial DMTet from fitting to feed diffusion.
- After sampling, back to `nvdiffrec/eval.py` → convert grids to DMTet mesh → `.obj` files.
- Rendering pipeline in `nvdiffrec/lib/render/*`, including CUDA-accelerated BSDFs.
d. Visualization
- Blender script (`blender_viz/blender_script.py`) loads `.obj` and produces scene renders.

##### Architectural Patterns & Principles
- Modular pipeline: each stage is a stand-alone script or Python module.
- Clear separation of configuration (configs/) from code.
- Reuse of nvdiffrec library for geometry and rendering.
- Use of metadata JSON to decouple data indexing from actual I/O.

##### Technologies & Frameworks
- Python 3.8+
- PyTorch (model & training) + CUDA 11.6
- PyTorch3D (3D ops)
- ml_collections (config management)
- Blender Python API (visualization)
- NVlabs nvdiffrec (DMTet fitting & CUDA rendering kernels)

##### Essential Elements to Capture
- All major Python modules (configs/, data/, lib/diffusion/, nvdiffrec/)
- Data stores (grid volumes, metadata JSON, checkpoints, .obj meshes)
- Execution scripts (`main_diffusion.py`, `fit_dmtets.py`, `eval.py`, `blender_script.py`)
- External tool dependencies (PyTorch, Blender, CUDA)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Stage
    subgraph "Data Preparation" 
        direction TB
        RawMeshes["Raw ShapeNet Meshes"]:::data
        FitDMTet["fit_dmtets.py"]:::data
        TetData["Tetrahedral Data"]:::data
        Tet2Grid["tets_to_3dgrid.py"]:::data
        GridVolumes[".npy Grid Volumes"]:::data
        SaveMeta["save_meta.py"]:::data
        MetadataJSON["Metadata JSON Index"]:::data
    end

    %% Diffusion Model Stage
    subgraph "Diffusion Model" 
        direction TB
        Configs["configs/*.py"]:::ml
        MainDiff["main_diffusion.py"]:::ml
        Trainer["trainer.py"]:::ml
        Losses["losses.py"]:::ml
        Sampling["sampling.py"]:::ml
        Evaler["evaler.py"]:::ml
        Checkpoint[".pt Checkpoints"]:::storage
    end

    %% Reconstruction & Rendering Stage
    subgraph "DMTet Reconstruction & Rendering"
        direction TB
        EvalGrid["eval.py"]:::render
        OBJMesh[".obj Mesh Files"]:::storage
        subgraph "Rendering Pipeline"
            direction TB
            RenderCore["render.py"]:::render
            CUDAHelpers["renderutils/ (CUDA BSDFs)"]:::render
            FinalImages["Final Rendered Images"]:::render
        end
    end

    %% Visualization Stage
    subgraph "Visualization (Blender)"
        direction TB
        BlenderScript["blender_script.py"]:::render
        BlenderScene["scene.blend"]:::render
        BlenderOutput["Blender Renders"]:::render
    end

    %% External Dependencies
    PyTorch["PyTorch / CUDA"]:::external
    BlenderExt["Blender API"]:::external

    %% Data Flow Arrows
    RawMeshes -->|"to DMTet fitting"| FitDMTet
    FitDMTet -->|"DMTet dicts"| TetData
    TetData -->|"to 3D grid"| Tet2Grid
    Tet2Grid -->|"grid volumes"| GridVolumes
    GridVolumes -->|"indexed by"| SaveMeta
    SaveMeta -->|"metadata JSON"| MetadataJSON

    MetadataJSON -->|"feeds"| Trainer
    GridVolumes -->|"feeds"| Trainer
    Configs -->|"inject hyperparams"| MainDiff
    MainDiff -->|"calls Trainer"| Trainer
    Trainer -->|"uses Losses"| Losses
    MainDiff -->|"mode uncond/cond"| Sampling
    Sampling -->|"calls Evaler"| Evaler
    Evaler -->|"sample grids"| GridVolumes
    Sampling -->|"uses Checkpoint"| Checkpoint
    Trainer -->|"produces"| Checkpoint

    Checkpoint -->|"for reconstruction"| EvalGrid
    GridVolumes -->|"input grids"| EvalGrid
    EvalGrid -->|"outputs mesh"| OBJMesh
    OBJMesh -->|"to render"| RenderCore
    RenderCore -->|"with CUDA helpers"| CUDAHelpers
    CUDAHelpers -->|"generate"| FinalImages

    OBJMesh -->|"to Blender"| BlenderScript
    BlenderScript -->|"uses scene"| BlenderScene
    BlenderScript -->|"produces"| BlenderOutput

    %% External dependencies arrows
    Trainer --> PyTorch
    Sampling --> PyTorch
    RenderCore --> PyTorch
    CUDAHelpers --> PyTorch
    BlenderScript --> BlenderExt

    %% Click Events
    click FitDMTet "https://github.com/lzzcd001/meshdiffusion/blob/main/nvdiffrec/fit_dmtets.py"
    click Tet2Grid "https://github.com/lzzcd001/meshdiffusion/blob/main/data/tets_to_3dgrid.py"
    click SaveMeta "https://github.com/lzzcd001/meshdiffusion/blob/main/metadata/save_meta.py"
    click Configs "https://github.com/lzzcd001/meshdiffusion/blob/main/configs/default_configs.py"
    click MainDiff "https://github.com/lzzcd001/meshdiffusion/blob/main/main_diffusion.py"
    click Trainer "https://github.com/lzzcd001/meshdiffusion/blob/main/lib/diffusion/trainer.py"
    click Losses "https://github.com/lzzcd001/meshdiffusion/blob/main/lib/diffusion/losses.py"
    click Sampling "https://github.com/lzzcd001/meshdiffusion/blob/main/lib/diffusion/sampling.py"
    click Evaler "https://github.com/lzzcd001/meshdiffusion/blob/main/lib/diffusion/evaler.py"
    click EvalGrid "https://github.com/lzzcd001/meshdiffusion/blob/main/nvdiffrec/eval.py"
    click RenderCore "https://github.com/lzzcd001/meshdiffusion/blob/main/nvdiffrec/lib/render/render.py"
    click CUDAHelpers "https://github.com/lzzcd001/meshdiffusion/tree/main/nvdiffrec/lib/render/renderutils/"
    click BlenderScript "https://github.com/lzzcd001/meshdiffusion/blob/main/blender_viz/blender_script.py"

    %% Styles
    classDef data fill:#D0E8FF,stroke:#005B9A,color:#003F6B;
    classDef ml fill:#DFF5D8,stroke:#2E7D32,color:#1B5E20;
    classDef render fill:#FFE4B3,stroke:#FF8F00,color:#E65100;
    classDef storage fill:#E0E0E0,stroke:#757575,color:#424242;
    classDef external fill:#F5F5F5,stroke:#BDBDBD,color:#424242;
```
