# lidan233/caddreamer architecture

> Architecture diagram of the GitHub repository lidan233/caddreamer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lidan233/caddreamer
- Repository: https://github.com/lidan233/caddreamer

## Overview

##### Project Type and Purpose
- CADDreamer is a research-grade, full-stack machine-learning and CAD reconstruction pipeline. Its goal is to transform a single RGB image into a compact, editable CAD B-rep via three major stages: (1) multi-view normal map diffusion, (2) neural 3D reconstruction (NEUS), and (3) CAD primitive segmentation and fitting (using RANSAC and FreeCAD/OpenCascade).

##### File Structure Overview
- Top-level:
- README.md, requirements.txt, setup.sh – documentation and environment setup
- configs/ – YAML files controlling training and test configurations
- cached_output/ – generated intermediate and final outputs
- test_*.py scripts – CLI entry points for each stage
- automl/, instant_nsr_pl/, mvdiffusion/, neus/, pyransac/, utils/, sam_pt/ – main code modules
- Key components:
- mvdiffusion/ – implements the multi-view diffusion model (PyTorch UNet, transformer blocks)
- instant_nsr_pl/ – PyTorch Lightning wrappers for neural surface reconstruction (NEUS)
- neus/ – pure-Python implementation of NEUS inference, surface fitting, primitive intersection/stitching
- automl/ – utility code for AutoencoderKL (latent diffusion VAE) fine-tuning
- utils/ – shared logging, I/O, visualization, callbacks

##### Architectural Patterns and Design Principles
- Modular, stage-based pipeline orchestrated by Python CLI scripts.
- Separation of concerns:
- Diffusion Stage (mvdiffusion + automl/ckpts)
- Reconstruction Stage (instant_nsr_pl + neus)
- CAD Fitting Stage (neus + pyransac + FreeCAD/OpenCascade)
- Config-driven via YAML files in configs/.
- Hybrid Python/C++ integration: heavy ML in Python; performance-critical RANSAC in C++ (pyransac + pybind11).

##### Main Components and Interactions
- Input: single-view RGB image
→ Feature Extraction & Diffusion (mvdiffusion.pipeline_mvdiffusion_image, AutoencoderKL)
→ Generates multi-view normal semantic maps
→ NEUS Reconstruction (instant_nsr_pl.systems + neus.models)
→ Produces a 3D mesh (point cloud + normals → implicit field → surface)
→ Segmentation & Primitive Detection (neus.fit_surfaces + pyransac)
→ Identifies and fits CAD primitives to mesh
→ Intersection & Stitching (neus.intersectionocc + FreeCAD/OpenCascade)
→ Generates final CAD STEP file

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Store
    subgraph "Data Store" 
        DS1["cached_output/"]:::datastore
        DS2["configs/"]:::datastore
        DS3["ckpts/"]:::datastore
    end

    %% Config Manager
    subgraph "Config Manager" 
        CM1["configs/mvdiffusion-joint-ortho-6views.yaml"]:::config
        CM2["configs/train/testing_4090_stage_1_cad_6views-lvis.yaml"]:::config
        CM3["instant_nsr_pl/launch.py"]:::config
        CM4["neus/launch.py"]:::config
    end

    %% User CLI
    subgraph "User CLI Scripts" 
        CLI1["test_mvdiffusion_seq.py"]:::cli
        CLI2["test_real_images.py"]:::cli
        CLI3["test_syne_images_stage_2_segmentation.py"]:::cli
        CLI4["test_syne_images_stage_3_generate_step.py"]:::cli
    end

    %% Diffusion Module
    subgraph "Diffusion Module" 
        D1["pipeline_mvdiffusion_image.py"]:::ml
        D2["unet_mv2d_blocks.py"]:::ml
        D3["unet_mv2d_condition.py"]:::ml
        D4["transformer_mv2d.py"]:::ml
        D5["normal_utils.py"]:::ml
        D6["AutoencoderKL.py"]:::ml
    end

    %% Reconstruction Module
    subgraph "Reconstruction Module" 
        R1["instant_nsr_pl/systems/neus.py"]:::ml
        R2["instant_nsr_pl/systems/neus_ortho.py"]:::ml
        R3["instant_nsr_pl/systems/neus_pinhole.py"]:::ml
        R4["instant_nsr_pl/models/neus.py"]:::ml
        R5["instant_nsr_pl/models/nerf.py"]:::ml
        R6["instant_nsr_pl/models/network_utils.py"]:::ml
        R7["neus/models/neus.py"]:::ml
        R8["neus/models/nerf.py"]:::ml
    end

    %% Primitive Fitting Module
    subgraph "Primitive Fitting Module" 
        P1["neus/fit_surfaces/fitting_one_surface.py"]:::cad
        P2["neus/fit_surfaces/fitting_utils.py"]:::cad
        P3["neus/fit_surfaces/generate_segmentation.py"]:::cad
        P4["neus/fit_surfaces/intersection.py"]:::cad
        P5["neus/fit_surfaces/primitive_forward.py"]:::cad
        P6["pyransac/primitives/"]:::cad
        P7["pyransac/build.sh"]:::cad
        P8["pyransac/cmake-build-release/liblibransac.a"]:::cad
    end

    %% CAD Builder
    subgraph "CAD Builder" 
        C1["neus/intersectionocc.py"]:::cad
        C2["neus/intersectiontrace.py"]:::cad
    end

    %% External Dependencies
    PythonEnv["Conda Python Env"]:::external
    Torch["PyTorch / Lightning"]:::external
    Bind["pybind11"]:::external
    OCCT["OpenCascade & FreeCAD"]:::external

    %% Connections
    CLI1 -->|"reads configs"| CM1
    CLI2 -->|"reads configs"| CM2
    CLI3 -->|"reads configs"| CM2
    CLI4 -->|"reads configs"| CM2

    CLI1 -->|"input RGB"| DS2
    DS2 -->|"YAML params"| D1
    D1 -->|"latent"| D6
    D1 -->|"multi-view normals"| DS1

    D1 -->|"features"| R1
    D2 & D3 & D4 & D5 --> D1

    R1 -->|"point cloud + normals"| R4
    R4 -->|"implicit field"| DS1
    R7 & R8 --> R1
    R2 & R3 --> R1

    R1 -->|"mesh .obj"| DS1
    R1 -->|"invokes pybind11"| Bind

    DS1 -->|".obj"| P1
    P1 -->|"segmented primitives"| P3
    P1 -->|"fits surfaces"| P2
    P2 -->|"primitives"| DS1
    P4 -->|"intersection data"| C1
    P5 -->|"forward projection"| DS1

    C1 -->|"stitching"| C2
    C2 -->|"export .step"| DS1

    %% External Integration
    PythonEnv --> CLI1
    PythonEnv --> D1
    PythonEnv --> R1
    PythonEnv --> P1
    PythonEnv --> C1

    Torch --> D1
    Torch --> R1

    OCCT --> C1

    %% Click Events
    click CLI1 "https://github.com/lidan233/caddreamer/blob/master/test_mvdiffusion_seq.py"
    click CLI2 "https://github.com/lidan233/caddreamer/blob/master/test_real_images.py"
    click CLI3 "https://github.com/lidan233/caddreamer/blob/master/test_syne_images_stage_2_segmentation.py"
    click CLI4 "https://github.com/lidan233/caddreamer/blob/master/test_syne_images_stage_3_generate_step.py"
    click CM1 "https://github.com/lidan233/caddreamer/blob/master/configs/mvdiffusion-joint-ortho-6views.yaml"
    click CM2 "https://github.com/lidan233/caddreamer/blob/master/configs/train/testing_4090_stage_1_cad_6views-lvis.yaml"
    click CM3 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/launch.py"
    click CM4 "https://github.com/lidan233/caddreamer/blob/master/neus/launch.py"
    click D1 "https://github.com/lidan233/caddreamer/blob/master/mvdiffusion/pipelines/pipeline_mvdiffusion_image.py"
    click D2 "https://github.com/lidan233/caddreamer/blob/master/mvdiffusion/models/unet_mv2d_blocks.py"
    click D3 "https://github.com/lidan233/caddreamer/blob/master/mvdiffusion/models/unet_mv2d_condition.py"
    click D4 "https://github.com/lidan233/caddreamer/blob/master/mvdiffusion/models/transformer_mv2d.py"
    click D5 "https://github.com/lidan233/caddreamer/blob/master/mvdiffusion/data/normal_utils.py"
    click D6 "https://github.com/lidan233/caddreamer/blob/master/automl/AutoencoderKL.py"
    click R1 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/systems/neus.py"
    click R2 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/systems/neus_ortho.py"
    click R3 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/systems/neus_pinhole.py"
    click R4 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/models/neus.py"
    click R5 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/models/nerf.py"
    click R6 "https://github.com/lidan233/caddreamer/blob/master/instant_nsr_pl/models/network_utils.py"
    click R7 "https://github.com/lidan233/caddreamer/blob/master/neus/models/neus.py"
    click R8 "https://github.com/lidan233/caddreamer/blob/master/neus/models/nerf.py"
    click P1 "https://github.com/lidan233/caddreamer/blob/master/neus/fit_surfaces/fitting_one_surface.py"
    click P2 "https://github.com/lidan233/caddreamer/blob/master/neus/fit_surfaces/fitting_utils.py"
    click P3 "https://github.com/lidan233/caddreamer/blob/master/neus/fit_surfaces/generate_segmentation.py"
    click P4 "https://github.com/lidan233/caddreamer/blob/master/neus/fit_surfaces/intersection.py"
    click P5 "https://github.com/lidan233/caddreamer/blob/master/neus/fit_surfaces/primitive_forward.py"
    click P6 "https://github.com/lidan233/caddreamer/tree/master/pyransac/primitives/"
    click P7 "https://github.com/lidan233/caddreamer/blob/master/pyransac/build.sh"
    click P8 "https://github.com/lidan233/caddreamer/blob/master/pyransac/cmake-build-release/liblibransac.a"
    click C1 "https://github.com/lidan233/caddreamer/blob/master/neus/intersectionocc.py"
    click C2 "https://github.com/lidan233/caddreamer/blob/master/neus/intersectiontrace.py"

    %% Styles
    classDef ml fill:#D0E6FF,stroke:#0366d6,color:#0366d6
    classDef cad fill:#E6FFEA,stroke:#28A745,color:#28A745
    classDef config fill:#F0F0F0,stroke:#999,color:#333
    classDef cli fill:#FFF4E6,stroke:#D97706,color:#D97706
    classDef datastore fill:#F4F4F5,stroke:#6B7280,color:#6B7280
    classDef external fill:#F3E8FF,stroke:#7C3AED,color:#7C3AED
```
