# jayliu0313/shape-guided architecture

> Architecture diagram of the GitHub repository jayliu0313/shape-guided, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jayliu0313/shape-guided
- Repository: https://github.com/jayliu0313/shape-guided

## Overview

##### Project Type and Purpose
- This is a research-grade machine-learning pipeline for 3D anomaly detection and segmentation.
- It ingests RGB images + 3D point clouds (MvTec3D-AD), preprocesses data, trains a 3D “expert” network, builds a dual-memory bank, and runs inference to produce anomaly heatmaps.

##### Main Architectural Components
a. Data Layer
- External Dataset (MvTec3D-AD)
- Local storage (raw point clouds, RGB images)
b. Preprocessing & Patch Generation
- tools/preprocessing.py → background removal
- cut_patches.py → divides point clouds into local patches
c. 3D Expert Model Training
- train_3Dmodel.py
- core/model.py & core/pointnet2_utils.py → PointNet2-based feature extractor
- checkpoint/storage (best_ckpt/*.pth)
d. Dual-Memory Construction (Inference Prep)
- main.py reads trained checkpoint
- core/shape_guide_core.py builds two memories:
– RGB-guided memory
– SDF (signed distance function) memory
e. Inference & Visualization
- core/features.py, core/rgb_sdf_feature.py compute complementary heatmaps
- tools/visualize_loss.py, utils/visualize_util.py for plotting results
f. Utilities & Configuration
- utils/* modules for helper functions, data I/O, metrics (AUROC, AUPRO)
- Configuration.pickle / .txt files in checkpoint for hyperparams

##### Relationships & Data Flow
1) Dataset → 2) Preprocessing → 3) Patch Generation → 4) Model Training → 5) Memory Construction → 6) Inference → 7) Output Heatmaps
- That preprocessing and patch-cutting must run in sequence.

##### Architectural Patterns & Design Principles
- Modular pipeline: separate scripts for each stage
- Core library modules (core/, utils/) implement reusable logic
- Checkpointing for reproducibility & skipping expensive steps
- Dual-memory design: two parallel memory stores fused at inference

##### Technologies & Frameworks
- Python 3.6+
- PyTorch 1.7+, CUDA
- PointNet2 architecture (pointnet2_utils)
- NumPy, Open3D or custom point-cloud tools (implicitly in preprocessing)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Layer
    subgraph "External Dependencies" 
        direction TB
        Dataset["MvTec3D-AD Dataset"]:::external
    end

    subgraph "Data Layer"
        direction TB
        MVTEC_UTIL["mvtec3d_util.py"]:::util
        Storage["Local Storage\n(raw .ply, RGB images)"]:::data
    end

    Dataset -->|“download”| MVTEC_UTIL
    MVTEC_UTIL -->|“save raw files”| Storage

    %% Preprocessing & Patch Generation
    subgraph "Preprocessing & Patch Generation"
        direction TB
        Preproc["preprocessing.py"]:::data
        PatchCut["cut_patches.py"]:::data
    end

    Storage -->|“raw .ply files”| Preproc
    Preproc -->|“filtered cloud”| PatchCut
    click Preproc "https://github.com/jayliu0313/shape-guided/blob/main/tools/preprocessing.py"
    click PatchCut "https://github.com/jayliu0313/shape-guided/blob/main/cut_patches.py"

    %% 3D Expert Model Training
    subgraph "3D Expert Model Training"
        direction TB
        TrainScript["train_3Dmodel.py"]:::model
        CoreModel["core/model.py"]:::model
        PN2Utils["pointnet2_utils.py"]:::util
        CKPT["Checkpoint\n(best_ckpt/*.pth)"]:::data
    end

    PatchCut -->|“patch grid”| TrainScript
    TrainScript -->|“uses model”| CoreModel
    TrainScript -->|“uses utils”| PN2Utils
    TrainScript -->|“save checkpoint”| CKPT
    click TrainScript "https://github.com/jayliu0313/shape-guided/blob/main/train_3Dmodel.py"
    click CoreModel "https://github.com/jayliu0313/shape-guided/blob/main/core/model.py"
    click PN2Utils "https://github.com/jayliu0313/shape-guided/blob/main/utils/pointnet2_utils.py"

    %% Dual-Memory Construction
    subgraph "Dual-Memory Construction"
        direction TB
        MainEntry["main.py"]:::model
        MemoryBuilder["shape_guide_core.py"]:::memory
        RGBMem["RGB-guided Memory"]:::memory
        SDFMem["SDF Memory"]:::memory
    end

    CKPT -->|“ckpt_*.pth”| MainEntry
    MainEntry -->|“load checkpoint”| MemoryBuilder
    MemoryBuilder --> RGBMem
    MemoryBuilder --> SDFMem
    click MainEntry "https://github.com/jayliu0313/shape-guided/blob/main/main.py"
    click MemoryBuilder "https://github.com/jayliu0313/shape-guided/blob/main/core/shape_guide_core.py"

    %% Inference & Visualization
    subgraph "Inference & Visualization"
        direction TB
        Features["features.py"]:::model
        Fusion["rgb_sdf_feature.py"]:::model
        VizLoss["visualize_loss.py"]:::util
        VizUtil["visualize_util.py"]:::util
        Output["Anomaly Heatmap\n(heatmap.png)"]:::data
    end

    RGBMem -->|“RGB features”| Features
    SDFMem -->|“SDF features”| Features
    Features --> Fusion
    Fusion -->|“fused map”| VizLoss
    VizLoss -->|“plots”| VizUtil
    VizUtil -->|“save heatmap”| Output
    click Features "https://github.com/jayliu0313/shape-guided/blob/main/core/features.py"
    click Fusion "https://github.com/jayliu0313/shape-guided/blob/main/core/rgb_sdf_feature.py"
    click VizLoss "https://github.com/jayliu0313/shape-guided/blob/main/tools/visualize_loss.py"
    click VizUtil "https://github.com/jayliu0313/shape-guided/blob/main/utils/visualize_util.py"

    %% Utilities & Configuration
    subgraph "Utilities & Configuration"
        direction TB
        AUROCP["au_pro_util.py"]:::util
        UtilsAll["utils.py"]:::util
        PN1Utils["pointnet_util.py"]:::util
        ConfigPKL["Configuration.pickle"]:::data
        ConfigTXT["Configuration.txt"]:::data
    end

    UtilsAll -->|“metrics”| AUROCP
    UtilsAll -->|“I/O, logging”| PN1Utils
    CKPT --> ConfigPKL
    CKPT --> ConfigTXT
    click AUROCP "https://github.com/jayliu0313/shape-guided/blob/main/utils/au_pro_util.py"
    click UtilsAll "https://github.com/jayliu0313/shape-guided/blob/main/utils/utils.py"
    click PN1Utils "https://github.com/jayliu0313/shape-guided/blob/main/utils/pointnet_util.py"
    click ConfigPKL "https://github.com/jayliu0313/shape-guided/blob/main/checkpoint/best_ckpt/Configuration.pickle"
    click ConfigTXT "https://github.com/jayliu0313/shape-guided/blob/main/checkpoint/best_ckpt/Configuration.txt"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef model fill:#90EE90,stroke:#333,stroke-width:1px
    classDef memory fill:#FFA500,stroke:#333,stroke-width:1px
    classDef util fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef external fill:#ffffff,stroke-dasharray: 5 5,stroke:#333,stroke-width:1px
```
