# wmtlab/graingrasp architecture

> Architecture diagram of the GitHub repository wmtlab/graingrasp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wmtlab/graingrasp
- Repository: https://github.com/wmtlab/graingrasp

## Overview

##### Project type and purpose
- GrainGrasp is a research codebase for “Dexterous Grasp Generation with Fine-grained Contact Guidance.”
- It provides both training and inference pipelines for generating hand grasps on 3D objects using point-cloud neural networks (PointCVAE, SupNet) and a differentiable hand model (MANO).
- Key subcomponents: data processing, model training, inference/optimization, and visualization.

##### File structure
Top-level directories and files reveal the major subsystems:
- “PointCVAE/” and “SupNet/”: two neural architectures for grasp synthesis.
- “DCoG/”: DexCoGModel class and associated files for the complete inference pipeline.
- “mano/”: MANO hand model code and parameters.
- “dataprocess.py” + “utils/”: dataset loading, point-cloud sampling, loss functions, and visualization.
- “run_complete.py” & “run_only_opt.py”: entry scripts for full pipeline vs. optimization-only pipeline.
- “config.json” & “config.py”: global configuration for paths, hyperparameters, and dependencies.
- “sample/”: example input point clouds and meshes.
- “.imgs/”: sample output images.

##### Pull extra insights from the README
- Training requires the OBMan and ShapeNetCore.v2 datasets and produces model checkpoints under PointCVAE/model.pth and SupNet/model.pth.
- Inference uses pre-trained models and the MANO RIGHT hand mesh data.
- Dependencies: PyTorch, PyTorch3D, Open3D, Trimesh, NumPy, MANO.

##### Main components & interactions
A. Data Layer
- External datasets: OBMan (hand-object interactions) and ShapeNetCore.v2 (object meshes)
- Sample inputs: point clouds (.npy), mesh (.obj)
B. Data Processing
- dataprocess.py + utils/load_obman.py: point-cloud sampling, annotation loading
- utils/tools.py, utils/loss.py, utils/vis.py
C. Model Training
- PointCVAE
– network modules under PointCVAE/network (pointnet_encoder, cave, point_cvae_net)
– train.py, obman_dataset.py → produces PointCVAE/model.pth
- SupNet
– network modules under SupNet/network (pointnet_encoder, supnet)
– train.py, obman_dataset.py → produces SupNet/model.pth
D. MANO Hand Model
- mano/model.py, lbs.py, joints_info.py, utils.py → loads/manipulates hand mesh for forward kinematics & skinning
E. Inference & Optimization
- DCoGModel (DCoG/DCoGModel.py) orchestrates:
– loading object point clouds
– initializing grasps from SupNet and/or PointCVAE
– optimizing contact via “only_opt” or “complete” pipelines
- run_complete.py: full pipeline (initialization + optimization)
- run_only_opt.py: optimization only, given an initial pose
F. Visualization & Output
- utils/vis.py, Open3D & Trimesh to render final hand-object meshes/point clouds
- Outputs saved as .png and .npy under sample workflows

##### Architectural patterns & principles
- Layered architecture: Data → Processing → Model Training/Inference → Visualization
- Modular separation of concerns: two independent generative networks (PointCVAE, SupNet) share dataset and utility code.
- Model-driven flow: trained checkpoints feed into inference module (DCoGModel).
- Config-driven deployment: single config.json toggles dataset paths & hyperparameters.

##### Essential elements
- Clear, descriptive labels on all components (e.g., “PointCVAE Training Module,” “MANO Hand Model”).

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Data Layer"
        ext1["OBMan Dataset"]:::data
        ext2["ShapeNetCore.v2 Dataset"]:::data
        sample["Sample Inputs (.npy, .obj)"]:::data
    end
    click sample "https://github.com/wmtlab/graingrasp/tree/main/sample/"

    subgraph "Data Processing"
        dp["dataprocess.py"]:::process
        load["load_obman.py"]:::process
        annot["annotate.py"]:::process
        loss["loss.py"]:::process
        tools["tools.py"]:::process
        vis_util["vis.py"]:::process
    end
    click dp "https://github.com/wmtlab/graingrasp/blob/main/dataprocess.py"
    click load "https://github.com/wmtlab/graingrasp/blob/main/utils/load_obman.py"
    click annot "https://github.com/wmtlab/graingrasp/blob/main/utils/annotate.py"
    click loss "https://github.com/wmtlab/graingrasp/blob/main/utils/loss.py"
    click tools "https://github.com/wmtlab/graingrasp/blob/main/utils/tools.py"
    click vis_util "https://github.com/wmtlab/graingrasp/blob/main/utils/vis.py"

    subgraph "Model Training"
        subgraph "PointCVAE Training"
            p_train["PointCVAE/train.py"]:::train
            p_ds["PointCVAE/obman_dataset.py"]:::train
            p_net["PointCVAE Network"]:::train
        end
        subgraph "SupNet Training"
            s_train["SupNet/train.py"]:::train
            s_ds["SupNet/obman_dataset.py"]:::train
            s_net["SupNet Network"]:::train
        end
    end
    click p_train "https://github.com/wmtlab/graingrasp/blob/main/PointCVAE/train.py"
    click p_ds "https://github.com/wmtlab/graingrasp/blob/main/PointCVAE/obman_dataset.py"
    click s_train "https://github.com/wmtlab/graingrasp/blob/main/SupNet/train.py"
    click s_ds "https://github.com/wmtlab/graingrasp/blob/main/SupNet/obman_dataset.py"

    subgraph "Saved Checkpoints"
        ckpt1["PointCVAE/model.pth"]:::train
        ckpt2["SupNet/model.pth"]:::train
    end
    click ckpt1 "https://github.com/wmtlab/graingrasp/blob/main/PointCVAE/model.pth"
    click ckpt2 "https://github.com/wmtlab/graingrasp/blob/main/SupNet/model.pth"

    subgraph "Inference & Optimization"
        inf_ctrl["DCoGModel Controller (DCoG/DCoGModel.py)"]:::infer
        p_inf["PointCVAE Inference (inference.py)"]:::infer
        s_inf["SupNet Inference (inference.py)"]:::infer
    end
    click inf_ctrl "https://github.com/wmtlab/graingrasp/blob/main/DCoG/DCoGModel.py"
    click p_inf "https://github.com/wmtlab/graingrasp/blob/main/PointCVAE/inference.py"
    click s_inf "https://github.com/wmtlab/graingrasp/blob/main/SupNet/inference.py"

    subgraph "MANO Hand Model"
        mano_mod["MANO Model (mano/model.py)"]:::ext
    end
    click mano_mod "https://github.com/wmtlab/graingrasp/blob/main/mano/model.py"

    subgraph "Visualization & Output"
        viz["Final Rendering (vis.py, Open3D/Trimesh)"]:::process
    end

    subgraph "Entry Scripts"
        run_all["run_complete.py"]:::ext
        run_opt["run_only_opt.py"]:::ext
        grain["GrainGrasp.py"]:::ext
    end
    click run_all "https://github.com/wmtlab/graingrasp/blob/main/run_complete.py"
    click run_opt "https://github.com/wmtlab/graingrasp/blob/main/run_only_opt.py"
    click grain "https://github.com/wmtlab/graingrasp/blob/main/GrainGrasp.py"

    subgraph "Configuration"
        cfg_json["config.json"]:::ext
        cfg_py["config.py"]:::ext
    end
    click cfg_json "https://github.com/wmtlab/graingrasp/blob/main/config.json"
    click cfg_py "https://github.com/wmtlab/graingrasp/blob/main/config.py"

    subgraph "External Dependencies"
        pytorch["PyTorch"]:::ext
        pyt3d["PyTorch3D"]:::ext
        open3d["Open3D"]:::ext
        trimesh["Trimesh"]:::ext
        numpy["NumPy"]:::ext
        mano_dep["MANO Library"]:::ext
    end

    %% Connections
    ext1 -->|"raw meshes"| dp
    ext2 -->|"raw meshes"| dp
    sample -->|"input files"| dp

    dp -->|"processed point clouds"| p_train
    dp -->|"processed point clouds"| s_train

    p_train -->|"trained weights"| ckpt1
    s_train -->|"trained weights"| ckpt2

    ckpt1 -->|"weights"| inf_ctrl
    ckpt2 -->|"weights"| inf_ctrl

    run_all -->|"full pipeline"| inf_ctrl
    run_opt -->|"opt-only pipeline"| inf_ctrl

    inf_ctrl -->|"init grasps"| p_inf
    inf_ctrl -->|"init grasps"| s_inf

    inf_ctrl -->|"optimized pose"| mano_mod
    mano_mod -->|"mesh params"| viz
    inf_ctrl -->|"final clouds"| viz

    cfg_json --> dp
    cfg_json --> p_train
    cfg_json --> s_train
    cfg_json --> inf_ctrl

    pytorch --> p_train
    pytorch --> s_train
    pytorch --> inf_ctrl

    pyt3d --> p_train
    open3d --> viz
    trimesh --> viz
    numpy --> dp
    mano_dep --> mano_mod

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef process fill:#90EE90,stroke:#000;
    classDef train fill:#FFA500,stroke:#000;
    classDef infer fill:#9370DB,stroke:#000;
    classDef ext fill:#D3D3D3,stroke:#000;
```
