How meowuu7/geneoh-diffusion works
Project type and purpose
• This is a machine-learning research codebase implementing the “Generalizable Hand-Object Interaction (HOI) Denoising” diffusion model in PyTorch.
• It provides end-to-end pipelines: data preparation, training diffusion models (motion & spatial), inference/denoising on test datasets (GRAB, TACO, HOI4D), mesh reconstruction, and visualization.
Core logical components
a. Data Sources (External)
• GRAB, HOI4D, TACO raw and pre-processed data under /data/...
• Anchor weights & mesh files under /assets/...
b. Preprocessing & Data Loaders
• prepare/ scripts (shell) to download and organise datasets
• Python loader modules in data_loaders/ and shared utilities in common/ & utils/
c. Core Model & Diffusion Engine
• Network definitions in model/ (DGCNN, PointNet2, MDM, SMPL models)
• Diffusion algorithm and training losses in diffusion/
• MANO hand-model layer provided by the local manopth/ package
d. Training Pipeline
• Shell scripts scripts/train/*.sh call Python trainers in train/ (e.g., train_mdm.py, training_loop.py)
• Produces checkpoints in ckpts/
e. Inference/Evaluation Pipeline
• Shell scripts scripts/val/*.sh orchestrate denoising and reconstruction
• Sample inference entrypoints in sample/ (e.g., predict_grab.py)
• Mesh reconstruction utilities (inside utils/ and common/rend_utils.py)
f. Visualization
Interactions & Data Flow
• Start: CLI (shell) → selects dataset paths & parameters →
Data loaders in data_loaders/ + shared common/ modules load .npy/.pkl →
Preprocessed data fed into diffusion networks from diffusion/ + model definitions from model/ →
Output latent/object/hand trajectories →
Mesh reconstruction (MANO in manopth/ + rendering utils) →
Save denoised trajectories under data/.../result →
Visualization scripts pick up results and generate figures/videos.
Architectural patterns & design principles
• Modular design: clear separation between data I/O, model logic, training loops, inference scripts, and visualization
• Reuse of shared utilities: common/ vs. project-specific utils/ for multidomain support
• Shell script orchestration: reproducible pipelines for training & evaluation
• Pluggable dataset backends: each dataset loader conforms to a common interface
• External dependence isolation: local manopth/ ensures consistent MANO behavior