How zju3dv/gvhmr works
GVHMR is a human-motion recovery system that estimates 3D body motion in gravity-aligned world coordinates from video-derived observations. The documented workflow is video input, optional camera-motion estimation, motion recovery, then evaluation or visualization; training and dataset testing are also supported. The graph includes the model’s conditioning, decoding, and post-processing layers, plus documented training-data and metric branches. Some demo and preprocessing wiring is unsampled, so uncertain links are omitted or marked unverified rather than inferred from module names.
Main entry points: gvhmr_pipeline.py Recovery pipeline · store_gvhmr.py GVHMR config · build_gvhmr.py Demo builder · gvhmr_pl.py Training model
Components 21
Inputs and preprocessing
- Demo builder
hmr4d/build_gvhmr.py - Video I/O
hmr4d/utils/video_io_utils.py - Pose features
hmr4d/utils/preproc/vitpose.py - Camera motion
hmr4d/utils/preproc/relpose/simple_vo.py
Motion recovery
- Training model
hmr4d/model/gvhmr/gvhmr_pl.py - Recovery pipeline
hmr4d/model/gvhmr/pipeline/gvhmr_pipeline.py - Motion transformer
hmr4d/network/gvhmr/relative_transformer.py - Motion decoder
hmr4d/model/gvhmr/utils/endecoder.py - Motion postprocess
hmr4d/model/gvhmr/utils/postprocess.py - Motion statistics
hmr4d/model/gvhmr/utils/stats_compose.py
Training and evaluation
- Image-motion data
hmr4d/dataset/imgfeat_motion/base_dataset.py - BEDLAM dataset
hmr4d/dataset/bedlam/bedlam.py - Motion datasets
hmr4d/dataset/pure_motion - Data module
hmr4d/datamodule/mocap_trainX_testY.py - Train and test
tools/train.py - GVHMR config
hmr4d/configs/store_gvhmr.py - Dataset metrics
hmr4d/model/gvhmr/callbacks
Geometry and visualization
- SMPL body models
hmr4d/utils/smplx_utils.py - World coordinates
hmr4d/utils/geo/hmr_global.py - Motion visualization
hmr4d/utils/vis/renderer.py
Other components
- Video user
Connections 23
- Video user → Demo builder: runs demo
- Demo builder → Video I/O: uses video tools
- Demo builder → Pose features: extracts pose
- Demo builder → Camera motion: estimates camera
- Train and test → GVHMR config: loads configuration
tools/train.py - GVHMR config → BEDLAM dataset: registers dataset
hmr4d/configs/store_gvhmr.py - GVHMR config → Motion datasets: registers datasets
hmr4d/configs/store_gvhmr.py - GVHMR config → Training model: registers model
hmr4d/configs/store_gvhmr.py - GVHMR config → Motion decoder: registers decoder
hmr4d/configs/store_gvhmr.py - GVHMR config → Dataset metrics: registers metrics
hmr4d/configs/store_gvhmr.py - BEDLAM dataset → Image-motion data: inherits dataset
hmr4d/dataset/bedlam/bedlam.py - Data module → BEDLAM dataset: loads samples
- Data module → Motion datasets: loads samples
- Training model → Recovery pipeline: runs pipeline
- Recovery pipeline → Motion decoder: decodes predictions
hmr4d/model/gvhmr/pipeline/gvhmr_pipeline.py - Recovery pipeline → Motion statistics: normalizes features
hmr4d/model/gvhmr/pipeline/gvhmr_pipeline.py - Recovery pipeline → Motion transformer: invokes denoiser
- Training model → Motion postprocess: post-processes output
hmr4d/model/gvhmr/gvhmr_pl.py - Motion postprocess → Motion decoder: uses decoder
hmr4d/model/gvhmr/utils/postprocess.py - Recovery pipeline → SMPL body models: computes body joints
hmr4d/model/gvhmr/pipeline/gvhmr_pipeline.py - Recovery pipeline → World coordinates: uses world transforms
hmr4d/model/gvhmr/pipeline/gvhmr_pipeline.py - Dataset metrics → SMPL body models: evaluates body output
hmr4d/model/gvhmr/callbacks/metric_rich.py - Dataset metrics → Motion visualization: visualizes results
hmr4d/model/gvhmr/callbacks/metric_rich.py