How twehrbein/footmr works
FootMR refines 3D foot motion in monocular human-motion capture. The demo workflow preprocesses a video into tracked-person, 2D-pose, and camera-motion cues, obtains an initial human recovery, then lifts foot-keypoint sequences to refined 3D motion and renders results. Training and evaluation form a separate supported branch using motion and image-feature datasets and metrics on MOOF, MOYO, and RICH. The graph preserves documented capabilities where implementation excerpts are incomplete; some subsystem links therefore cite the README rather than sampled call sites.
Main entry points: footmr_pipeline.py FootMR Pipeline · demo.py Demo Runner · train.py Training Runner · mocap_trainX_testY.py Motion Data Module
Components 26
Video Reconstruction
- Demo Runner
tools/demo.py - Person Tracking
hmr4d/utils/preproc/tracker.py - 2D Pose Extraction
hmr4d/utils/preproc/vitpose.py - Sapiens Pose
hmr4d/utils/preproc/sapiens.py - Camera Motion
hmr4d/utils/preproc/slam.py - Initial Recovery
hmr4d/network/hmr2 - Demo Model Adapter
hmr4d/model/footmr/footmr_pl_demo.py - Motion Rendering
hmr4d/utils/vis/renderer.py - Rendered Video
hmr4d/utils/video_io_utils.py
Foot Refinement
- FootMR Pipeline
hmr4d/model/footmr/pipeline/footmr_pipeline.py - Foot Transformer
hmr4d/network/footmr/foot_transformer.py - Motion Encoding
hmr4d/model/footmr/utils/endecoder.py - Motion Statistics
hmr4d/model/footmr/utils/stats_compose.py - Motion Postprocess
hmr4d/model/footmr/utils/postprocess.py
Training and Evaluation
- Training Runner
tools/train.py - Motion Data Module
hmr4d/datamodule/mocap_trainX_testY.py - Image-Feature Data
hmr4d/dataset/imgfeat_motion/base_dataset.py - Motion Datasets
hmr4d/dataset/pure_motion - FootMR Trainer
hmr4d/model/footmr/footmr_pl.py - Benchmark Metrics
hmr4d/model/footmr/callbacks - Evaluation Utilities
hmr4d/utils/eval/eval_utils.py
Motion Foundations
- SMPL Body Model
hmr4d/utils/smplx_utils.py - Camera Geometry
hmr4d/utils/geo/hmr_cam.py - Global Coordinates
hmr4d/utils/geo/hmr_global.py - Rotation Math
hmr4d/utils/matrix.py
Other components
- Video User
Connections 28
- Video User → Demo Runner: provides video
- Demo Runner → Person Tracking: tracks person
tools/demo.py - Demo Runner → 2D Pose Extraction: extracts pose
tools/demo.py - Demo Runner → Sapiens Pose: optionally extracts
tools/demo.py - Demo Runner → Camera Motion: estimates motion
tools/demo.py - Demo Runner → Initial Recovery: gets initial recovery
- Demo Runner → Demo Model Adapter: runs refinement
tools/demo.py - Demo Model Adapter → FootMR Pipeline: invokes pipeline
- FootMR Pipeline → Foot Transformer: predicts motion
hmr4d/model/footmr/pipeline/footmr_pipeline.py - FootMR Pipeline → Motion Encoding: decodes output
hmr4d/model/footmr/pipeline/footmr_pipeline.py - FootMR Pipeline → Motion Statistics: normalizes with
hmr4d/model/footmr/pipeline/footmr_pipeline.py - FootMR Pipeline → Motion Postprocess: postprocesses
hmr4d/model/footmr/pipeline/footmr_pipeline.py - FootMR Pipeline → Camera Geometry: computes translation
hmr4d/model/footmr/pipeline/footmr_pipeline.py - FootMR Pipeline → Global Coordinates: converts coordinates
hmr4d/model/footmr/pipeline/footmr_pipeline.py - Motion Encoding → Motion Statistics: uses statistics
hmr4d/model/footmr/utils/endecoder.py - Foot Transformer → SMPL Body Model: outputs body motion
- FootMR Pipeline → Motion Rendering: supplies refined motion
- Demo Runner → Motion Rendering: renders results
tools/demo.py - Motion Rendering → Rendered Video: writes frames
tools/demo.py - Training Runner → Motion Data Module: configures training
- Motion Data Module → Image-Feature Data: loads image features
- Motion Data Module → Motion Datasets: loads motion data
- Training Runner → FootMR Trainer: trains model
- FootMR Trainer → FootMR Pipeline: uses refinement
- Training Runner → Benchmark Metrics: runs benchmarks
- Benchmark Metrics → Evaluation Utilities: computes metrics
hmr4d/model/footmr/callbacks/metric_moof.py - Camera Geometry → Rotation Math: uses transforms
hmr4d/utils/geo/hmr_cam.py - Global Coordinates → Rotation Math: uses transforms
hmr4d/utils/geo/hmr_global.py