How saidwivedi/tokenhmr works
Project type and purpose
• TokenHMR is a research-oriented, open-source computer-vision & machine-learning tool for 3D human mesh recovery. It comprises two major pipelines:
a. Tokenization (VQ-style pose token learning)
b. TokenHMR (human mesh recovery using tokenized poses and Threshold-Adaptive Loss Scaling)
File structure
• Top-level directories:
• tokenization/ ← code, configs, data loaders & training scripts for the tokenizer (Method a)
• tokenhmr/ ← code, Hydra configs, data loaders, models, train/eval/demo scripts for the HMR network (Method b)
• demo_sample/ ← sample images & video for quick demos
• data/ ← SMPL/SMPLH body-model files & downloaded checkpoints
• assets/ ← figures, teaser gifs
• tokenhmr/lib/ ← organized into configs, datasets, models, utils
• tokenization/… ← split into configs, dataset, models, utils, scripts
README for additional insights
• Two-stage method:
1) Tokenization: continuous SMPLH poses → discrete tokens
2) TokenHMR: image/video → keypoints (Detectron2/PHALP) → pose tokens → mesh parameters
• Heavy use of Hydra for config management
• External dependencies: PyTorch, Detectron2, a fork of PHALP
• Data sources: AMASS, MOYO (tokenization), BEDLAM, 4DHumans, 3DPW, EMDB (HMR)
Don’t overthink—capture all essential components
• Labels match directory names for quick reference
• How scripts (train_poseVQ.py, train.py, eval.py, demo.py, track.py) orchestrate pipelines
• Checkpoints flow from training modules to demo modules