How vegetebird/mhformer works
Project type and purpose
• This is a research‐grade, full-stack deep learning codebase for 3D human pose estimation called “MHFormer: Multi‐Hypothesis Transformer for 3D Human Pose Estimation.”
• The core functionality:
a) Load 2D keypoints (from ground truth or detector)
b) Feed sequences of 2D keypoints into a transformer‐based model (MHFormer)
c) Output 3D joint coordinates
d) Provide training, testing, and an in‐the‐wild demo pipeline using YOLOv3 and HRNet for 2D detection.
File structure
Top‐level organization:
• checkpoint/pretrained: stores downloaded pretrained weights
• common: data‐loading, skeleton definitions, utilities
• dataset: scripts and processed .npz data for Human3.6M
• model: MHFormer definition and sub‐modules (transformer, hypothesis generator)
• demo: live pipeline for videos (lib subfolder with YOLOv3 & HRNet detector code, video I/O, visualization)
• main.py: entrypoint for train/test
• requirements.txt: environment dependencies
Key patterns:
• Modular code (common / model / demo)
• Clear separation of data, model definition, and application scripts
• Use of PyTorch for model, YAML configs for detectors
README for additional insights
• Describes two workflows:
• Training / testing on Human3.6M dataset using main.py
• Demo pipeline: video → YOLOv3 human detection → HRNet keypoint extraction → MHFormer 3D estimation → visualization
• Lists external dependencies (PyTorch, torchvision, sorted detector, pretrained weights)
• Points to directory conventions: dataset/, checkpoint/, demo/video/