How ayluuur/mediapipe2mano works
MediaPipe2Mano is a C++ runtime and Qt viewer that turns mirrored RGB camera or video frames into tracked hand landmarks, MANO/IK hand geometry, and an interactive 3D preview. The README documents three viewer modes, depth estimation, filtering, interaction, and parallel IK. The graph uses documented capabilities and paths present in the tree; implementation excerpts were not provided, so uncertain internal call wiring is omitted.
Main entry points: core.cpp Runtime Core · MediaPipe2ManoViewer.cpp Viewer Runtime Client · application.cpp Mode Selection · runtime.cpp Frame Processing
Components 22
Viewer
- Mode Selection
MediaPipe2ManoViewer/application.cpp - Mode Config
MediaPipe2ManoRuntime/include/MediaPipe2Mano/RuntimeConfig.h - Viewer Runtime Client
MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp - 2D And 3D Preview
MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp
Hand Runtime
- Runtime API
MediaPipe2ManoRuntime/include/MediaPipe2Mano/Runtime.h - Runtime Configuration
MediaPipe2ManoRuntime/src/runtime_config.cpp - Frame Processing
MediaPipe2ManoRuntime/src/runtime.cpp - Hand Detector
MediaPipe2ManoRuntime/src/detector.cpp - MediaPipe Bridge
ThirdParty/MediaPipe/bridge.cc - Runtime Core
MediaPipe2ManoRuntime/src/core.cpp
Pose Processing
- Hand Tracking
MediaPipe2ManoRuntime/src/core.cpp - Depth Estimation
MediaPipe2ManoRuntime/src/core.cpp - MANO Inverse Kinematics
MediaPipe2ManoRuntime/src/core.cpp - Skeleton Output
MediaPipe2ManoRuntime/src/skeleton.cpp - Interaction State
MediaPipe2ManoRuntime/src/core.cpp
Models And Rendering
- MANO Models
- MediaPipe Models
- Qt UI
- OpenCV
- Open3D Renderer
Other components
- Viewer User
- Camera Or Video
Connections 21
- Viewer User → Mode Selection: selects mode
- Mode Selection → Mode Config: selects config
- Mode Selection → Viewer Runtime Client: starts
- Camera Or Video → Viewer Runtime Client: provides frames
- Viewer Runtime Client → Runtime API: submits frames
- Viewer Runtime Client → Runtime Configuration: sets options
- Runtime API → Frame Processing: dispatches
- Frame Processing → Hand Detector: requests detection
- Hand Detector → MediaPipe Bridge: runs hand inference
- MediaPipe Bridge → MediaPipe Models: loads model
- Frame Processing → Runtime Core: processes results
- Runtime Core → Hand Tracking: stabilizes hands
- Runtime Core → Depth Estimation: estimates depth
- Runtime Core → MANO Inverse Kinematics: solves pose
- MANO Inverse Kinematics → MANO Models: uses hand model
- Runtime Core → Skeleton Output: produces skeleton
- Runtime Core → Interaction State: updates interaction
- Viewer Runtime Client → 2D And 3D Preview: presents results
- 2D And 3D Preview → Qt UI: displays UI
- 2D And 3D Preview → OpenCV: draws annotations
- 2D And 3D Preview → Open3D Renderer: renders hand mesh