# ayluuur/mediapipe2mano architecture

> Architecture diagram of the GitHub repository ayluuur/mediapipe2mano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ayluuur/mediapipe2mano
- Repository: https://github.com/ayluuur/mediapipe2mano

## Overview

MediaPipe2Mano is a C++ runtime and Qt viewer that turns mirrored RGB camera or video frames into tracked hand landmarks, MANO/IK hand geometry, and an interactive 3D preview. The README documents three viewer modes, depth estimation, filtering, interaction, and parallel IK. The graph uses documented capabilities and paths present in the tree; implementation excerpts were not provided, so uncertain internal call wiring is omitted.

Main entry points: [`MediaPipe2ManoRuntime/src/core.cpp`](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp) (Runtime Core), [`MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp`](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp) (Viewer Runtime Client), [`MediaPipe2ManoViewer/application.cpp`](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/application.cpp) (Mode Selection), [`MediaPipe2ManoRuntime/src/runtime.cpp`](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/runtime.cpp) (Frame Processing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_viewer["Viewer"]
  node_application["Mode Selection<br/>[application.cpp]"]
  node_viewer_config["Mode Config<br/>[RuntimeConfig.h]"]
  node_viewer_runtime["Viewer Runtime Client"]
  node_preview["2D And 3D Preview"]
end

subgraph group_runtime["Hand Runtime"]
  node_runtime_api["Runtime API<br/>[Runtime.h]"]
  node_runtime_config["Runtime Configuration<br/>[runtime_config.cpp]"]
  node_runtime_orchestrator["Frame Processing<br/>[runtime.cpp]"]
  node_detector["Hand Detector<br/>[detector.cpp]"]
  node_mediapipe_bridge["MediaPipe Bridge<br/>[bridge.cc]"]
  node_core["Runtime Core<br/>[core.cpp]"]
end

subgraph group_tracking["Pose Processing"]
  node_tracking["Hand Tracking<br/>[core.cpp]"]
  node_depth["Depth Estimation<br/>[core.cpp]"]
  node_mano_ik["MANO Inverse Kinematics<br/>[core.cpp]"]
  node_skeleton["Skeleton Output<br/>[skeleton.cpp]"]
  node_interaction["Interaction State<br/>[core.cpp]"]
end

subgraph group_integrations["Models And Rendering"]
  node_mano_models["MANO Models"]
  node_mediapipe_models["MediaPipe Models"]
  node_qt["Qt UI"]
  node_opencv["OpenCV"]
  node_open3d["Open3D Renderer"]
end

node_user(("Viewer User"))
node_frames(("Camera Or Video"))

node_user -->|"selects mode"| node_application
node_application -->|"selects config"| node_viewer_config
node_application -->|"starts"| node_viewer_runtime
node_frames -->|"provides frames"| node_viewer_runtime
node_viewer_runtime -->|"submits frames"| node_runtime_api
node_viewer_runtime -->|"sets options"| node_runtime_config
node_runtime_api -->|"dispatches"| node_runtime_orchestrator
node_runtime_orchestrator -->|"requests detection"| node_detector
node_detector -->|"runs hand inference"| node_mediapipe_bridge
node_mediapipe_bridge -->|"loads model"| node_mediapipe_models
node_runtime_orchestrator -->|"processes results"| node_core
node_core -->|"stabilizes hands"| node_tracking
node_core -->|"estimates depth"| node_depth
node_core -->|"solves pose"| node_mano_ik
node_mano_ik -->|"uses hand model"| node_mano_models
node_core -->|"produces skeleton"| node_skeleton
node_core -->|"updates interaction"| node_interaction
node_viewer_runtime -->|"presents results"| node_preview
node_preview -->|"displays UI"| node_qt
node_preview -->|"draws annotations"| node_opencv
node_preview -->|"renders hand mesh"| node_open3d

click node_application "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/application.cpp"
click node_viewer_config "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/include/MediaPipe2Mano/RuntimeConfig.h"
click node_viewer_runtime "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp"
click node_runtime_api "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/include/MediaPipe2Mano/Runtime.h"
click node_runtime_config "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/runtime_config.cpp"
click node_runtime_orchestrator "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/runtime.cpp"
click node_detector "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/detector.cpp"
click node_mediapipe_bridge "https://github.com/ayluuur/mediapipe2mano/blob/main/ThirdParty/MediaPipe/bridge.cc"
click node_core "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp"
click node_tracking "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp"
click node_depth "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp"
click node_mano_ik "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp"
click node_skeleton "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/skeleton.cpp"
click node_interaction "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp"
click node_preview "https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_application,node_viewer_config,node_viewer_runtime,node_preview,node_user toneBlue
class node_runtime_api,node_runtime_config,node_runtime_orchestrator,node_detector,node_mediapipe_bridge,node_core toneAmber
class node_tracking,node_depth,node_mano_ik,node_skeleton,node_interaction toneMint
class node_mano_models,node_mediapipe_models,node_qt,node_opencv,node_open3d toneRose
class node_frames toneIndigo
```

## Components

### Viewer

- [Mode Selection](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/application.cpp): `MediaPipe2ManoViewer/application.cpp`
- [Mode Config](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/include/MediaPipe2Mano/RuntimeConfig.h): `MediaPipe2ManoRuntime/include/MediaPipe2Mano/RuntimeConfig.h`
- [Viewer Runtime Client](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp): `MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp`
- [2D And 3D Preview](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp): `MediaPipe2ManoViewer/MediaPipe2ManoViewer.cpp`

### Hand Runtime

- [Runtime API](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/include/MediaPipe2Mano/Runtime.h): `MediaPipe2ManoRuntime/include/MediaPipe2Mano/Runtime.h`
- [Runtime Configuration](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/runtime_config.cpp): `MediaPipe2ManoRuntime/src/runtime_config.cpp`
- [Frame Processing](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/runtime.cpp): `MediaPipe2ManoRuntime/src/runtime.cpp`
- [Hand Detector](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/detector.cpp): `MediaPipe2ManoRuntime/src/detector.cpp`
- [MediaPipe Bridge](https://github.com/ayluuur/mediapipe2mano/blob/main/ThirdParty/MediaPipe/bridge.cc): `ThirdParty/MediaPipe/bridge.cc`
- [Runtime Core](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp): `MediaPipe2ManoRuntime/src/core.cpp`

### Pose Processing

- [Hand Tracking](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp): `MediaPipe2ManoRuntime/src/core.cpp`
- [Depth Estimation](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp): `MediaPipe2ManoRuntime/src/core.cpp`
- [MANO Inverse Kinematics](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp): `MediaPipe2ManoRuntime/src/core.cpp`
- [Skeleton Output](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/skeleton.cpp): `MediaPipe2ManoRuntime/src/skeleton.cpp`
- [Interaction State](https://github.com/ayluuur/mediapipe2mano/blob/main/MediaPipe2ManoRuntime/src/core.cpp): `MediaPipe2ManoRuntime/src/core.cpp`

### Models And Rendering

- **MANO Models**
- **MediaPipe Models**
- **Qt UI**
- **OpenCV**
- **Open3D Renderer**

### Other components

- **Viewer User**
- **Camera Or Video**

## Connections

- Viewer User → Mode Selection: selects mode
- Mode Selection → Mode Config: selects config
- Mode Selection → Viewer Runtime Client: starts
- Camera Or Video → Viewer Runtime Client: provides frames
- Viewer Runtime Client → Runtime API: submits frames
- Viewer Runtime Client → Runtime Configuration: sets options
- Runtime API → Frame Processing: dispatches
- Frame Processing → Hand Detector: requests detection
- Hand Detector → MediaPipe Bridge: runs hand inference
- MediaPipe Bridge → MediaPipe Models: loads model
- Frame Processing → Runtime Core: processes results
- Runtime Core → Hand Tracking: stabilizes hands
- Runtime Core → Depth Estimation: estimates depth
- Runtime Core → MANO Inverse Kinematics: solves pose
- MANO Inverse Kinematics → MANO Models: uses hand model
- Runtime Core → Skeleton Output: produces skeleton
- Runtime Core → Interaction State: updates interaction
- Viewer Runtime Client → 2D And 3D Preview: presents results
- 2D And 3D Preview → Qt UI: displays UI
- 2D And 3D Preview → OpenCV: draws annotations
- 2D And 3D Preview → Open3D Renderer: renders hand mesh