How google-ai-edge/mediapipe works
MediaPipe is an on-device machine-learning toolkit spanning cross-platform Tasks APIs and a lower-level packet-and-calculator graph framework. A typical user supplies media or other task input and receives inference results; the repository also includes specialized processing such as face-landmark-to-3D-geometry conversion. The tree and excerpts show framework primitives, inference support, and geometry implementations, but not a complete task invocation path, so uncertain workflow links are dashed. The README documents broader audio, decision, generative-AI, retrieval, text, and vision capabilities not individually sampled here.
Main entry points: calculator_graph.h Calculator graph · tasks Solution families · inference_calculator.cc Tensor inference · geometry_pipeline_calculator.cc Geometry calculator
Components 23
Product interfaces
- Tasks APIs
mediapipe/tasks - Solution families
mediapipe/tasks - Task results
mediapipe/tasks
Graph runtime
- Framework API
mediapipe/framework/calculator_framework.h - Calculator graph
mediapipe/framework/calculator_graph.h - Packets
mediapipe/framework/packet.h - Stream management
mediapipe/framework/input_stream_manager.cc - Calculator library
mediapipe/calculators
Inference processing
- Tensor preparation
mediapipe/calculators/tensor - Tensor inference
mediapipe/calculators/tensor/inference_calculator.cc - Inference I/O mapping
mediapipe/calculators/tensor/inference_io_mapper.h - Inference feedback
mediapipe/calculators/tensor/inference_feedback_manager.cc
Domain processing
- Face landmarks
mediapipe/modules/face_geometry/geometry_pipeline_calculator.cc - Geometry calculator
mediapipe/modules/face_geometry/geometry_pipeline_calculator.cc - 3D geometry pipeline
mediapipe/modules/face_geometry/libs/geometry_pipeline.cc - Tasks geometry
mediapipe/tasks/cc/vision/face_geometry/calculators/geometry_pipeline_calculator.cc - Tasks geometry pipeline
mediapipe/tasks/cc/vision/face_geometry/libs/geometry_pipeline.cc - Audio processing
mediapipe/calculators/audio - Vision processing
mediapipe/calculators/image - Text processing
mediapipe/calculators/tensor
Other components
- App developer
- Media input
- Application output
Connections 22
- App developer → Tasks APIs: uses
README.md - App developer → Framework API: builds with
README.md - Media input → Tasks APIs: supplies input
- Tasks APIs → Solution families: exposes
README.md - Tasks APIs → Calculator graph: routes through
- Framework API → Calculator graph: provides graph API
mediapipe/framework/calculator_framework.h - Calculator graph → Packets: exchanges
- Calculator graph → Stream management: dispatches streams
- Calculator graph → Calculator library: runs
- Calculator library → Tensor preparation: prepares tensors
mediapipe/calculators/tensor/inference_feedback_manager_litert.cc - Tensor preparation → Tensor inference: provides tensors
- Tensor inference → Inference I/O mapping: remaps I/O
- Tensor inference → Inference feedback: manages feedback
- Tensor inference → Task results: produces results
- Solution families → Audio processing: supports audio
README.md - Solution families → Vision processing: supports vision
README.md - Solution families → Text processing: supports text
README.md - Face landmarks → Geometry calculator: feeds geometry
mediapipe/modules/face_geometry/geometry_pipeline_calculator.cc - Geometry calculator → 3D geometry pipeline: invokes pipeline
mediapipe/modules/face_geometry/geometry_pipeline_calculator.cc - Tasks geometry → Tasks geometry pipeline: invokes pipeline
mediapipe/tasks/cc/vision/face_geometry/calculators/geometry_pipeline_calculator.cc - Task results → Application output: returns result
- Geometry calculator → Application output: emits geometry