How hailo-ai/hailort works
HailoRT is a host-side runtime for Hailo inference devices, with a C/C++ library, CLI, Python API, and GStreamer element. A typical user workflow is to use an API or CLI to configure a device and run inference, then consume results or statistics. The graph preserves the documented integration surfaces and the library’s model, pipeline, and device layers. GenAI and transport subsystems are represented from the tree; their internal relationships are unverified because the excerpts do not show their call paths.
Main entry points: hailort.h C API · pipeline.cpp Inference pipeline · inference_manager.cpp GenAI inference · hailortcli.cpp HailoRT CLI
Components 21
User interfaces
- HailoRT CLI
hailort/hailortcli/hailortcli.cpp - Python API
hailort/libhailort/bindings/python/src/pyhailort.cpp - GStreamer HailoNet
hailort/libhailort/bindings/gstreamer/gst-hailo/gsthailonet.cpp
Inference runtime
- C API
hailort/libhailort/include/hailo/hailort.h - HEF model
hailort/libhailort/src/hef/hef.cpp - Infer model
hailort/libhailort/include/hailo/infer_model.hpp - Network groups
hailort/libhailort/include/hailo/network_group.hpp - Pipeline builder
hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp - Inference pipeline
hailort/libhailort/src/net_flow/pipeline/pipeline.cpp - Post-processing
hailort/libhailort/src/net_flow/ops/yolov8_post_process.cpp - Buffer storage
hailort/libhailort/src/utils/buffer_storage.cpp - Resource manager
hailort/libhailort/src/core_op/resource_manager/resource_manager.cpp - Inference statistics
hailort/libhailort/include/hailo/runtime_statistics.hpp
Device communication
- Device API
hailort/libhailort/src/device_common/device.cpp - HRPC transport
hailort/hrpc/client.cpp - Hailo device driver
Generative AI
- GenAI inference
hailort/hailort_server/genai/inference_manager.cpp - LLM inference
hailort/hailort_server/genai/llm/llm_inference_manager.cpp - Speech to text
hailort/hailort_server/genai/speech2text/speech2text_server.cpp - Vision-language
hailort/hailort_server/genai/vlm/vlm_server.cpp
Other components
- Application user
Connections 22
- Application user → HailoRT CLI: invokes
- Application user → Python API: uses
- Application user → GStreamer HailoNet: uses
- Application user → C API: calls
- HailoRT CLI → Device API: controls device
- HailoRT CLI → Inference pipeline: runs inference
- Python API → C API: wraps
- GStreamer HailoNet → C API: uses runtime
- C API → HEF model: loads model
- C API → Infer model: configures inference
- Infer model → Network groups: configures
hailort/libhailort/include/hailo/infer_model.hpp - Pipeline builder → Inference pipeline: builds
hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp - Pipeline builder → Post-processing: adds operations
hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp - Inference pipeline → Buffer storage: uses buffers
- Resource manager → HEF model: reads metadata
hailort/libhailort/src/core_op/resource_manager/resource_manager.cpp - Device API → HRPC transport: uses transport
- HRPC transport → Hailo device driver: communicates
- GenAI inference → Infer model: runs inference
- LLM inference → GenAI inference: extends
- Speech to text → GenAI inference: uses inference
- Vision-language → GenAI inference: uses inference
- Inference pipeline → Inference statistics: reports metrics