# hailo-ai/hailort architecture

> Architecture diagram of the GitHub repository hailo-ai/hailort, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hailo-ai/hailort
- Repository: https://github.com/hailo-ai/hailort

## Overview

HailoRT is a host-side runtime for Hailo inference devices, with a C/C++ library, CLI, Python API, and GStreamer element. A typical user workflow is to use an API or CLI to configure a device and run inference, then consume results or statistics. The graph preserves the documented integration surfaces and the library’s model, pipeline, and device layers. GenAI and transport subsystems are represented from the tree; their internal relationships are unverified because the excerpts do not show their call paths.

Main entry points: [`hailort/libhailort/include/hailo/hailort.h`](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/hailort.h) (C API), [`hailort/libhailort/src/net_flow/pipeline/pipeline.cpp`](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/pipeline/pipeline.cpp) (Inference pipeline), [`hailort/hailort_server/genai/inference_manager.cpp`](https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/inference_manager.cpp) (GenAI inference), [`hailort/hailortcli/hailortcli.cpp`](https://github.com/hailo-ai/hailort/blob/master/hailort/hailortcli/hailortcli.cpp) (HailoRT CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_cli["HailoRT CLI<br/>[hailortcli.cpp]"]
  node_python["Python API<br/>[pyhailort.cpp]"]
  node_gst["GStreamer HailoNet<br/>[gsthailonet.cpp]"]
end

subgraph group_runtime["Inference runtime"]
  node_capi["C API<br/>[hailort.h]"]
  node_hef["HEF model<br/>[hef.cpp]"]
  node_infer_model["Infer model<br/>[infer_model.hpp]"]
  node_network_group["Network groups<br/>[network_group.hpp]"]
  node_pipeline_builder["Pipeline builder"]
  node_pipeline["Inference pipeline<br/>[pipeline.cpp]"]
  node_postprocess["Post-processing"]
  node_buffers["Buffer storage<br/>[buffer_storage.cpp]"]
  node_resource_manager["Resource manager"]
  node_stats["Inference statistics"]
end

subgraph group_device["Device communication"]
  node_device_api["Device API<br/>[device.cpp]"]
  node_hrpc["HRPC transport<br/>[client.cpp]"]
  node_driver["Hailo device driver"]
end

subgraph group_genai["Generative AI"]
  node_genai_manager["GenAI inference"]
  node_llm_manager["LLM inference"]
  node_s2t_server["Speech to text"]
  node_vlm_server["Vision-language<br/>[vlm_server.cpp]"]
end

node_developer(("Application user"))

node_developer -->|"invokes"| node_cli
node_developer -->|"uses"| node_python
node_developer -->|"uses"| node_gst
node_developer -->|"calls"| node_capi
node_cli -->|"controls device"| node_device_api
node_cli -->|"runs inference"| node_pipeline
node_python -->|"wraps"| node_capi
node_gst -->|"uses runtime"| node_capi
node_capi -.->|"loads model"| node_hef
node_capi -.->|"configures inference"| node_infer_model
node_infer_model -.->|"configures"| node_network_group
node_pipeline_builder -->|"builds"| node_pipeline
node_pipeline_builder -->|"adds operations"| node_postprocess
node_pipeline -.->|"uses buffers"| node_buffers
node_resource_manager -->|"reads metadata"| node_hef
node_device_api -.->|"uses transport"| node_hrpc
node_hrpc -.->|"communicates"| node_driver
node_genai_manager -->|"runs inference"| node_infer_model
node_llm_manager -->|"extends"| node_genai_manager
node_s2t_server -.->|"uses inference"| node_genai_manager
node_vlm_server -.->|"uses inference"| node_genai_manager
node_pipeline -.->|"reports metrics"| node_stats

click node_cli "https://github.com/hailo-ai/hailort/blob/master/hailort/hailortcli/hailortcli.cpp"
click node_python "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/bindings/python/src/pyhailort.cpp"
click node_gst "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/bindings/gstreamer/gst-hailo/gsthailonet.cpp"
click node_capi "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/hailort.h"
click node_hef "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/hef/hef.cpp"
click node_infer_model "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/infer_model.hpp"
click node_network_group "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/network_group.hpp"
click node_pipeline_builder "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp"
click node_pipeline "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/pipeline/pipeline.cpp"
click node_postprocess "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/ops/yolov8_post_process.cpp"
click node_buffers "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/utils/buffer_storage.cpp"
click node_resource_manager "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/core_op/resource_manager/resource_manager.cpp"
click node_device_api "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/device_common/device.cpp"
click node_hrpc "https://github.com/hailo-ai/hailort/blob/master/hailort/hrpc/client.cpp"
click node_genai_manager "https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/inference_manager.cpp"
click node_llm_manager "https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/llm/llm_inference_manager.cpp"
click node_s2t_server "https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/speech2text/speech2text_server.cpp"
click node_vlm_server "https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/vlm/vlm_server.cpp"
click node_stats "https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/runtime_statistics.hpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_python,node_gst,node_developer toneBlue
class node_capi,node_hef,node_infer_model,node_network_group,node_pipeline_builder,node_pipeline,node_postprocess,node_buffers,node_resource_manager,node_stats toneAmber
class node_device_api,node_hrpc,node_driver toneMint
class node_genai_manager,node_llm_manager,node_s2t_server,node_vlm_server toneRose
```

## Components

### User interfaces

- [HailoRT CLI](https://github.com/hailo-ai/hailort/blob/master/hailort/hailortcli/hailortcli.cpp): `hailort/hailortcli/hailortcli.cpp`
- [Python API](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/bindings/python/src/pyhailort.cpp): `hailort/libhailort/bindings/python/src/pyhailort.cpp`
- [GStreamer HailoNet](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/bindings/gstreamer/gst-hailo/gsthailonet.cpp): `hailort/libhailort/bindings/gstreamer/gst-hailo/gsthailonet.cpp`

### Inference runtime

- [C API](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/hailort.h): `hailort/libhailort/include/hailo/hailort.h`
- [HEF model](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/hef/hef.cpp): `hailort/libhailort/src/hef/hef.cpp`
- [Infer model](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/infer_model.hpp): `hailort/libhailort/include/hailo/infer_model.hpp`
- [Network groups](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/network_group.hpp): `hailort/libhailort/include/hailo/network_group.hpp`
- [Pipeline builder](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp): `hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp`
- [Inference pipeline](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/pipeline/pipeline.cpp): `hailort/libhailort/src/net_flow/pipeline/pipeline.cpp`
- [Post-processing](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/net_flow/ops/yolov8_post_process.cpp): `hailort/libhailort/src/net_flow/ops/yolov8_post_process.cpp`
- [Buffer storage](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/utils/buffer_storage.cpp): `hailort/libhailort/src/utils/buffer_storage.cpp`
- [Resource manager](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/core_op/resource_manager/resource_manager.cpp): `hailort/libhailort/src/core_op/resource_manager/resource_manager.cpp`
- [Inference statistics](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/include/hailo/runtime_statistics.hpp): `hailort/libhailort/include/hailo/runtime_statistics.hpp`

### Device communication

- [Device API](https://github.com/hailo-ai/hailort/blob/master/hailort/libhailort/src/device_common/device.cpp): `hailort/libhailort/src/device_common/device.cpp`
- [HRPC transport](https://github.com/hailo-ai/hailort/blob/master/hailort/hrpc/client.cpp): `hailort/hrpc/client.cpp`
- **Hailo device driver**

### Generative AI

- [GenAI inference](https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/inference_manager.cpp): `hailort/hailort_server/genai/inference_manager.cpp`
- [LLM inference](https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/llm/llm_inference_manager.cpp): `hailort/hailort_server/genai/llm/llm_inference_manager.cpp`
- [Speech to text](https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/speech2text/speech2text_server.cpp): `hailort/hailort_server/genai/speech2text/speech2text_server.cpp`
- [Vision-language](https://github.com/hailo-ai/hailort/blob/master/hailort/hailort_server/genai/vlm/vlm_server.cpp): `hailort/hailort_server/genai/vlm/vlm_server.cpp`

### Other components

- **Application user**

## Connections

- Application user → HailoRT CLI: invokes
- Application user → Python API: uses
- Application user → GStreamer HailoNet: uses
- Application user → C API: calls
- HailoRT CLI → Device API: controls device
- HailoRT CLI → Inference pipeline: runs inference
- Python API → C API: wraps
- GStreamer HailoNet → C API: uses runtime
- C API → HEF model: loads model
- C API → Infer model: configures inference
- Infer model → Network groups: configures (evidence: `hailort/libhailort/include/hailo/infer_model.hpp`)
- Pipeline builder → Inference pipeline: builds (evidence: `hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp`)
- Pipeline builder → Post-processing: adds operations (evidence: `hailort/libhailort/src/net_flow/pipeline/async_pipeline_builder.cpp`)
- Inference pipeline → Buffer storage: uses buffers
- Resource manager → HEF model: reads metadata (evidence: `hailort/libhailort/src/core_op/resource_manager/resource_manager.cpp`)
- Device API → HRPC transport: uses transport
- HRPC transport → Hailo device driver: communicates
- GenAI inference → Infer model: runs inference
- LLM inference → GenAI inference: extends
- Speech to text → GenAI inference: uses inference
- Vision-language → GenAI inference: uses inference
- Inference pipeline → Inference statistics: reports metrics
