How xilinx/xrt works
1. What this repo is
- This is AMD’s XRT runtime stack: a host-side software platform that lets applications manage NPUs/FPGAs and program them with xclbin images.
- The repo is split into user-space runtime libraries, kernel/driver components, device-emulation backends, and developer tools/tests.
2. Top-level execution flow
- An application enters through XRT/OpenCL/HIP-style APIs.
- Those calls flow into the runtime core, which resolves platform/device state, loads xclbin metadata, manages buffers/queues/context, and then dispatches to a backend-specific device/shim layer.
- The backend talks to the actual kernel driver on Linux/Windows, or to emulation components in HW/SW emu modes.
3. Public API surface
- src/runtime_src/core/include and src/include contain the exported headers: classic xrt.h, xclhal2.h, OpenCL compatibility headers, experimental xrt/* APIs, and AIE-related interfaces.
- These headers define the stable boundary for application developers, while the implementation lives deeper under core, xocl, xrt, and hip.
4. Core runtime engine
- src/runtime_src/core/common is the shared runtime backbone: device/context/kernel/buffer abstractions, xclbin parsing, error handling, tracing, scheduling, profiling, system info, and utility code.
- It also contains reusable infrastructure for async queues, module loading, config parsing, archive handling, and telemetry.
- This layer is the main place where device-agnostic runtime behavior is implemented.
5. Backend split by platform and transport
- src/runtime_src/core/pcie, src/runtime_src/core/edge, and src/runtime_src/core/xrt represent platform-specific adaptations.
- PCIe paths cover Alveo and similar cards; edge paths cover Zynq/Versal-style systems with tighter PS/PL integration and AIE/firmware-specific flows; xrt/device provides lower-level HAL abstractions.
- The runtime chooses the right backend depending on host OS, device family, and execution mode.
6. Driver boundary and kernel-space components
- Linux drivers live under src/runtime_src/core/pcie/driver/linux/xocl and edge DRM/ZOCL code under src/runtime_src/core/edge/drm/zocl.
- These drivers expose ioctl/sysfs/DRM entry points, manage buffer objects, command submission, hardware context state, interrupts, mailbox communication, and device-specific subsystems like PCIe, memory, clocks, and flash.
- Windows support is present under src/runtime_src/core/pcie/driver/windows and src/runtime_src/core/pcie/windows/alveo, matching the same logical driver boundary in a different OS model.
7. Emulation and simulation support
- src/runtime_src/core/pcie/emulation, src/runtime_src/core/edge/hw_emu, and src/runtime_src/core/edge/sw_emu provide hardware-emulation and software-emulation shims.
- These backends mimic device, scheduler, memory, and mailbox behavior so the same host APIs can be exercised without real hardware.
- This is a key architectural boundary: the host runtime code stays mostly the same, while the backend is swapped for emulation.
8. OpenCL compatibility layer
- src/runtime_src/xocl implements the OpenCL-facing runtime: API entry points (clCreateBuffer, clEnqueue*, etc.), core OpenCL objects, and ICD integration.
- It acts as a compatibility layer over XRT primitives so existing OpenCL-style applications can use the same underlying device/runtime stack.
- The icd and khronos subtrees show that this is a wrapper layer rather than the fundamental runtime.
9. HIP integration
- src/runtime_src/hip provides HIP-facing APIs and core objects for device, memory, stream, event, graph, and module handling.
- Architecturally, this is another frontend over the shared runtime/device machinery, aimed at a different application ecosystem.
- It is not a separate backend; it reuses the same device/runtime abstractions below it.
10. AIE and edge-specific control paths
- The edge tree contains extensive AIE support: graph control, profiling, parser code, PS kernels, and plugins for AIE/trace/debug.
- These pieces sit alongside PS/PL control and are important on Versal/Zynq-style platforms where the runtime must manage both programmable logic and AI Engine subsystems.
- This is a distinct subsystem from the generic PCIe buffer/command path.
11. xclbin and metadata tooling
- src/runtime_src/tools/xclbinutil is the canonical xclbin packing/unpacking utility.
- It handles sections for bitstreams, memory topology, connectivity, debug IP, AIE metadata, signatures, and vendor/user metadata.
- This tool feeds the runtime because the runtime and drivers rely on xclbin section data to understand what a device image contains and how to configure it.
12. Operational tools and diagnostics
- src/runtime_src/core/tools contains the user-facing operational utilities: xbutil2, xbmgmt2, xbflash2, xbtop, xbtracer, and supporting libraries.
- These tools inspect health, telemetry, firmware status, PCIe info, memory, clocks, AIE state, and can also perform configuration or flashing tasks.
- They sit on top of the same runtime/driver APIs used by applications, but are focused on administration and observability.
13. Python bindings and scripting
- src/python/pybind11 exposes a Python binding (pyxrt) for selected runtime APIs.
- src/runtime_src/tools/scripts contains packaging, environment setup, platform build helpers, and release-oriented scripts.
- These are auxiliary surfaces around the core runtime, not separate execution paths.
14. Tests and validation
- tests/unit_test, tests/xrt, tests/hip, tests/python, tests/pyopencl, and tests/mcdm cover API correctness, device behavior, emulation, and platform-specific features.
- The tests mirror the main architectural layers: API-level validation, backend/runtime behavior, driver interaction, and tooling correctness.
- This helps confirm that the same abstractions work across hardware, emulation, Linux, and Windows.
15. Build, packaging, and CI
- The repo is CMake-driven, with many platform-specific build scripts under build/ and CI workflows under .github/workflows.
- Packaging targets include Linux packages, Windows builds, embedded/Yocto flows, DKMS driver packaging, and cloud/container variants.
- This build structure reflects the architecture: one codebase producing multiple runtime/driver/tool combinations for different host OSes and device families.
Main entry points: tests Tests · common Core Common · device HAL Device · api Core API
Components 20
Public APIs
Headers and frontend API surfaces used by applications.
- XRT Headers · Stable exported runtime headers for XRT, OpenCL compatibility, experimental APIs, and AIE interfaces.
src/runtime_src/core/include - Extra Headers · Additional public headers and compatibility includes outside the core include tree.
src/include
Runtime Core
Shared device-agnostic runtime and backend selection.
- Core API · Implementation of the device, buffer, queue, context, kernel, module, and xclbin-facing XRT API objects.
src/runtime_src/core/common/api - Core Common · Shared runtime infrastructure: parsing, tracing, scheduling, profiling, config, utilities, and system info.
src/runtime_src/core/common - HAL Device · Lower-level device and HAL abstractions that bridge core runtime logic to platform implementations.
src/runtime_src/xrt/device
Platform Backends
PCIe, edge, emulation, and OS-specific device shims.
- PCIe Common · Device and system abstractions for PCIe-attached platforms.
src/runtime_src/core/pcie/common - Linux Backend · Linux PCIe runtime shim and driver-facing host-side implementation.
src/runtime_src/core/pcie/linux - Windows Backend · Windows PCIe runtime implementation for Alveo-class devices.
src/runtime_src/core/pcie/windows/alveo - Edge Common · Edge/embedded runtime abstractions and edge-specific device/system glue.
src/runtime_src/core/edge/common - Edge User · User-space edge runtime, device management, AIE objects, and plugin hooks.
src/runtime_src/core/edge/user - ZOCL Driver · Kernel-side DRM/ZOCL implementation for edge devices, including ioctl, buffer, KDS, mailbox, and AIE paths.
src/runtime_src/core/edge/drm/zocl - Edge Emu · Hardware/software emulation shims for edge platforms.
src/runtime_src/core/edge/hw_emu - PCIe Emu · Hardware/software emulation backends for PCIe-style platforms.
src/runtime_src/core/pcie/emulation
Drivers & Tools
Kernel-space drivers, operational tools, and utilities.
- OpenCL Layer · OpenCL-facing API entry points, core OpenCL objects, ICD integration, and XRT compatibility glue.
src/runtime_src/xocl - HIP Layer · HIP-facing API and core objects built on the shared runtime machinery.
src/runtime_src/hip - xclbinutil · xclbin pack/unpack utility for sections, metadata, signatures, and platform-specific image data.
src/runtime_src/tools/xclbinutil - Ops Tools · xbutil2, xbmgmt2, xbflash2, xbtop, xbtracer, and related reporting/diagnostic utilities.
src/runtime_src/core/tools
Language & Validation
Language bindings, packaging scripts, and tests.
- Python Bindings · pyxrt Python bindings for selected runtime APIs.
src/python/pybind11 - Scripts · Environment setup, packaging, build helpers, and release/installation scripts.
src/runtime_src/tools/scripts - Tests · Repository-wide runtime, frontend, emulation, and tooling validation suites.
tests
Connections 27
- XRT Headers → Core API: declares · Public headers define the API surface implemented by the core runtime.
- Extra Headers → Core API: declares · Compatibility headers mirror the same runtime interface.
- Core API → Core Common: uses · API objects delegate to shared device-agnostic runtime services.
- Core Common → HAL Device: dispatches · Common runtime resolves to HAL/device abstractions.
- HAL Device → PCIe Common: targets · HAL abstractions bind to PCIe device/system implementations.
- HAL Device → Edge Common: targets · HAL abstractions also bind to edge device/system implementations.
- HAL Device → PCIe Emu: targets · HAL layer can swap in PCIe emulation backends.
- HAL Device → Edge Emu: targets · HAL layer can swap in edge emulation backends.
- PCIe Common → Linux Backend: implements · Linux backend provides the main PCIe runtime/driver path.
- PCIe Common → Windows Backend: implements · Windows backend provides the PCIe path on that OS.
- PCIe Common → PCIe Emu: pairs with · PCIe common logic is reused by emulation shims.
- Edge Common → Edge User: supports · Edge common logic feeds the user-space edge runtime.
- Edge Common → ZOCL Driver: supports · Edge common logic pairs with kernel-side ZOCL/DRM support.
- Edge Common → Edge Emu: supports · Edge common logic also feeds hardware/software emulation.
- OpenCL Layer → Core Common: wraps · OpenCL layer wraps shared XRT primitives.
- HIP Layer → Core Common: wraps · HIP layer reuses the same core runtime machinery.
- xclbinutil → Core Common: feeds · xclbin metadata and section handling informs runtime/device loading.
- Ops Tools → Core Common: queries · Operational tools inspect runtime state through shared APIs.
- Ops Tools → Linux Backend: controls · Administrative tools talk to deployed Linux drivers and devices.
- Ops Tools → ZOCL Driver: controls · Edge management tools interact with ZOCL and edge subsystems.
- Python Bindings → Core API: binds · Python binding exposes selected API entry points.
- Scripts → Tests: drives · Build and setup scripts support test and packaging workflows.
- Tests → OpenCL Layer: validates · Tests cover OpenCL compatibility behavior.
- Tests → HIP Layer: validates · Tests cover HIP-facing behavior.
- Tests → Core API: validates · Tests exercise the shared XRT API layer.
- Tests → PCIe Emu: validates · Tests cover emulation paths as well as hardware paths.
- Tests → Edge User: validates · Tests cover edge runtime and AIE-oriented flows.