# krai/kilt-mlperf architecture

> Architecture diagram of the GitHub repository krai/kilt-mlperf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/krai/kilt-mlperf
- Repository: https://github.com/krai/kilt-mlperf

## Overview

##### Project Type and Purpose
- This is an open-source C++ inference library and benchmarking framework (not a web app).
- Core purpose: provide a flexible, high-performance ML inference engine (KILT) with pluggable devices, models, data sources, configuration tools, and benchmarks for MLPerf.

##### File Structure
Top‐level directories:
– benchmarks/…      : benchmarking harnesses (network & standalone) for various ML tasks (BERT, classification, object-detection)
– config/…          : configuration abstractions and parsers (env, JSON, translation tables)
– devices/…         : device back ends (dummy, ONNX) implementing the device interface
– plugins/…         : optional algorithmic extensions (e.g., NMS-ABP)
Top‐level interfaces and core:
– iconfig.h, idatasource.h, idevice.h, imodel.h : abstract interfaces
– kilt.h / kilt_impl.h / kilt_utils.h           : core API and implementation
– affinity.h, tracing.h                         : cross-cutting utilities (thread affinity, instrumentation)

##### README
- Confirms it’s a C++ inference library for MLPerf Inference.
- MIT-licensed, intended for integration into larger toolchains.

##### Main Components & Their Relationships
A. Core API Layer
– kilt.h / kilt_impl.h: Public API entrypoints (initialize, load model, run inference)
– Utilities: affinity.h, tracing.h
B. Interfaces / Abstractions
– IDevice: device plug-in contract
– IModel: model loader/executor contract
– IDataSource: pre/post-processing contract
– IConfig: configuration loader contract
C. Config Subsystem
– config_tools (env, JSON, translation) → implements IConfig
– config/kilt_config.h: high-level library settings
D. Device Subsystem
– devices/dummy and devices/onnx → implementations of IDevice
– Each has device_config, translation tables, device.h
E. Plugin Subsystem
– plugins/nms-abp: implements inference post-processing outside core (implements IModel or IDevice hooks)
F. Benchmarking Harness
– benchmarks/…/standalone and benchmarks/…/network
– Harness code uses Core API, loads devices/models/config, runs timing loops, reports results

##### Architectural Patterns & Principles
- Interface-based design for extensibility (IDevice, IModel, IDataSource).
- Plugin architecture: dynamic or static registration of device back ends and algorithms.
- Separation of concerns: config, core, device, model, data, benchmarks split into their own subsystems.
- Dependency inversion: core depends on interfaces, concrete implementations bound at runtime.

– Notes: annotate critical interactions (e.g., “IDevice::executeInference()”)

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% Core API Layer
  subgraph "Core API Layer"
    KiltH["kilt.h"]:::core
    KiltImpl["kilt_impl.h"]:::core
    KiltUtils["kilt_utils.h"]:::core
    Affinity["affinity.h"]:::core
    Tracing["tracing.h"]:::core
  end

  %% Interfaces / Abstractions
  subgraph "Interfaces / Abstractions"
    IConfig["IConfig (iconfig.h)"]:::iface
    IDataSource["IDataSource (idatasource.h)"]:::iface
    IDevice["IDevice (idevice.h)"]:::iface
    IModel["IModel (imodel.h)"]:::iface
  end

  %% Config Subsystem
  subgraph "Config Subsystem"
    KiltConfig["kilt_config.h"]:::subsys
    ConfigTools["config_tools.h"]:::subsys
    EnvReader["env/read_config.h"]:::subsys
    JSONReader["json/read_config.h"]:::subsys
    TranslateK["kilt_translate.h"]:::subsys
    TranslateTools["translate_tools.h"]:::subsys
    TranslationTable["translation_table.h"]:::subsys
  end

  %% Device Subsystem
  subgraph "Device Subsystem"
    DummyDevice["device.h (dummy)"]:::subsys
    DummyCfg["config/device_config.h (dummy)"]:::subsys
    DummyTrans["device_translation_table.h"]:::subsys
    ONNXDevice["device.h (onnx)"]:::subsys
    ONNXCfg["config/device_config.h (onnx)"]:::subsys
  end

  %% Plugin Subsystem
  subgraph "Plugin Subsystem"
    NMS["nms_abp.h"]:::plugin
    NMSCfg["nms_abp_config.h"]:::plugin
    Bitcasts["bitcasts.h"]:::plugin
    FP16["fp16.h"]:::plugin
    Priors1["MV1_priors.bin"]:::plugin
    Priors2["R34_priors.bin"]:::plugin
    Priors3["retinanet_priors.bin"]:::plugin
  end

  %% Benchmark Harness
  subgraph "Benchmark Harness"
    Binned["binned_harness.cpp"]:::bench
    HarnessCPP["harness.cpp"]:::bench
    HarnessCfg["harness_config.h"]:::bench
  end

  %% Relationships
  Binned -->|"invokes"| KiltH
  Binned -->|"invokes"| KiltImpl
  Binned -->|"invokes"| KiltUtils
  Binned -->|"uses"| IDevice
  Binned -->|"uses"| IModel
  Binned -->|"uses"| IDataSource
  Binned -->|"loads"| IConfig

  KiltH -->|"calls"| IDevice
  KiltH -->|"calls"| IModel
  KiltH -->|"calls"| IDataSource
  KiltH -->|"calls"| IConfig
  KiltH -->|"loads"| KiltConfig

  IDevice -.->|"implements"| DummyDevice
  IDevice -.->|"implements"| ONNXDevice

  IConfig -.->|"implements"| ConfigTools
  IConfig -.->|"implements"| EnvReader
  IConfig -.->|"implements"| JSONReader
  IConfig -.->|"implements"| TranslateK
  IConfig -.->|"implements"| TranslateTools
  IConfig -.->|"implements"| TranslationTable

  NMS -.->|"post-process hook"| IModel
  NMS -.->|"post-process hook"| IDevice

  %% Click Events
  click KiltH "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/kilt.h"
  click KiltImpl "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/kilt_impl.h"
  click KiltUtils "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/kilt_utils.h"
  click Affinity "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/affinity.h"
  click Tracing "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/tracing.h"
  click IConfig "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/iconfig.h"
  click IDataSource "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/idatasource.h"
  click IDevice "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/idevice.h"
  click IModel "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/imodel.h"
  click KiltConfig "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/kilt_config.h"
  click ConfigTools "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/config_tools/config_tools.h"
  click EnvReader "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/config_tools/env/read_config.h"
  click JSONReader "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/config_tools/json/read_config.h"
  click TranslateK "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/translate/kilt_translate.h"
  click TranslateTools "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/translate/translate_tools.h"
  click TranslationTable "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/config/translate/x/translation_table.h"
  click DummyDevice "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/devices/dummy/device.h"
  click DummyCfg "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/devices/dummy/config/device_config.h"
  click DummyTrans "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/devices/dummy/device_translation_table.h"
  click ONNXDevice "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/devices/onnx/device.h"
  click ONNXCfg "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/devices/onnx/config/device_config.h"
  click NMS "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/nms_abp.h"
  click NMSCfg "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/nms_abp_config.h"
  click Bitcasts "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/bitcasts.h"
  click FP16 "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/fp16.h"
  click Priors1 "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/data/MV1_priors.bin"
  click Priors2 "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/data/R34_priors.bin"
  click Priors3 "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/plugins/nms-abp/data/retinanet_priors.bin"
  click Binned "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/benchmarks/harness/binned_harness.cpp"
  click HarnessCPP "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/benchmarks/harness/harness.cpp"
  click HarnessCfg "https://github.com/krai/kilt-mlperf/blob/mlperf_4.1/benchmarks/harness/config/harness_config.h"

  %% Styles
  classDef core fill:#D0E8FF,stroke:#0366d6,color:#0366d6;
  classDef iface fill:#E8F1FF,stroke:#1A73E8,color:#1A73E8;
  classDef subsys fill:#E6F4EA,stroke:#1A7F37,color:#1A7F37;
  classDef plugin fill:#FFF4E5,stroke:#D19A66,color:#D19A66;
  classDef bench fill:#FFF0F1,stroke:#D14836,color:#D14836;
```
