# itlab-vision/dl-benchmark architecture

> Architecture diagram of the GitHub repository itlab-vision/dl-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/itlab-vision/dl-benchmark
- Repository: https://github.com/itlab-vision/dl-benchmark

## Overview

##### Main Components Identification

a) Core Benchmark Components:
- Accuracy Checker (src/accuracy_checker)
- Benchmark Engine (src/benchmark)
- Model Converters (src/model_converters)
- Quantization Tools (src/quantization)
- Remote Control System (src/remote_control)
- TVM Autotuning (src/tvm_autotuning)

b) Framework Support Layer:
- Multiple DL Framework Implementations (OpenVINO, TensorFlow, PyTorch, etc.)
- C++ and Python APIs
- Framework Wrappers and Processes

c) Infrastructure Components:
- Docker Containerization
- Deployment Tools
- Configuration Management
- Results Collection and Reporting

**┌─ User Interface Layer ─────────────────────┐**
- Config Maker GUI                         │
- Command Line Interface                   │
- ┬───────────────────────┘
- ↓
**┌─ Orchestration Layer ──────────────────────┐**
- Benchmark Engine                         │
- Remote Control                           │
- Deployment Manager                       │
- ┬───────────────────────┘
- ↓
**┌─ Framework Abstraction Layer ──────────────┐**
- Framework Wrappers:                        │
- OpenVINO  - TensorFlow  - PyTorch       │
- MXNet     - ONNX        - TVM           │
- Others                                   │
- ┬───────────────────────┘
- ↓
**┌─ Core Services Layer ────────────────────────┐**
- Accuracy Checker                           │
- Model Converters                           │
- Quantization Tools                         │
- Autotuning                                 │
- ┬───────────────────────┘
- ↓
**┌─ Infrastructure Layer ────────────────────────┐**
- Docker Containers                           │
- Results Storage                            │
- Configuration Management                    │
- ┘

##### Connection Types

Different types of connections:

##### Additional Elements

Include:
- Version information
- Supported hardware platforms (Intel CPUs, RISC-V, etc.)
- Framework version compatibility

##### Key Interactions to Highlight

- Model flow: conversion → quantization → benchmarking
- Results collection and reporting pipeline
- Remote execution paths
- Docker container interactions

##### Deployment View

- Local execution environment
- Remote execution capabilities
- Docker container organization
- Results collection and storage

Remember to include notes about:
- Supported model formats
- Hardware compatibility
- Framework version requirements
- Benchmark metrics collection points

This system design should help developers understand both the high-level architecture and the detailed interactions between components while maintaining clarity about the project's benchmarking purposes.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles
    classDef ui fill:#9370DB,stroke:#333,stroke-width:2px
    classDef orchestration fill:#4682B4,stroke:#333,stroke-width:2px
    classDef framework fill:#3CB371,stroke:#333,stroke-width:2px
    classDef core fill:#4169E1,stroke:#333,stroke-width:2px
    classDef infra fill:#FFA500,stroke:#333,stroke-width:2px

    %% User Interface Layer
    subgraph UI["User Interface Layer"]
        ConfigGUI["Config Maker GUI"]
        CLI["Command Line Interface"]
    end

    %% Orchestration Layer
    subgraph Orchestration["Orchestration Layer"]
        BenchEngine["Benchmark Engine"]
        RemoteCtrl["Remote Control"]
        DeployMgr["Deployment Manager"]
    end

    %% Framework Layer
    subgraph Frameworks["Framework Abstraction Layer"]
        FWrappers["Framework Wrappers"]
        OpenVINO["OpenVINO"]
        TensorFlow["TensorFlow"]
        PyTorch["PyTorch"]
        ONNX["ONNX"]
        TVM["TVM"]
    end

    %% Core Services Layer
    subgraph CoreServices["Core Services Layer"]
        AccChecker["Accuracy Checker"]
        ModelConv["Model Converters"]
        QuantTools["Quantization Tools"]
        Autotuning["TVM Autotuning"]
        InferenceEngine["Inference Layer"]
    end

    %% Infrastructure Layer
    subgraph Infrastructure["Infrastructure Layer"]
        Docker["Docker Containers"]
        Results["Results Storage"]
        Config["Configuration Management"]
    end

    %% Connections
    UI --> Orchestration
    Orchestration --> Frameworks
    Frameworks --> CoreServices
    CoreServices --> Infrastructure

    %% Internal Connections
    ConfigGUI --> CLI
    BenchEngine --> RemoteCtrl
    RemoteCtrl --> DeployMgr
    FWrappers --> OpenVINO & TensorFlow & PyTorch & ONNX & TVM
    AccChecker --> ModelConv
    ModelConv --> QuantTools
    QuantTools --> Autotuning
    Docker --> Results
    Results --> Config

    %% Click Events
    click AccChecker "https://github.com/itlab-vision/dl-benchmark/tree/master/src/accuracy_checker/"
    click BenchEngine "https://github.com/itlab-vision/dl-benchmark/tree/master/src/benchmark/"
    click ModelConv "https://github.com/itlab-vision/dl-benchmark/tree/master/src/model_converters/"
    click QuantTools "https://github.com/itlab-vision/dl-benchmark/tree/master/src/quantization/"
    click RemoteCtrl "https://github.com/itlab-vision/dl-benchmark/tree/master/src/remote_control/"
    click Autotuning "https://github.com/itlab-vision/dl-benchmark/tree/master/src/tvm_autotuning/"
    click FWrappers "https://github.com/itlab-vision/dl-benchmark/tree/master/src/benchmark/frameworks/"
    click Docker "https://github.com/itlab-vision/dl-benchmark/tree/master/docker/"
    click Results "https://github.com/itlab-vision/dl-benchmark/tree/master/results/"
    click Config "https://github.com/itlab-vision/dl-benchmark/tree/master/src/configs/"
    click ConfigGUI "https://github.com/itlab-vision/dl-benchmark/tree/master/src/config_maker/"
    click InferenceEngine "https://github.com/itlab-vision/dl-benchmark/tree/master/src/inference/"
    click DeployMgr "https://github.com/itlab-vision/dl-benchmark/tree/master/src/deployment/"

    %% Legend
    subgraph Legend
        UI_Legend["User Interface"]:::ui
        Orch_Legend["Orchestration"]:::orchestration
        Frame_Legend["Framework"]:::framework
        Core_Legend["Core Services"]:::core
        Infra_Legend["Infrastructure"]:::infra
    end

    %% Style Applications
    class ConfigGUI,CLI ui
    class BenchEngine,RemoteCtrl,DeployMgr orchestration
    class FWrappers,OpenVINO,TensorFlow,PyTorch,ONNX,TVM framework
    class AccChecker,ModelConv,QuantTools,Autotuning,InferenceEngine core
    class Docker,Results,Config infra
```
