# llandsmeer/iotf architecture

> Architecture diagram of the GitHub repository llandsmeer/iotf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/llandsmeer/iotf
- Repository: https://github.com/llandsmeer/iotf

## Overview

**Top Layer: Model Definition**
**Middle Layer: Runner Framework**
**Bottom Layer: Hardware Execution Layer**

##### Component Details

a) Model Definition Layer:
- Subcomponents for:
- Initial neuron state generation
- ONNX model creation
- TensorFlow function creation
- 3D connection sampling
- Timestep handling

b) Runner Framework Layer (Core):
- Branch out to show the different runner implementations:
- ONNX Runners (CPU, CUDA, TensorRT)
- TensorFlow Runners (CPU, TPU)
- Hardware-Specific Runners (Graphcore, GroqChip)

c) Hardware Execution Layer:
- CPU
- CUDA GPUs
- TensorRT
- Graphcore IPUs
- Groq GroqChips
- TPUs

##### Data Flow Indicators

- Model compilation flow (Model → Runners)
- Execution paths (Runners → Hardware)
- Benchmark data collection paths
- Test validation flows

##### Detailed Elements

- The plugin architecture for different hardware backends
- The optimization variants (especially for GroqChip runners)
- The separation between base implementations and hardware-specific optimizations
- The benchmark data collection and reporting system

##### Notes Section

A notes section indicating:
- Hardware requirements
- Optional components (based on available hardware)
- Key performance considerations
- Testing requirements

- Hierarchical component representation
- Clear inheritance and dependency visualization
- Export to high-resolution formats

Remember to emphasize the benchmark nature of the project and how it enables performance comparison across different AI acceleration platforms for neural simulation workloads.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Style Definitions
    classDef model fill:#b3d9ff,stroke:#333,stroke-width:2px
    classDef runner fill:#90EE90,stroke:#333,stroke-width:2px
    classDef hardware fill:#ffd9b3,stroke:#333,stroke-width:2px
    classDef benchmark fill:#d9b3ff,stroke:#333,stroke-width:2px
    classDef core fill:#f2f2f2,stroke:#333,stroke-width:2px

    %% Model Definition Layer
    subgraph ModelLayer["Model Definition Layer"]
        Model["Inferior Olive Neuron Model"]
        InitState["Initial Neuron State Generator"]
        ONNXModel["ONNX Model Creator"]
        TFFunc["TensorFlow Function Creator"]
        Conn3D["3D Connection Sampler"]
        TimeStep["Timestep Handler"]
    end

    %% Runner Framework Layer
    subgraph RunnerLayer["Runner Framework Layer"]
        BaseRunner["Base Runner"]
        ONNXBase["ONNX Base Runner"]
        TFBase["TensorFlow Base Runner"]
        
        %% ONNX Runners
        ONNXCPU["ONNX CPU Runner"]
        ONNXCUDA["ONNX CUDA Runner"]
        ONNXTRT["ONNX TensorRT Runner"]
        
        %% TF Runners
        TFTPU["TensorFlow TPU Runner"]
        
        %% Hardware-Specific Runners
        GraphCore["Graphcore Runner"]
        GraphCoreStream["Graphcore Streaming"]
        GroqChip["GroqChip Runner"]
        GroqOpt1["GroqChip Opt1"]
        GroqOpt2["GroqChip Opt2 NoCopy"]
    end

    %% Hardware Execution Layer
    subgraph HardwareLayer["Hardware Execution Layer"]
        CPU["CPU"]
        CUDA["CUDA GPUs"]
        TRT["TensorRT"]
        IPU["Graphcore IPUs"]
        GROQ["Groq GroqChips"]
        TPU["TPUs"]
    end

    %% Benchmark Components
    subgraph BenchLayer["Benchmark Components"]
        BenchFrame["Benchmark Framework"]
        EnergyMeas["Energy Measurement"]
        ModelConfig["Model Configuration"]
        TestInfra["Testing Infrastructure"]
    end

    %% Relationships
    Model --> InitState
    Model --> ONNXModel
    Model --> TFFunc
    Model --> Conn3D
    Model --> TimeStep

    BaseRunner --> ONNXBase
    BaseRunner --> TFBase

    ONNXBase --> ONNXCPU
    ONNXBase --> ONNXCUDA
    ONNXBase --> ONNXTRT

    TFBase --> TFTPU

    BaseRunner --> GraphCore
    BaseRunner --> GroqChip
    GraphCore --> GraphCoreStream
    GroqChip --> GroqOpt1
    GroqChip --> GroqOpt2

    ONNXCPU --> CPU
    ONNXCUDA --> CUDA
    ONNXTRT --> TRT
    GraphCore --> IPU
    GroqChip --> GROQ
    TFTPU --> TPU

    %% Click Events
    click InitState "https://github.com/llandsmeer/iotf/blob/main/ioperf/model/make_initial_neuron_state.py"
    click ONNXModel "https://github.com/llandsmeer/iotf/blob/main/ioperf/model/make_onnx_model.py"
    click TFFunc "https://github.com/llandsmeer/iotf/blob/main/ioperf/model/make_tf_function.py"
    click Conn3D "https://github.com/llandsmeer/iotf/blob/main/ioperf/model/sample_connections_3d.py"
    click TimeStep "https://github.com/llandsmeer/iotf/blob/main/ioperf/model/timestep.py"
    click BaseRunner "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/base_runner.py"
    click ONNXBase "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/onnx_base_runner.py"
    click ONNXCPU "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/onnx_cpu_runner.py"
    click ONNXCUDA "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/onnx_cuda_runner.py"
    click ONNXTRT "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/onnx_tensorrt_runner.py"
    click TFBase "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/tf_base_runner.py"
    click TFTPU "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/tf_tpu_runner.py"
    click GraphCore "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/graphcore_runner.py"
    click GroqChip "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/groqchip_runner.py"
    click BenchFrame "https://github.com/llandsmeer/iotf/blob/main/ioperf/bench/benchmark.py"
    click EnergyMeas "https://github.com/llandsmeer/iotf/blob/main/ioperf/bench/energy_bench.py"
    click ModelConfig "https://github.com/llandsmeer/iotf/blob/main/ioperf/bench/model_configuration.py"
    click TestInfra "https://github.com/llandsmeer/iotf/tree/main/ioperf/bench/tests"
    click GroqOpt1 "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/groqchip_runner_opt1.py"
    click GroqOpt2 "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/groqchip_runner_opt2_nocopy.py"
    click GraphCoreStream "https://github.com/llandsmeer/iotf/blob/main/ioperf/runners/graphcore_streaming_runner.py"

    %% Style Applications
    class Model,InitState,ONNXModel,TFFunc,Conn3D,TimeStep model
    class BaseRunner,ONNXBase,TFBase,ONNXCPU,ONNXCUDA,ONNXTRT,TFTPU,GraphCore,GraphCoreStream,GroqChip,GroqOpt1,GroqOpt2 runner
    class CPU,CUDA,TRT,IPU,GROQ,TPU hardware
    class BenchFrame,EnergyMeas,ModelConfig,TestInfra benchmark
```
