# texasinstruments/edgeai-tidl-tools architecture

> Architecture diagram of the GitHub repository texasinstruments/edgeai-tidl-tools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/texasinstruments/edgeai-tidl-tools
- Repository: https://github.com/texasinstruments/edgeai-tidl-tools

## Overview

##### Project Type and Purpose
- This is an open-source toolchain (“Edge AI TIDL Tools”) for compiling and running DNNs on TI embedded devices. It provides:
- A model‐compilation pipeline (Python & C++ APIs)
- Integration with three “Open Source Runtimes” (TFLite, ONNX Runtime, TVM/DLR)
- Examples, scripts, test data, and Docker support for both X86_PC host and TI SOC target

##### File‐Structure Analysis
- Top‐level directories
- examples/ – CMake–based C++ examples (examples/osrt_cpp), Python APIs (examples/osrt_python), Jupyter notebooks, model prototxts
- osrt-model-tools/ – Python package for ONNX and TFLite model optimization (src/ops, utilities, setup.py)
- scripts/ – Docker build/run, orchestration scripts to compile and run examples, debug tools
- test_data/ – golden outputs for validation
- setup.sh, setup_env.sh – environment-variable setup scripts
- Architectural patterns
- Layered: API layer → compilation layer → runtime layer → hardware abstraction
- Plugin architecture: OSRT runtimes load TIDL delegate to offload to DSP/MMA
- Build orchestration via CMake for C++ and pip/setup.py for Python

##### Insights from README
- Two main streams: Data‐scientist workflow (model compile on X86 → model artifacts) and Embedded‐engineer workflow (inference on X86_PC or TI SOC)
- Three supported runtimes with identical high‐level flow (compile → generate artifacts → load + infer)
- Setup paths: native or Docker for host, CMake build for C++ examples, Python APIs for scripting

##### Components
a. Main Components
- Host Environment (X86_PC)
– Setup Scripts (setup.sh, Dockerfiles)
– Python API & osrt_model_tools (model optimizer)
– C++ Examples / CMake build system
– Runtimes (TFLite, ONNX Runtime, TVM/DLR)
– TIDL Delegate / DSP Compiler
– Model Artifacts, Output Binaries, Output Images (file stores)
– Orchestration Scripts (run_python_examples.sh, run_compile_model_zoo.sh)
- Target Environment (TI SOC)
– Deployed Model Artifacts
– Python & C++ Inference Apps
– TIDL Runtime Libraries (lib files)
– On-device hardware: Cortex-A cores, C7x DSP, MMA accelerator
- External Services
– Docker Engine
– pip / system package manager
– Git (versioning)

b. Relationships & Data Flow
1. User invokes “compile” via Python CLI or C++ example →
2. osrt_model_tools / C++ compiler loads ONNX/TFLite model →
4. Inference layer loads artifacts into chosen OSRT runtime →
5. Runtime schedules layers: unsupported layers run on ARM (via CPU kernels), supported layers offloaded to C7x/MMA through TIDL delegate →
6. Outputs written back to file store → comparison/validation (scripts, test_data)

c. Architectural Patterns & Principles
- Layer separation: clear API vs compile vs runtime vs hardware abstraction
- Plugin/Delegate pattern for runtime extensibility
- Scripted orchestration for reproducible builds & benchmarks
- Modular Python package for model optimizations

d. Key Technologies & Frameworks
- CMake, GCC/Clang (C++ build)
- Python 3.10, pip, setuptools (osrt_model_tools)
- Docker (optional host setup)
- ONNX Runtime, TensorFlow Lite, TVM/DLR
- TI’s TIDL libraries (DSP compiler & runtime)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    subgraph "External Services"
        Docker("Docker Engine"):::cloud
        pip("pip/System Packages"):::cloud
        Git("Git"):::cloud
    end

    %% Host Environment
    subgraph "Host Environment" 
        subgraph "Setup Scripts"
            SetupSh["setup.sh"]:::host
            SetupEnv["setup_env.sh"]:::host
        end
        subgraph "Docker Support"
            DF1["Dockerfile"]:::host
            DF2["Dockerfile_GPU"]:::host
            BuildDock["build_docker.sh"]:::host
            RunDock["run_docker.sh"]:::host
            SetupDock["setup_docker.sh"]:::host
        end
        subgraph "Python API & Model Tools"
            SetupPy["setup.py"]:::host
            InitPy["__init__.py"]:::host
            OnnxTools["onnx_tools/…"]:::host
            TfliteTools["tflite_tools/…"]:::host
            Optimizer["tidl_onnx_model_optimizer"]:::host
        end
        subgraph "C++ Examples & Build"
            CMake["CMakeLists.txt"]:::host
            ONNXCPP["onnx_main.cpp/.h"]:::host
            TFLCPP["tfl_main.cpp/.h"]:::host
            DLRCpp["dlr_main.cpp/.h"]:::host
            PreProc["pre_process/…"]:::host
            PostProc["post_process/…"]:::host
        end
        subgraph "Python Examples"
            ONNXPY["onnxrt_ep.py"]:::host
            TFLPY["tflrt_delegate.py"]:::host
            TVMPY["tvm_dlr/*.py"]:::host
            UtilsPY["common_utils.py"]:::host
            ConfigPY["config_utils/…"]:::host
        end
        subgraph "Advanced Examples"
            AdvPy["advanced_examples/…"]:::host
            AdvCpp["advanced_examples/…"]:::host
        end
        subgraph "Orchestration Scripts"
            CompZoo["run_compile_model_zoo.sh"]:::host
            RunPyEx["run_python_examples.sh"]:::host
            RunONNX["run_onnx_models.sh"]:::host
            RunTFL["run_tfl_models.sh"]:::host
            RunDLR["run_dlr_models.sh"]:::host
        end
        Docs["docs/…"]:::host
    end

    %% Data Stores
    Artifacts["Model Artifacts"]:::store
    TestData["Test Data"]:::store
    Outputs["Output Binaries"]:::store

    %% Target Environment
    subgraph "Target Environment"
        DeployedArt["Deployed Artifacts"]:::store
        subgraph "Inference Apps"
            InferencePy["Python Inference App"]:::target
            InferenceCpp["C++ Inference App"]:::target
        end
        TIDLLibs["TIDL Runtime Libraries"]:::target
        subgraph "Hardware"
            CPU["Cortex-A CPU"]:::hardware
            DSP["C7x DSP"]:::hardware
            MMA["MMA Accelerator"]:::hardware
        end
    end

    %% Flows
    Docker --> DF1 & DF2 & BuildDock & RunDock & SetupDock
    pip --> SetupSh & SetupEnv & SetupPy
    Git --> Docs
    SetupSh & SetupEnv --> SetupPy & CMake
    BuildDock & SetupDock --> DF1
    SetupPy --> InitPy & OnnxTools & TfliteTools & Optimizer
    CMake --> ONNXCPP & TFLCPP & DLRCpp
    ONNXCPP & TFLCPP & DLRCpp --> Artifacts
    InitPy & OnnxTools & TfliteTools & Optimizer --> Artifacts
    CompZoo & RunPyEx --> Artifacts
    Artifacts --> DeployedArt
    DeployedArt --> InferencePy & InferenceCpp
    InferencePy & InferenceCpp --> TIDLLibs
    TIDLLibs --> DSP & MMA
    Outputs --> TestData

    %% Click Events
    click SetupSh "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/setup.sh"
    click SetupEnv "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/setup_env.sh"
    click DF1 "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/scripts/docker/Dockerfile"
    click DF2 "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/scripts/docker/Dockerfile_GPU"
    click BuildDock "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/docker/build_docker.sh"
    click RunDock "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/docker/run_docker.sh"
    click SetupDock "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/docker/setup_docker.sh"
    click SetupPy "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/osrt-model-tools/setup.py"
    click InitPy "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/osrt-model-tools/osrt_model_tools/__init__.py"
    click OnnxTools "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/osrt-model-tools/osrt_model_tools/onnx_tools/"
    click TfliteTools "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/osrt-model-tools/osrt_model_tools/tflite_tools/"
    click Optimizer "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/osrt-model-tools/osrt_model_tools/onnx_tools/tidl_onnx_model_optimizer/"
    click CMake "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_cpp/CMakeLists.txt"
    click ONNXCPP "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_cpp/ort/onnx_main.cpp"
    click TFLCPP "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_cpp/tfl/tfl_main.cpp"
    click DLRCpp "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_cpp/dlr/dlr_main.cpp"
    click ONNXPY "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_python/ort/onnxrt_ep.py"
    click TFLPY "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_python/tfl/tflrt_delegate.py"
    click TVMPY "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/examples/osrt_python/tvm_dlr/"
    click UtilsPY "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/examples/osrt_python/common_utils.py"
    click ConfigPY "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/examples/osrt_python/config_utils/"
    click AdvPy "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/examples/osrt_python/advanced_examples/"
    click AdvCpp "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/examples/osrt_cpp/advanced_examples/"
    click CompZoo "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/run_compile_model_zoo.sh"
    click RunPyEx "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/run_python_examples.sh"
    click RunONNX "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/osrt_cpp_scripts/run_onnx_models.sh"
    click RunTFL "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/osrt_cpp_scripts/run_tfl_models.sh"
    click RunDLR "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/scripts/osrt_cpp_scripts/run_dlr_models.sh"
    click Docs "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/docs/"
    click TestData "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/test_data/"
    click Outputs "https://github.com/texasinstruments/edgeai-tidl-tools/tree/master/test_data/"
    click DeployedArt "https://github.com/texasinstruments/edgeai-tidl-tools/blob/master/update_target.sh"

    %% Styles
    classDef host fill:#dae8fc,stroke:#6c8ebf,color:#054d84;
    classDef target fill:#d5e8d4,stroke:#82b366,color:#2e7d32;
    classDef store fill:#f8cecc,stroke:#b85450,color:#a12614;
    classDef cloud fill:#fff2cc,stroke:#d6b656,color:#7f6000;
    classDef hardware fill:#e1d5e7,stroke:#9673a6,color:#603d91;
```
