# tensorflow/tflite-micro architecture

> Architecture diagram of the GitHub repository tensorflow/tflite-micro, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tensorflow/tflite-micro
- Repository: https://github.com/tensorflow/tflite-micro

## Overview

##### Main Components
- Runtime Core:
- The TFLM interpreter which handles model loading (typically via FlatBuffers), tensor management, and scheduling of micro kernels.
- Micro Mutable Op Resolver: A component that maps model operators to corresponding kernel implementations.
- Memory Planner and Allocator: Modules that handle the pre-allocation and management of memory for tensors in highly-constrained environments.
- Kernel Libraries:
- A collection of micro-kernel implementations (for operations like convolution, fully connected, pooling, etc.) optimized for various target architectures such as Cortex-M, Hexagon, RISC-V, and Xtensa.
- Domain-specific kernels for signal processing (the “signal” directory) and TensorFlow Lite kernel functions.
- Code Generation & Build Infrastructure:
- Code-generation utilities (e.g., in the “codegen” folder) that help produce source code wrappers or interface code.
- Build configuration files and scripts – Bazel BUILD files, WORKSPACE, and various CI scripts in the “.github/workflows” directory – that show how the code is built, tested, and integrated with third-party libraries.
- Platform/Hardware-Specific Layers:
- Directory partitions for specific hardware support (e.g., “tensorflow/lite/micro/ceva”, “tensorflow/lite/micro/xtensa”, “tensorflow/lite/micro/cortex_m”, etc.) that encapsulate adaptation to different ISAs and board-level integrations.
- Additional Tools & Integration:
- Python modules and support scripts (in “python” and “docs”) used for tests, model analysis, and integration (e.g., generating integration tests, model benchmarks, or conversions).
- Third-party libraries (e.g., FlatBuffers, Ruy, CMSIS, Hexagon, and others) that provide support infrastructure.

##### Representing Relationships and Data Flows
- Clearly differentiate the “runtime” side (the micro-interpreter, op resolver, memory planner, and kernel library) from the “build and integration” side (code generation, tests, CI pipelines, and integration with third-party dependencies).
- That the input ML model (stored as a FlatBuffer) is fed to the TensorFlow Lite Micro Interpreter, which in turn uses the Op Resolver to dispatch operations to specialized kernel functions.
- Build-time transformations (through the codegen components and BUILD scripts) that compile or optimize the system for specific hardware targets.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Distinguish between generic ML inference logic (handled by the micro-interpreter and op resolver) and hardware-specialized optimizations (target-specific kernel implementations).
- Modularity and Extensibility: Use of an extensible micro mutable op resolver allows for new operations to be added without major architectural changes.
- Portability: The organization supports multiple hardware targets by encapsulating platform-specific differences within dedicated directories and build configurations.
- Efficiency in Memory Management: Because microcontrollers typically have limited resources, the design emphasizes static or pre-allocated memory planning.

##### Notable Technologies and Frameworks
- Bazel as the primary build system (thus incorporating numerous BUILD files and the WORKSPACE file).
- Python for test harnesses, integration test generation, and potentially code-generation scripts.
- CI/CD integration via GitHub Actions, as shown in numerous workflow YAML files.
- Third-party integration with libraries like FlatBuffers for model representation, CMSIS or vendor-specific drivers/libs for optimized kernel operations, and additional tools like Ruy which may be used in quantization or low-level math support.

##### Additional Notes
- Do not try to display every file from the file tree; rather, focus on high-level modules and how they interact.
- Integration of hardware abstraction layers (with different directories for Hexagon, Xtensa, Cortex-M, etc.) and show that each is a specialized implementation of the same core interface.
- How the system supports unit testing and continuous integration, given the plethora of workflows and test scripts.

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Inference Engine (Runtime)
    subgraph "Inference Engine (Runtime)"
        Model["ML Model (FlatBuffer)"]:::runtime
        Interpreter["Micro Interpreter"]:::runtime
        OpResolver["Micro Mutable Op Resolver"]:::runtime
        Memory["Memory Planner & Allocator"]:::runtime
        subgraph "Operation Kernels"
            Kernels["General Kernels"]:::runtime
            SignalKernels["Signal Processing Kernels"]:::runtime
        end
    end

    %% Build & Integration Environment
    subgraph "Build & Integration"
        BuildSystem["Build System (Bazel, WORKSPACE, BUILD)"]:::build
        CodeGen["Code Generation"]:::build
        CI["CI/CD Pipelines"]:::build
        Testing["Testing/Validation"]:::build
    end

    %% Platform/Hardware-Specific Layers
    subgraph "Platform Specific"
        CortexM["Cortex-M Support"]:::platform
        CEVA["CEVA Integration"]:::platform
        Hexagon["Hexagon Support"]:::platform
        RISCV["RISC-V Support"]:::platform
        Xtensa["Xtensa Optimizations"]:::platform
    end

    %% Additional Tools & Integration
    subgraph "Additional Tools"
        Python["Python Modules & Test Harnesses"]:::tools
        Docs["Documentation & Integration"]:::tools
    end

    %% Data Flow Connections
    Model -->|"feed"| Interpreter
    Interpreter -->|"dispatches"| OpResolver
    Interpreter -->|"allocates"| Memory
    OpResolver -->|"selects"| Kernels
    Memory -->|"plans"| Kernels
    Kernels -->|"executes on"| CortexM
    Kernels -->|"executes on"| CEVA
    Kernels -->|"executes on"| Hexagon
    Kernels -->|"executes on"| RISCV
    Kernels -->|"executes on"| Xtensa

    %% Build Integration Flow
    CodeGen -->|"generates code for"| Interpreter
    BuildSystem -->|"compiles"| Interpreter
    CI -->|"triggers"| Testing
    Testing -->|"utilizes"| Python

    %% Additional Tools Connection
    Docs --- Python

    %% Click Events
    click Interpreter "https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/micro_interpreter.cc"
    click OpResolver "https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/micro_mutable_op_resolver.h"
    click Memory "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/memory_planner"
    click Kernels "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/kernels"
    click SignalKernels "https://github.com/tensorflow/tflite-micro/tree/main/signal/micro/kernels"
    click CodeGen "https://github.com/tensorflow/tflite-micro/tree/main/codegen"
    click BuildSystem "https://github.com/tensorflow/tflite-micro/tree/main/WORKSPACE"
    click CI "https://github.com/tensorflow/tflite-micro/tree/main/.github/workflows"
    click CortexM "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/cortex_m_corstone_300"
    click CEVA "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/ceva"
    click Hexagon "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/hexagon"
    click RISCV "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/riscv32_generic"
    click Xtensa "https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/kernels/xtensa"
    click Python "https://github.com/tensorflow/tflite-micro/tree/main/python/tflite_micro"
    click Docs "https://github.com/tensorflow/tflite-micro/tree/main/docs"

    %% Styles
    classDef runtime fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef build fill:#A9DFBF,stroke:#1E8449,stroke-width:2px;
    classDef platform fill:#F5B7B1,stroke:#943126,stroke-width:2px;
    classDef tools fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    
    class Model,Interpreter,OpResolver,Memory,Kernels,SignalKernels runtime;
    class BuildSystem,CodeGen,CI,Testing build;
    class CortexM,CEVA,Hexagon,RISCV,Xtensa platform;
    class Python,Docs tools;
```
