# pytorch/executorch architecture

> Architecture diagram of the GitHub repository pytorch/executorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pytorch/executorch
- Repository: https://github.com/pytorch/executorch

## Overview

##### Project Type and Purpose
- ExecuTorch is a full-stack, on-device AI framework for both inference and training.
- It provides end-to-end pipelines: model export from PyTorch to a portable format, IR transformations, code generation, hardware-specific backends, and a lightweight runtime.
- It targets mobile and embedded platforms (iOS, Android, Linux, microcontrollers) with hardware acceleration (Apple CoreML/MPS, Arm Ethos-U, Cadence, MediaTek, NXP, OpenVINO, Qualcomm, Vulkan, XNNPACK).

##### High-Level File Structure
- Top-Level Directories
- `src/` and `executorch/`: Python front-end APIs, CLI tools, user examples, documentation.
- `exir/`: Intermediate representation (IR) library, passes, emitters.
- `codegen/`: Code generation from IR to C++ or platform code.
- `kernel/` & `kernels/`: Hand-written portable, optimized, quantized kernels.
- `backends/`: Hardware delegates and partitioners (one subfolder per vendor).
- `runtime/`: Core C++ runtime, executor, memory manager, platform abstraction.
- `extension/`: Bindings for Android, iOS (Swift/C++/JNI) and other extension modules.
- `examples/`: End-to-end demos per platform and per backend.
- `docs/`: Sphinx-based documentation, tutorials, architecture guides.
- `devtools/`, `tools/`, `profiler/`: Internal tooling for debugging, visualization, CI.
- `CMakeLists.txt`, `Package.swift`, `buckconfig`, `.github/`: Build and CI config (CMake, SwiftPM, Buck2, GitHub Actions).

##### Architecture from the README
- ExecuTorch wraps PyTorch models into a portable PTE (Portable Executorch) format.
- The PTE is then lowered into the ExecuTorch IR (`exir`), transformed by passes (e.g., quantization, delegate insertion).
- `codegen` emits optimized C++ or platform-specific code for kernels and executors.
- `kernel` libs provide CPU/edge kernels; `backends` supply delegate implementations.
- `runtime` executes the generated or delegated graph on device, with a hardware-abstraction layer.

a. Main Components
- Front-end & APIs (Python/Swift/Java modules in `src/`, `executorch/`, `extension/`)
- Export & IR (PTE exporter → `exir` pipeline → transformation passes)
- Code Generation (`codegen` produces C++/delegate code)
- Kernel Libraries (`kernels/portable`, `kernels/optimized`, `kernels/quantized`)
- Backend Delegates (`backends/<vendor>/...`, partitioners + runtime delegates)
- Runtime Core (`runtime/`: executor, memory manager, platform layer)
- Extensions & Examples (platform bindings: Android JNI, iOS Swift, WASM)
- DevTools & CI (benchmarking, profiler, test suites, Docker orchestrations)

b. Relationships & Data Flow
1. User script / PyTorch model
↓ export via Python CLI
2. PTE file
↓ parsed by `exir` IR front-end
3. IR passes (graph transforms, quantizer, partitioner)
↓
↓
5. `codegen` → generate C++ code or platform binaries
↓
6. Link against kernel libs and backend delegates
↓
7. Deployed to device → `runtime` executes graph
↓
8. Optional profiling / debugging via DevTools

c. Architectural Patterns & Principles
- Layered architecture: Front-end → IR → Codegen → Runtime.
- Plugin/delegate pattern: Each vendor backend implements a common interface.
- IR transformation pipeline: Reusable passes for quantization, fusion, lowering.
- Separation of concerns: Python vs C++ vs platform binding layers.
- Monorepo modularity: Each component in its own directory with tests.

d. Key Technologies & Frameworks
- Languages: Python (front-end), C++ (kernels & runtime), Swift/Objective-C (iOS), Java/Kotlin (Android), FlatBuffers.
- Build: CMake, SwiftPM (`Package.swift`), Buck2 (`.buckconfig`), Bazel integrations (`targets.bzl`).
- CI: GitHub Actions, Docker scripts (`.ci/docker`), conda environments.
- Documentation: Sphinx (`docs/`), Markdown tutorials.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Frontend & APIs"
        FE1["Python CLI & API\n(src/executorch/)"]:::python
        FE2["User Python API\n(executorch/)"]:::python
    end

    subgraph "Model Export & IR Front-end"
        EX1["PTE Exporter & Registry\n(export/)"]:::python
        EX2["IR Library & Passes\n(exir/)"]:::python
    end

    subgraph "IR Transformation & Partitioning"
        IRP1["Core IR Passes\n(exir/passes/)"]:::python
        IRP2["Backend IR\n(exir/backend/)"]:::python
        IRP3["Vendor Partitioners & Quantizers\n(backends/*/*/partition/, quantizer/)"]:::backend
    end

    subgraph "Code Generation"
        CG["Codegen Templates & Emitter\n(codegen/)"]:::cpp
    end

    subgraph "Kernel Libraries"
        KL1["Portable CPU Kernels\n(kernels/portable/)"]:::cpp
        KL2["Optimized CPU Kernels\n(kernels/optimized/)"]:::cpp
        KL3["Quantized Kernels\n(kernels/quantized/)"]:::cpp
    end

    subgraph "Backend Delegates"
        BD1["Apple CoreML Delegate\n(backends/apple/coreml/)"]:::backend
        BD2["Apple MPS Delegate\n(backends/apple/mps/)"]:::backend
        BD3["Arm Ethos-U & Vela\n(backends/arm/ethosu/, arm/runtime/)"]:::backend
        BD4["Qualcomm QNN Delegate\n(backends/qualcomm/partition/, qualcomm/runtime/)"]:::backend
        BD5["Vulkan Delegate\n(backends/vulkan/partitioner/, vulkan/runtime/)"]:::backend
        BD6["XNNPack Delegate\n(backends/xnnpack/partition/, xnnpack/runtime/)"]:::backend
        BDx["Other Vendors\n(backends/mediatek/, nxp/, openvino/, cadence/, cortex_m/)"]:::backend
    end

    subgraph "Runtime Core & Executor"
        RT1["Core Libraries & Abstraction\n(runtime/core/, platform/)"]:::cpp
        RT2["Executor & Memory Manager\n(runtime/executor/, memory_manager.h)"]:::cpp
        RT3["Kernel Registry\n(runtime/kernel/, operator_registry.h)"]:::cpp
    end

    subgraph "Extensions & Mobile Bindings"
        EXB1["Android JNI & Java API\n(extension/android/)"]:::ext
        EXB2["iOS Swift/C++ Bindings\n(extension/apple/)"]:::ext
        EXB3["Python C++ Bindings\n(extension/pybindings/)"]:::ext
    end

    subgraph "Examples & Demos"
        EXM["Examples & Demos\n(examples/)"]:::support
    end

    subgraph "DevTools & CI"
        DT1["DevTools\n(devtools/)"]:::support
        DT2["CI & Tools\n(tools/, .ci/)"]:::support
        DT3["Profiler\n(profiler/)"]:::support
    end

    FE1 --> EX1
    FE2 --> EX1
    EX1 --> EX2
    EX2 --> IRP1
    IRP1 --> IRP2
    IRP2 --> IRP3
    IRP3 --> CG
    CG --> KL1
    CG --> KL2
    CG --> KL3
    KL1 --> BD1
    KL2 --> BD2
    KL3 --> BD3
    CG --> BD4
    CG --> BD5
    CG --> BD6
    CG --> BDx
    BD1 --> RT1
    BD2 --> RT1
    BD3 --> RT1
    BD4 --> RT1
    BD5 --> RT1
    BD6 --> RT1
    BDx --> RT1
    RT1 --> RT2
    RT2 --> RT3
    RT3 --> EXB1
    RT3 --> EXB2
    RT3 --> EXB3
    RT3 --> EXM
    RT3 --> DT1
    RT3 --> DT2
    RT3 --> DT3

    click FE1 "https://github.com/pytorch/executorch/tree/main/src/executorch/"
    click FE2 "https://github.com/pytorch/executorch/tree/main/executorch/"
    click EX1 "https://github.com/pytorch/executorch/tree/main/export/"
    click EX2 "https://github.com/pytorch/executorch/tree/main/exir/"
    click IRP1 "https://github.com/pytorch/executorch/tree/main/exir/passes/"
    click IRP2 "https://github.com/pytorch/executorch/tree/main/exir/backend/"
    click IRP3 "https://github.com/pytorch/executorch/tree/main/backends/"
    click CG "https://github.com/pytorch/executorch/tree/main/codegen/"
    click KL1 "https://github.com/pytorch/executorch/tree/main/kernels/portable/"
    click KL2 "https://github.com/pytorch/executorch/tree/main/kernels/optimized/"
    click KL3 "https://github.com/pytorch/executorch/tree/main/kernels/quantized/"
    click BD1 "https://github.com/pytorch/executorch/tree/main/backends/apple/coreml/"
    click BD2 "https://github.com/pytorch/executorch/tree/main/backends/apple/mps/"
    click BD3 "https://github.com/pytorch/executorch/tree/main/backends/arm/ethosu/"
    click BD4 "https://github.com/pytorch/executorch/tree/main/backends/qualcomm/partition/"
    click BD5 "https://github.com/pytorch/executorch/tree/main/backends/vulkan/partitioner/"
    click BD6 "https://github.com/pytorch/executorch/tree/main/backends/xnnpack/partition/"
    click BDx "https://github.com/pytorch/executorch/tree/main/backends/mediatek/"
    click RT1 "https://github.com/pytorch/executorch/tree/main/runtime/core/"
    click RT2 "https://github.com/pytorch/executorch/blob/main/runtime/executor/memory_manager.h"
    click RT3 "https://github.com/pytorch/executorch/blob/main/runtime/kernel/operator_registry.h"
    click EXB1 "https://github.com/pytorch/executorch/tree/main/extension/android/"
    click EXB2 "https://github.com/pytorch/executorch/tree/main/extension/apple/"
    click EXB3 "https://github.com/pytorch/executorch/tree/main/extension/pybindings/"
    click EXM "https://github.com/pytorch/executorch/tree/main/examples/"
    click DT1 "https://github.com/pytorch/executorch/tree/main/devtools/"
    click DT2 "https://github.com/pytorch/executorch/tree/main/tools/"
    click DT3 "https://github.com/pytorch/executorch/tree/main/profiler/"

    classDef python fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef cpp fill:#d4edda,stroke:#333,stroke-width:1px
    classDef backend fill:#ffe5b4,stroke:#333,stroke-width:1px
    classDef ext fill:#e2e0f7,stroke:#333,stroke-width:1px
    classDef support fill:#f0f0f0,stroke:#333,stroke-width:1px
```
