# google-ai-edge/ai-edge-torch architecture

> Architecture diagram of the GitHub repository google-ai-edge/ai-edge-torch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/google-ai-edge/ai-edge-torch
- Repository: https://github.com/google-ai-edge/ai-edge-torch

## Overview

##### Project type and purpose
- This is an open-source Python library (“AI Edge Torch”) that converts PyTorch models into TensorFlow Lite flatbuffers, and provides a native Generative API for authoring and optimizing transformer-based LLMs for on-device inference.

##### File structure
- Top-level folders:
- build and CI configs (Bazel, GitHub workflows, WORKSPACE)
- docs (pytorch_converter walkthroughs, generative doc)
- ai_edge_torch/ – the main Python package, subdivided into:
– _convert/    : core conversion pipeline, FX-based graph transforms
– generative/ : transformer layer implementations, quantization recipes, example conversion tools
– odml_torch/  : StableHLO/MLIR builders, TF-integration, JAX bridge
– quantize/    : quantization utilities, pt2e quantizer
– debug/, lowertools/, hlfb/ : helper and fallback modules
– testing/     : test harness, model coverage
– examples/    : Python and C++ usage guides, end-to-end tests
- Recognize plugin-style FX passes, builder pattern, clear separation of API vs. transformation vs. backends.

##### README for additional insights
- Two main features:
- PyTorch Converter (Beta): uses torch.export() → FX passes → MLIR/StableHLO → TFLite exporter
- Generative API (Alpha): implements efficient transformer layers, supports Lora, KV cache, quant recipes, E2E examples
- External integrations: PyTorch, tf-nightly/TensorFlow Lite, MediaPipe LLM Inference API.

## Diagram (Mermaid)

```mermaid
graph TB
    %% User API Layer
    subgraph "User API Layer"
        style UserAPI fill:#BBDEFB,stroke:#1E88E5,stroke-width:2px
        UserAPI["ai_edge_torch.__init__.py\nai_edge_torch.model.py\nai_edge_torch._config.py"]:::api
    end

    %% Core Conversion Pipeline
    subgraph "Core Conversion Pipeline"
        style CoreConv fill:#C8E6C9,stroke:#43A047,stroke-width:2px
        Conv["conversion.py"]:::core
        Converter["converter.py"]:::core
        ConvUtils["conversion_utils.py"]:::core
        Signature["signature.py"]:::core
    end

    %% FX Pass Manager
    subgraph "FX Pass Manager"
        style FXPass fill:#C8E6C9,stroke:#43A047,stroke-width:2px
        BuildAten["build_aten_composite_pass.py"]:::core
        DeadCode["eliminate_dead_code_pass.py"]:::core
        LayoutOpt["optimize_layout_transposes_pass/"]:::core
        RemoveOutputs["remove_non_user_outputs_pass.py"]:::core
    end

    %% Composite & MLIR Builder
    subgraph "Composite & MLIR Builder"
        style MLIRBuilder fill:#FFE0B2,stroke:#FB8C00,stroke-width:2px
        StableHLO["stablehlo_composite_builder.py"]:::builder
        MarkTensor["mark_tensor.py"]:::builder
        Lowerings["lowerings/"]:::builder
        ExportUtils["export_utils.py"]:::builder
        ExportIR["export.py"]:::builder
    end

    %% Backend Exporters
    subgraph "Backend Exporters"
        style Backends fill:#FFE0B2,stroke:#FB8C00,stroke-width:2px
        TFIntegration["tf_integration.py"]:::builder
        TorchTFL["experimental/torch_tfl/"]:::builder
        TFLLowerings["_lowerings.py"]:::builder
    end

    %% Generative API Stack
    subgraph "Generative API Stack"
        style Generative fill:#E1BEE7,stroke:#8E24AA,stroke-width:2px
        GenLayers["layers/"]:::gen
        GenQuant["quantize/"]:::gen
        GenUtils["utilities/"]:::gen
        GenCustom["custom_ops/"]:::gen
    end

    %% Bridge & Utility Modules
    subgraph "Bridge & Utility Modules"
        style Utils fill:#CFD8DC,stroke:#546E7A,stroke-width:2px
        JAXBridge["jax_bridge/"]:::util
        Lowertools["lowertools/"]:::util
        DebugMod["debug/"]:::util
    end

    %% Examples & Tests
    subgraph "Examples & Tests"
        style Examples fill:#B3E5FC,stroke:#0288D1,stroke-width:2px
        ExamplesPy["examples/"]:::tests
        GenExamples["generative/examples/"]:::tests
        ConvTest["_convert/test/"]:::tests
        GenTest["generative/test/"]:::tests
        OdmlTest["odml_torch/test/"]:::tests
        TestingMod["testing/"]:::tests
        E2ETests["e2e_tests/"]:::tests
    end

    %% External Dependencies
    PyTorch["PyTorch"]:::external
    TFLite["TensorFlow Lite"]:::external
    MediaPipe["MediaPipe LLM API"]:::external

    %% Connections
    UserAPI -->|"calls"| Conv
    Conv -->|"imports"| Converter
    ConvUtils --> Converter
    Signature --> Converter

    Converter -->|"produces FX GraphModule"| BuildAten
    BuildAten --> DeadCode
    DeadCode --> LayoutOpt
    LayoutOpt --> RemoveOutputs
    RemoveOutputs --> StableHLO

    StableHLO --> MarkTensor --> Lowerings --> ExportUtils --> ExportIR
    ExportIR --> TFIntegration --> TorchTFL --> TFLLowerings

    GenLayers -->|"hooks into"| Conv
    GenQuant --> ConvUtils
    GenUtils --> Converter
    GenCustom --> Converter

    JAXBridge --> StableHLO
    Lowertools --> Conv
    DebugMod --> Converter

    ExamplesPy --> UserAPI
    GenExamples --> UserAPI
    ConvTest --> ExportIR
    GenTest --> ExportIR
    OdmlTest --> ExportIR
    TestingMod --> UserAPI
    E2ETests --> ExportIR

    PyTorch -.-> UserAPI
    TFLite -.-> TFIntegration
    MediaPipe -.-> ExportIR

    %% Click Events
    click UserAPI "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/__init__.py"
    click UserAPI "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/model.py"
    click UserAPI "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_config.py"
    click Conv "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/conversion.py"
    click Converter "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/converter.py"
    click ConvUtils "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/conversion_utils.py"
    click Signature "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/signature.py"
    click BuildAten "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/fx_passes/build_aten_composite_pass.py"
    click DeadCode "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/fx_passes/eliminate_dead_code_pass.py"
    click LayoutOpt "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/_convert/fx_passes/optimize_layout_transposes_pass/"
    click RemoveOutputs "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/_convert/fx_passes/remove_non_user_outputs_pass.py"
    click StableHLO "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/odml_torch/composite/stablehlo_composite_builder.py"
    click MarkTensor "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/odml_torch/composite/mark_tensor.py"
    click Lowerings "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/odml_torch/lowerings/"
    click ExportUtils "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/odml_torch/export_utils.py"
    click ExportIR "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/odml_torch/export.py"
    click TFIntegration "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/odml_torch/tf_integration.py"
    click TorchTFL "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/odml_torch/experimental/torch_tfl/"
    click TFLLowerings "https://github.com/google-ai-edge/ai-edge-torch/blob/main/ai_edge_torch/odml_torch/experimental/torch_tfl/_lowerings.py"
    click GenLayers "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/layers/"
    click GenQuant "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/quantize/"
    click GenUtils "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/utilities/"
    click GenCustom "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/custom_ops/"
    click JAXBridge "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/odml_torch/jax_bridge/"
    click Lowertools "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/lowertools/"
    click DebugMod "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/debug/"
    click ExamplesPy "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/examples/"
    click GenExamples "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/examples/"
    click ConvTest "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/_convert/test/"
    click GenTest "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/test/"
    click OdmlTest "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/odml_torch/test/"
    click TestingMod "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/testing/"
    click E2ETests "https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/e2e_tests/"

    %% Styles
    classDef api fill:#BBDEFB,stroke:#1E88E5,color:#0D47A1
    classDef core fill:#C8E6C9,stroke:#43A047,color:#1B5E20
    classDef builder fill:#FFE0B2,stroke:#FB8C00,color:#E65100
    classDef gen fill:#E1BEE7,stroke:#8E24AA,color:#4A148C
    classDef util fill:#CFD8DC,stroke:#546E7A,color:#263238
    classDef tests fill:#B3E5FC,stroke:#0288D1,color:#01579B
    classDef external fill:#FFFFFF,stroke-dasharray: 5 5,stroke:#9E9E9E,color:#616161
```
