# tenstorrent/tt-xla architecture

> Architecture diagram of the GitHub repository tenstorrent/tt-xla, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tenstorrent/tt-xla
- Repository: https://github.com/tenstorrent/tt-xla

## Overview

##### Project type and purpose
- TT-XLA is not a web app but a compiler/runtime integration project: it provides a PJRT (Portable JAX Runtime) plugin that lets JAX hand off computation to Tenstorrent hardware via a StableHLO→TT-MLIR compilation pipeline.
- Purpose: ingest JAX-jit-compiled graphs, produce SHLO IR, feed that into TT-MLIR, emit hardware binaries, and execute on Tenstorrent accelerators.

##### File structure
Top-level components:
- C++ “core” (inc/, src/): implements the PJRT plugin API bindings, buffer/device/executable abstractions, module building.
- python_package/: Python wrapper that installs TT-XLA as a JAX backend, handles plugin discovery, packaging, and entry-points.
- third_party/: external C-API, MLIR/TT-C++ libraries (pjrt_c_api, tt_forge_models, ttxla_tools).
- examples/: Python snippets for JAX and PyTorch demos.
- tests/: extensive pytest suite validating both PJRT plugin behavior and model‐runner infrastructure for single-chip and multi-chip scenarios under JAX and PyTorch.
- docs/: user-facing guides, Doxygen config, book.toml.
- CI/CD (​.github/): Dockerfiles, workflow scripts, build/test pipelines.

##### README for additional insights
- Confirms key layers: JAX front-end → PJRT interface → StableHLO → tt-mlir backend → Tenstorrent hardware.
- Points to related projects (TT-MLIR, TT-Forge-FE, TT-Torch) that implement analogous pipelines.

a. Main components (grouped in logical layers):
1) Python Front-End Layer
– User models & example scripts
– JAX Python runtime (jit, tracer)
– python_package/installer & PJRT client stub
2) PJRT Plugin Layer
– TT-XLA plugin dynamic library
– C++ bindings in inc/common/pjrt_implementation & src/common/pjrt_implementation
3) IR & Compiler Layer
– StableHLO IR representation (shlo graph)
– TT-MLIR compiler (external process or library)
4) Hardware Runtime Layer
– PJRT runtime calls down to Tenstorrent hardware driver
– Device buffers, memory management, event synchronization
5) Support & Infrastructure
– Tests harness (pytest, comparator/runners/connectors)
– Build system (CMake, Docker)
– Documentation generator (Doxygen, mdbook)

b. Relationships & interactions
– Highlight data flows: model parameters, compiled executable, device buffers, result tensors
– Test harness interacts with PJRT client and device connectors to validate outputs

c. Architectural patterns & design principles
– Plugin architecture (PJRT) to decouple front-end (JAX) from backend (TT-MLIR+hardware)
– Multi-layer IR translation: JAX’s XLA → StableHLO → MLIR → device code
– C++/Python mixed language bridge via CPython extension (dynamic lib)
– Layered testing: infrastructure modules (connectors, comparators) abstract device vs. host

d. Key technologies/frameworks
– Python: JAX, pytest, setuptools/pyproject.toml
– C++: PJRT C API, CMake build, Doxygen docs
– MLIR: TT-MLIR compiler invoked on SHLO graphs
– Hardware driver: Tenstorrent PJRT device backend

##### Diagramming suggestions
– Python layer (e.g., light blue),
– C++ plugin (green),
– IR/compiler (orange),
– Hardware/runtime (purple),
– Tests/CI (gray).
– Rectangles for components,
– Clouds or servers for external services (Docker registry, GitHub Actions).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Python Front-End Layer
    subgraph "Python Front-End Layer"
        direction TB
        A1["Example Scripts (JAX)"]:::python
        A2["Example Scripts (PyTorch)"]:::python
        A3["python_package Installer & PJRT Client Stub"]:::python
    end

    %% PJRT Plugin Layer
    subgraph "PJRT Plugin Layer"
        direction TB
        B1["Headers pjrt_implementation"]:::plugin
        B2["Sources pjrt_implementation"]:::plugin
        B3a["Dylib Entry Point"]:::plugin
        B3b["Platform Glue"]:::plugin
    end

    %% IR & Compiler Layer
    subgraph "IR & Compiler Layer"
        direction TB
        C1["StableHLO IR Utils & Module Builder"]:::compiler
        C2["Compile Options"]:::compiler
        C3["TT-MLIR Tools Wrapper"]:::compiler
    end

    %% Hardware Runtime Layer
    subgraph "Hardware Runtime Layer"
        direction TB
        D1["Tenstorrent PJRT Client & Device Interaction"]:::hardware
        D2["PJRT Runtime Entry-Point for Tenstorrent"]:::hardware
    end

    %% Support & Infrastructure
    subgraph "Support & Infrastructure"
        direction TB
        E1["Test Harness"]:::support
        E2["Single/Multi-Chip Tests"]:::support
        E3["Build System Config"]:::support
        E4["CI/CD Workflows & Dockerfiles"]:::support
        E5["Documentation"]:::support
    end

    %% Data & Control Flow
    A1 -->|"model & jit" | A3
    A2 -->|"model & jit"| A3
    A3 -->|"load plugin"| B3a
    A3 -->|"load plugin"| B3b
    B3a -->|"provides API"| B1
    B3a -->|"provides API"| B2
    B3b -->|"entry calls"| C1
    B2 -->|"generate SHLO"| C1
    B1 -->|"use headers"| C1
    C1 -->|"IR translation"| C2
    C2 -->|"apply opts"| C3
    C3 -->|"compile to binary"| D1
    D1 -->|"enqueue & execute"| D2
    D2 -->|"return tensor results"| A3

    %% Testing interactions
    E1 -->|"validate PJRT client"| A3
    E1 -->|"validate device interaction"| D1
    E2 -->|"run examples"| A1
    E2 -->|"run examples"| A2

    %% Click Events
    click A1 "https://github.com/tenstorrent/tt-xla/tree/main/examples/jax/"
    click A2 "https://github.com/tenstorrent/tt-xla/tree/main/examples/pytorch/"
    click A3 "https://github.com/tenstorrent/tt-xla/tree/main/python_package/"
    click B1 "https://github.com/tenstorrent/tt-xla/tree/main/inc/common/pjrt_implementation/"
    click B2 "https://github.com/tenstorrent/tt-xla/tree/main/src/common/pjrt_implementation/"
    click B3a "https://github.com/tenstorrent/tt-xla/blob/main/src/common/dylib_entry_point.cc.inc"
    click B3b "https://github.com/tenstorrent/tt-xla/tree/main/src/common/dylib_platform.cc/h"
    click C1 "https://github.com/tenstorrent/tt-xla/blob/main/src/common/module_builder.cc"
    click C2 "https://github.com/tenstorrent/tt-xla/blob/main/src/common/compile_options.cc"
    click C3 "https://github.com/tenstorrent/tt-xla/blob/main/ttxla_tools/ttxla_tools.py"
    click D1 "https://github.com/tenstorrent/tt-xla/blob/main/src/tt/client.cc"
    click D2 "https://github.com/tenstorrent/tt-xla/blob/main/src/tt/dylib_entry_point.cc"
    click E1 "https://github.com/tenstorrent/tt-xla/tree/main/tests/infra/"
    click E2 "https://github.com/tenstorrent/tt-xla/tree/main/tests/jax/"
    click E3 "https://github.com/tenstorrent/tt-xla/blob/main/CMakeLists.txt"
    click E4 "https://github.com/tenstorrent/tt-xla/tree/main/.github/workflows/"
    click E5 "https://github.com/tenstorrent/tt-xla/tree/main/docs/"

    %% Styles
    classDef python fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef plugin fill:#90EE90,stroke:#333,stroke-width:1px
    classDef compiler fill:#FFA500,stroke:#333,stroke-width:1px
    classDef hardware fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef support fill:#D3D3D3,stroke:#333,stroke-width:1px
```
