# tenstorrent/tt-torch architecture

> Architecture diagram of the GitHub repository tenstorrent/tt-torch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tenstorrent/tt-torch
- Repository: https://github.com/tenstorrent/tt-torch

## Overview

Based on our analysis of the file tree and README, this project is a front-end tool that bridges PyTorch (using PyTorch 2.0) with the torch-mlir infrastructure, ultimately interfacing with the more backend-oriented tt-mlir. In simpler terms, its purpose is to allow users to take PyTorch models and compile or lower them (via torch-mlir) into a format (tt-mlir) that can be consumed by specialized hardware or further compiler passes.

##### Main Components
a. PyTorch Frontend:
- This is where the user’s model is defined in PyTorch, and from where the conversion process is initiated.
- That it leverages PyTorch 2.0.
b. tt-torch Conversion Engine:
- “tt_torch” directory as the core of the conversion process.
- Within this, include submodules such as “dynamo” (for dynamic conversion), “onnx_compile” (if exporting models via ONNX is supported), and “tools” (for benchmarking, profiling, and generation of reports).
- C++ layer (in the “csrc” folder) providing bindings and interfacing with MLIR. This part is pivotal in connecting torch-mlir to tt-mlir.
c. MLIR/tt-mlir Integration:
- Interface between the conversion engine and the lower-level compilation or optimization provided by tt-mlir.
- Mention that torch-mlir is being used to lower PyTorch IR into MLIR, with tt-torch acting as a front-end to drive this process.
d. CI/CD and Build Infrastructure:
- A block for continuous integration (CI) and continuous deployment (CD) which is managed via the .github/workflows scripts and Dockerfiles.
- This component ensures that tests (unit, integration, e2e), benchmarks, and regression reports are run automatically.
e. Testing & Demos:
- Incorporate the “tests” directory representing the various model tests (e.g., ResNet, GPT2, and many others) which check the integrity of the conversions and correctness of outputs.
- “demos” folder (like the resnet demo) to illustrate sample usage.
f. Documentation & Developers’ Tools:
- “docs” directory as a component that provides architectural overviews, usage guides, and operation details (covering supported models and ops).
- Configuration files like pyproject.toml, setup.py, and CMakeLists.txt that support both building and integration testing.

##### Architectural Patterns and Principles
- Separation of concerns: the front-end (user interface/model ingestion) is clearly distinct from the compilation/conversion logic.
- The layered approach: high-level Python components call lower-level C++ (MLIR interfacing) modules.
- Modularity: many components (e.g., dynamo, onnx_compile, and various tools) are designed to be tested independently and integrated seamlessly.
- Incorporate the CI/CD pipeline as a separate “monitoring and quality assurance” layer that automates testing, benchmarking, and code style enforcement.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Layer"
        FE["PyTorch Frontend (PyTorch 2.0)"]:::frontend
    end

    subgraph "Conversion Layer"
        CE["tt-torch Conversion Engine"]:::conversion
        DYN["Dynamo"]:::conversion
        ONNX["ONNX Compile"]:::conversion
        TOLS["Tools"]:::conversion
        CPP["C++ Binding Layer"]:::cpp
        MLIR["MLIR/tt-mlir Integration"]:::integration
        EXT["External: torch-mlir/tt-mlir"]:::external
    end

    subgraph "CI/CD and Build"
        WORK["CI Workflows"]:::cicd
        DOCK["Dockerfiles"]:::cicd
        CMAKE["CMake Config"]:::cicd
        PYPROJ["PyProject Config"]:::cicd
        SETUP["Setup Config"]:::cicd
        CI["CI/CD Pipeline"]:::cicd
    end

    subgraph "Testing and Demos"
        TESTS["Tests"]:::testing
        DEMOS["Demos"]:::testing
    end

    subgraph "Documentation & Dev Tools"
        DOCS["Documentation"]:::docs
        DOCSRC["Docs Source"]:::docs
    end

    %% Data Flow Connections
    FE -->|"triggers_conversion"| CE
    CE -->|"has"| DYN
    CE -->|"has"| ONNX
    CE -->|"has"| TOLS
    CE -->|"has"| CPP
    CPP -->|"interfaces"| MLIR
    CE -->|"uses"| EXT
    CE -->|"build_trigger"| CI
    TESTS -->|"feedback"| CE
    DOCS -->|"supports"| CI
    DEMOS --> FE

    %% CI/CD Subgraph Internal Connections
    WORK --> CI
    DOCK --> CI
    CMAKE --> CI
    PYPROJ --> CI
    SETUP --> CI

    %% Click Events
    click FE "https://github.com/tenstorrent/tt-torch/tree/main/tests/models"
    click TESTS "https://github.com/tenstorrent/tt-torch/tree/main/tests"
    click DEMOS "https://github.com/tenstorrent/tt-torch/tree/main/demos/resnet"
    click CE "https://github.com/tenstorrent/tt-torch/tree/main/tt_torch"
    click DYN "https://github.com/tenstorrent/tt-torch/tree/main/tt_torch/dynamo"
    click ONNX "https://github.com/tenstorrent/tt-torch/tree/main/tt_torch/onnx_compile"
    click TOLS "https://github.com/tenstorrent/tt-torch/tree/main/tt_torch/tools"
    click CPP "https://github.com/tenstorrent/tt-torch/tree/main/tt_torch/csrc"
    click MLIR "https://github.com/tenstorrent/tt-torch/blob/main/tt_torch/csrc/tt-mlir-interface.cpp"
    click MLIR "https://github.com/tenstorrent/tt-torch/blob/main/tt_torch/csrc/tt-mlir-interface.hpp"
    click WORK "https://github.com/tenstorrent/tt-torch/tree/main/.github/workflows"
    click DOCK "https://github.com/tenstorrent/tt-torch/blob/main/.github/Dockerfile.*"
    click CMAKE "https://github.com/tenstorrent/tt-torch/blob/main/CMakeLists.txt"
    click PYPROJ "https://github.com/tenstorrent/tt-torch/blob/main/pyproject.toml"
    click SETUP "https://github.com/tenstorrent/tt-torch/blob/main/setup.py"
    click DOCS "https://github.com/tenstorrent/tt-torch/tree/main/docs"
    click DOCSRC "https://github.com/tenstorrent/tt-torch/tree/main/docs/src"

    %% Styles
    classDef frontend fill:#f9e79f,stroke:#b9770e,stroke-width:2px;
    classDef conversion fill:#aed6f1,stroke:#2874a6,stroke-width:2px;
    classDef cpp fill:#d6eaf8,stroke:#1b4f72,stroke-width:2px;
    classDef integration fill:#abebc6,stroke:#1e8449,stroke-width:2px;
    classDef cicd fill:#f5b7b1,stroke:#943126,stroke-width:2px;
    classDef testing fill:#d2b4de,stroke:#6c3483,stroke-width:2px;
    classDef docs fill:#fcf3cf,stroke:#7d6608,stroke-width:2px;
    classDef external fill:#f5cba7,stroke:#a04000,stroke-width:2px;
```
