How tenstorrent/tt-xla works
Project type and purpose
• TT-XLA is not a web app but a compiler/runtime integration project: it provides a PJRT (Portable JAX Runtime) plugin that lets JAX hand off computation to Tenstorrent hardware via a StableHLO→TT-MLIR compilation pipeline.
• Purpose: ingest JAX-jit-compiled graphs, produce SHLO IR, feed that into TT-MLIR, emit hardware binaries, and execute on Tenstorrent accelerators.
File structure
Top-level components:
• C++ “core” (inc/, src/): implements the PJRT plugin API bindings, buffer/device/executable abstractions, module building.
• python_package/: Python wrapper that installs TT-XLA as a JAX backend, handles plugin discovery, packaging, and entry-points.
• third_party/: external C-API, MLIR/TT-C++ libraries (pjrt_c_api, tt_forge_models, ttxla_tools).
• examples/: Python snippets for JAX and PyTorch demos.
• tests/: extensive pytest suite validating both PJRT plugin behavior and model‐runner infrastructure for single-chip and multi-chip scenarios under JAX and PyTorch.
• docs/: user-facing guides, Doxygen config, book.toml.
• CI/CD (.github/): Dockerfiles, workflow scripts, build/test pipelines.
README for additional insights
• Confirms key layers: JAX front-end → PJRT interface → StableHLO → tt-mlir backend → Tenstorrent hardware.
• Points to related projects (TT-MLIR, TT-Forge-FE, TT-Torch) that implement analogous pipelines.
a. Main components (grouped in logical layers):
1) Python Front-End Layer
– User models & example scripts
– JAX Python runtime (jit, tracer)
– python_package/installer & PJRT client stub
2) PJRT Plugin Layer
– TT-XLA plugin dynamic library
– C++ bindings in inc/common/pjrt_implementation & src/common/pjrt_implementation
3) IR & Compiler Layer
– StableHLO IR representation (shlo graph)
– TT-MLIR compiler (external process or library)
4) Hardware Runtime Layer
– PJRT runtime calls down to Tenstorrent hardware driver
– Device buffers, memory management, event synchronization
5) Support & Infrastructure
– Tests harness (pytest, comparator/runners/connectors)
– Build system (CMake, Docker)
– Documentation generator (Doxygen, mdbook)
b. Relationships & interactions
– Highlight data flows: model parameters, compiled executable, device buffers, result tensors
– Test harness interacts with PJRT client and device connectors to validate outputs
c. Architectural patterns & design principles
– Plugin architecture (PJRT) to decouple front-end (JAX) from backend (TT-MLIR+hardware)
– Multi-layer IR translation: JAX’s XLA → StableHLO → MLIR → device code
– C++/Python mixed language bridge via CPython extension (dynamic lib)
– Layered testing: infrastructure modules (connectors, comparators) abstract device vs. host
d. Key technologies/frameworks
– Python: JAX, pytest, setuptools/pyproject.toml
– C++: PJRT C API, CMake build, Doxygen docs
– MLIR: TT-MLIR compiler invoked on SHLO graphs
– Hardware driver: Tenstorrent PJRT device backend
Diagramming suggestions
– Python layer (e.g., light blue),
– C++ plugin (green),
– IR/compiler (orange),
– Hardware/runtime (purple),
– Tests/CI (gray).
– Rectangles for components,
– Clouds or servers for external services (Docker registry, GitHub Actions).