How tenstorrent/tt-torch works
Based on our analysis of the file tree and README, this project is a front-end tool that bridges PyTorch (using PyTorch 2.0) with the torch-mlir infrastructure, ultimately interfacing with the more backend-oriented tt-mlir. In simpler terms, its purpose is to allow users to take PyTorch models and compile or lower them (via torch-mlir) into a format (tt-mlir) that can be consumed by specialized hardware or further compiler passes.
Main Components
a. PyTorch Frontend:
• This is where the user’s model is defined in PyTorch, and from where the conversion process is initiated.
• That it leverages PyTorch 2.0.
b. tt-torch Conversion Engine:
• “tt_torch” directory as the core of the conversion process.
• Within this, include submodules such as “dynamo” (for dynamic conversion), “onnx_compile” (if exporting models via ONNX is supported), and “tools” (for benchmarking, profiling, and generation of reports).
• C++ layer (in the “csrc” folder) providing bindings and interfacing with MLIR. This part is pivotal in connecting torch-mlir to tt-mlir.
c. MLIR/tt-mlir Integration:
• Interface between the conversion engine and the lower-level compilation or optimization provided by tt-mlir.
• Mention that torch-mlir is being used to lower PyTorch IR into MLIR, with tt-torch acting as a front-end to drive this process.
d. CI/CD and Build Infrastructure:
• A block for continuous integration (CI) and continuous deployment (CD) which is managed via the .github/workflows scripts and Dockerfiles.
• This component ensures that tests (unit, integration, e2e), benchmarks, and regression reports are run automatically.
e. Testing & Demos:
• Incorporate the “tests” directory representing the various model tests (e.g., ResNet, GPT2, and many others) which check the integrity of the conversions and correctness of outputs.
• “demos” folder (like the resnet demo) to illustrate sample usage.
f. Documentation & Developers’ Tools:
• “docs” directory as a component that provides architectural overviews, usage guides, and operation details (covering supported models and ops).
• Configuration files like pyproject.toml, setup.py, and CMakeLists.txt that support both building and integration testing.
Architectural Patterns and Principles
• Separation of concerns: the front-end (user interface/model ingestion) is clearly distinct from the compilation/conversion logic.
• The layered approach: high-level Python components call lower-level C++ (MLIR interfacing) modules.
• Modularity: many components (e.g., dynamo, onnx_compile, and various tools) are designed to be tested independently and integrated seamlessly.
• Incorporate the CI/CD pipeline as a separate “monitoring and quality assurance” layer that automates testing, benchmarking, and code style enforcement.