# jwkim04/tesseract-npu architecture

> Architecture diagram of the GitHub repository jwkim04/tesseract-npu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jwkim04/tesseract-npu
- Repository: https://github.com/jwkim04/tesseract-npu

## Overview

Tesseract is a full-stack FPGA NPU for LLM inference. Its main workflow takes a llama.cpp graph, lowers supported operations into an op list, transfers inputs and launches through a common transport, then runs firmware on the control core to issue tile operations; results return to the host as logits. The README also documents PyTorch matmul offload, a bit-exact simulator, kernel SDK, and profiling/debug tools. The high-level edges below follow README statements; detailed call wiring in unsampled modules is not asserted.

Main entry points: [`sw/ggml-tesseract/ggml-tesseract.cpp`](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/ggml-tesseract.cpp) (ggml backend), [`sw/transport/tsr_kdev.cpp`](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev.cpp) (Transport protocol), [`sw/transport/tsr_transport.h`](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_transport.h) (Transport API), [`sw/ctrl/kernels/graph.cpp`](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/kernels/graph.cpp) (Graph firmware kernel)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_host["Host inference"]
  node_torch["PyTorch backend<br/>[backend.py]"]
end

subgraph group_backend["Graph backend"]
  node_ggml["ggml backend<br/>[ggml-tesseract.cpp]"]
  node_lowering["Graph lowering<br/>[tsr-lower.h]"]
  node_oplist["Op-list builder<br/>[tsr-opl.h]"]
  node_memory["Device memory<br/>[tsr-alloc.h]"]
  node_weights["Weight layout<br/>[tsr-wlayout.h]"]
end

subgraph group_execution["Execution stack"]
  node_runtime["Basis runtime<br/>[tsr_rt.cpp]"]
  node_transport["Transport protocol<br/>[tsr_kdev.cpp]"]
  node_transport_api["Transport API<br/>[tsr_transport.h]"]
  node_xrt_kind["XRT transport"]
  node_firmware["Graph firmware kernel<br/>[graph.cpp]"]
  node_opl_sdk["Kernel op-list SDK<br/>[opl.hpp]"]
  node_elf_tool["Firmware image tool<br/>[tsr_elf.h]"]
end

subgraph group_simulation["Simulation"]
  node_sim_kind["Simulator transport<br/>[tsr_kdev_sim.cpp]"]
  node_mock_kind["Mock transport<br/>[tsr_kdev_mock.cpp]"]
  node_simulator["Bit-exact simulator<br/>[tsr_sim.c]"]
  node_numerics["Numerical reference<br/>[tsr_ref.h]"]
  node_isa["ISA definitions<br/>[tsr_isa.h]"]
  node_tb_issue["Issue-path verification<br/>[issue_sim.py]"]
end

subgraph group_operations["Developer tools"]
  node_profiler["Profile reporting<br/>[prof_report.py]"]
  node_monitor["Device monitor<br/>[tsr_top.py]"]
  node_debugger["Device debugger<br/>[tsr_dbg.cpp]"]
end

node_llama(("llama.cpp"))
node_torch_user(("PyTorch user"))

node_llama -->|"submits graph"| node_ggml
node_ggml -->|"lowers graph"| node_lowering
node_ggml -->|"builds op list"| node_oplist
node_ggml -->|"launches through"| node_transport
node_ggml -->|"uses allocation"| node_memory
node_ggml -->|"uses weight layout"| node_weights
node_torch_user -->|"invokes backend"| node_torch
node_torch -.->|"offloads matmuls"| node_ggml
node_transport -->|"implements API"| node_transport_api
node_transport -->|"dispatches to"| node_sim_kind
node_transport -->|"dispatches to"| node_mock_kind
node_transport -->|"dispatches to"| node_xrt_kind
node_transport -->|"launches kernel"| node_firmware
node_firmware -->|"walks op list"| node_opl_sdk
node_sim_kind -->|"runs in process"| node_simulator
node_mock_kind -->|"models adapter"| node_simulator
node_simulator -->|"uses reference numerics"| node_numerics
node_runtime -->|"uses transport"| node_transport_api
node_runtime -->|"builds op list"| node_oplist
node_firmware -->|"issues tile ops"| node_isa
node_elf_tool -->|"checks image ABI"| node_opl_sdk
node_profiler -.->|"profiles device"| node_transport_api
node_monitor -.->|"monitors device"| node_transport_api
node_debugger -.->|"debugs control core"| node_transport_api
node_tb_issue -->|"models instructions"| node_isa

click node_ggml "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/ggml-tesseract.cpp"
click node_lowering "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-lower.h"
click node_oplist "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-opl.h"
click node_memory "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-alloc.h"
click node_weights "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-wlayout.h"
click node_torch "https://github.com/jwkim04/tesseract-npu/blob/main/sw/torch/tesseract_torch/backend.py"
click node_runtime "https://github.com/jwkim04/tesseract-npu/blob/main/sw/rt/tsr_rt.cpp"
click node_transport "https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev.cpp"
click node_transport_api "https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_transport.h"
click node_sim_kind "https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev_sim.cpp"
click node_mock_kind "https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev_mock.cpp"
click node_xrt_kind "https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_transport_xrt.cpp"
click node_simulator "https://github.com/jwkim04/tesseract-npu/blob/main/sw/sim/tsr_sim.c"
click node_numerics "https://github.com/jwkim04/tesseract-npu/blob/main/sw/sim/include/tsr_ref.h"
click node_firmware "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/kernels/graph.cpp"
click node_opl_sdk "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/include/tsr/opl.hpp"
click node_elf_tool "https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/tools/tsr-elf/tsr_elf.h"
click node_profiler "https://github.com/jwkim04/tesseract-npu/blob/main/sw/tools/prof/prof_report.py"
click node_monitor "https://github.com/jwkim04/tesseract-npu/blob/main/sw/tools/top/tsr_top.py"
click node_debugger "https://github.com/jwkim04/tesseract-npu/blob/main/sw/tools/tsr_dbg.cpp"
click node_isa "https://github.com/jwkim04/tesseract-npu/blob/main/sw/sim/include/tsr_isa.h"
click node_tb_issue "https://github.com/jwkim04/tesseract-npu/blob/main/tb/issue/issue_sim.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_torch,node_torch_user toneBlue
class node_ggml,node_lowering,node_oplist,node_memory,node_weights toneAmber
class node_runtime,node_transport,node_transport_api,node_xrt_kind,node_firmware,node_opl_sdk,node_elf_tool toneMint
class node_sim_kind,node_mock_kind,node_simulator,node_numerics,node_isa,node_tb_issue toneRose
class node_profiler,node_monitor,node_debugger,node_llama toneIndigo
```

## Components

### Host inference

- [PyTorch backend](https://github.com/jwkim04/tesseract-npu/blob/main/sw/torch/tesseract_torch/backend.py): `sw/torch/tesseract_torch/backend.py`

### Graph backend

- [ggml backend](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/ggml-tesseract.cpp): `sw/ggml-tesseract/ggml-tesseract.cpp`
- [Graph lowering](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-lower.h): `sw/ggml-tesseract/tsr-lower.h`
- [Op-list builder](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-opl.h): `sw/ggml-tesseract/tsr-opl.h`
- [Device memory](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-alloc.h): `sw/ggml-tesseract/tsr-alloc.h`
- [Weight layout](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ggml-tesseract/tsr-wlayout.h): `sw/ggml-tesseract/tsr-wlayout.h`

### Execution stack

- [Basis runtime](https://github.com/jwkim04/tesseract-npu/blob/main/sw/rt/tsr_rt.cpp): `sw/rt/tsr_rt.cpp`
- [Transport protocol](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev.cpp): `sw/transport/tsr_kdev.cpp`
- [Transport API](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_transport.h): `sw/transport/tsr_transport.h`
- [XRT transport](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_transport_xrt.cpp): `sw/transport/tsr_transport_xrt.cpp`
- [Graph firmware kernel](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/kernels/graph.cpp): `sw/ctrl/kernels/graph.cpp`
- [Kernel op-list SDK](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/include/tsr/opl.hpp): `sw/ctrl/include/tsr/opl.hpp`
- [Firmware image tool](https://github.com/jwkim04/tesseract-npu/blob/main/sw/ctrl/tools/tsr-elf/tsr_elf.h): `sw/ctrl/tools/tsr-elf/tsr_elf.h`

### Simulation

- [Simulator transport](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev_sim.cpp): `sw/transport/tsr_kdev_sim.cpp`
- [Mock transport](https://github.com/jwkim04/tesseract-npu/blob/main/sw/transport/tsr_kdev_mock.cpp): `sw/transport/tsr_kdev_mock.cpp`
- [Bit-exact simulator](https://github.com/jwkim04/tesseract-npu/blob/main/sw/sim/tsr_sim.c): `sw/sim/tsr_sim.c`
- [Numerical reference](https://github.com/jwkim04/tesseract-npu/blob/main/sw/sim/include/tsr_ref.h): `sw/sim/include/tsr_ref.h`
- [ISA definitions](https://github.com/jwkim04/tesseract-npu/blob/main/sw/sim/include/tsr_isa.h): `sw/sim/include/tsr_isa.h`
- [Issue-path verification](https://github.com/jwkim04/tesseract-npu/blob/main/tb/issue/issue_sim.py): `tb/issue/issue_sim.py`

### Developer tools

- [Profile reporting](https://github.com/jwkim04/tesseract-npu/blob/main/sw/tools/prof/prof_report.py): `sw/tools/prof/prof_report.py`
- [Device monitor](https://github.com/jwkim04/tesseract-npu/blob/main/sw/tools/top/tsr_top.py): `sw/tools/top/tsr_top.py`
- [Device debugger](https://github.com/jwkim04/tesseract-npu/blob/main/sw/tools/tsr_dbg.cpp): `sw/tools/tsr_dbg.cpp`

### Other components

- **llama.cpp**
- **PyTorch user**

## Connections

- llama.cpp → ggml backend: submits graph (evidence: `README.md`)
- ggml backend → Graph lowering: lowers graph (evidence: `sw/ggml-tesseract/ggml-tesseract.cpp`)
- ggml backend → Op-list builder: builds op list (evidence: `sw/ggml-tesseract/ggml-tesseract.cpp`)
- ggml backend → Transport protocol: launches through (evidence: `README.md`)
- ggml backend → Device memory: uses allocation (evidence: `sw/ggml-tesseract/ggml-tesseract.cpp`)
- ggml backend → Weight layout: uses weight layout (evidence: `sw/ggml-tesseract/ggml-tesseract.cpp`)
- PyTorch user → PyTorch backend: invokes backend (evidence: `README.md`)
- PyTorch backend → ggml backend: offloads matmuls
- Transport protocol → Transport API: implements API (evidence: `sw/transport/tsr_kdev.cpp`)
- Transport protocol → Simulator transport: dispatches to
- Transport protocol → Mock transport: dispatches to
- Transport protocol → XRT transport: dispatches to
- Transport protocol → Graph firmware kernel: launches kernel (evidence: `README.md`)
- Graph firmware kernel → Kernel op-list SDK: walks op list (evidence: `README.md`)
- Simulator transport → Bit-exact simulator: runs in process
- Mock transport → Bit-exact simulator: models adapter (evidence: `README.md`)
- Bit-exact simulator → Numerical reference: uses reference numerics (evidence: `sw/sim/include/tsr_ref.h`)
- Basis runtime → Transport API: uses transport (evidence: `sw/rt/tsr_rt.cpp`)
- Basis runtime → Op-list builder: builds op list (evidence: `sw/rt/tsr_rt.cpp`)
- Graph firmware kernel → ISA definitions: issues tile ops (evidence: `README.md`)
- Firmware image tool → Kernel op-list SDK: checks image ABI (evidence: `sw/ctrl/tools/tsr-elf/main.cpp`)
- Profile reporting → Transport API: profiles device
- Device monitor → Transport API: monitors device
- Device debugger → Transport API: debugs control core
- Issue-path verification → ISA definitions: models instructions