How jwkim04/tesseract-npu works
Tesseract is a full-stack FPGA NPU for LLM inference. Its main workflow takes a llama.cpp graph, lowers supported operations into an op list, transfers inputs and launches through a common transport, then runs firmware on the control core to issue tile operations; results return to the host as logits. The README also documents PyTorch matmul offload, a bit-exact simulator, kernel SDK, and profiling/debug tools. The high-level edges below follow README statements; detailed call wiring in unsampled modules is not asserted.
Main entry points: ggml-tesseract.cpp ggml backend · tsr_kdev.cpp Transport protocol · tsr_transport.h Transport API · graph.cpp Graph firmware kernel
Components 24
Host inference
- PyTorch backend
sw/torch/tesseract_torch/backend.py
Graph backend
- ggml backend
sw/ggml-tesseract/ggml-tesseract.cpp - Graph lowering
sw/ggml-tesseract/tsr-lower.h - Op-list builder
sw/ggml-tesseract/tsr-opl.h - Device memory
sw/ggml-tesseract/tsr-alloc.h - Weight layout
sw/ggml-tesseract/tsr-wlayout.h
Execution stack
- Basis runtime
sw/rt/tsr_rt.cpp - Transport protocol
sw/transport/tsr_kdev.cpp - Transport API
sw/transport/tsr_transport.h - XRT transport
sw/transport/tsr_transport_xrt.cpp - Graph firmware kernel
sw/ctrl/kernels/graph.cpp - Kernel op-list SDK
sw/ctrl/include/tsr/opl.hpp - Firmware image tool
sw/ctrl/tools/tsr-elf/tsr_elf.h
Simulation
- Simulator transport
sw/transport/tsr_kdev_sim.cpp - Mock transport
sw/transport/tsr_kdev_mock.cpp - Bit-exact simulator
sw/sim/tsr_sim.c - Numerical reference
sw/sim/include/tsr_ref.h - ISA definitions
sw/sim/include/tsr_isa.h - Issue-path verification
tb/issue/issue_sim.py
Developer tools
- Profile reporting
sw/tools/prof/prof_report.py - Device monitor
sw/tools/top/tsr_top.py - Device debugger
sw/tools/tsr_dbg.cpp
Other components
- llama.cpp
- PyTorch user
Connections 25
- llama.cpp → ggml backend: submits graph
README.md - ggml backend → Graph lowering: lowers graph
sw/ggml-tesseract/ggml-tesseract.cpp - ggml backend → Op-list builder: builds op list
sw/ggml-tesseract/ggml-tesseract.cpp - ggml backend → Transport protocol: launches through
README.md - ggml backend → Device memory: uses allocation
sw/ggml-tesseract/ggml-tesseract.cpp - ggml backend → Weight layout: uses weight layout
sw/ggml-tesseract/ggml-tesseract.cpp - PyTorch user → PyTorch backend: invokes backend
README.md - PyTorch backend → ggml backend: offloads matmuls
- Transport protocol → Transport API: implements API
sw/transport/tsr_kdev.cpp - Transport protocol → Simulator transport: dispatches to
- Transport protocol → Mock transport: dispatches to
- Transport protocol → XRT transport: dispatches to
- Transport protocol → Graph firmware kernel: launches kernel
README.md - Graph firmware kernel → Kernel op-list SDK: walks op list
README.md - Simulator transport → Bit-exact simulator: runs in process
- Mock transport → Bit-exact simulator: models adapter
README.md - Bit-exact simulator → Numerical reference: uses reference numerics
sw/sim/include/tsr_ref.h - Basis runtime → Transport API: uses transport
sw/rt/tsr_rt.cpp - Basis runtime → Op-list builder: builds op list
sw/rt/tsr_rt.cpp - Graph firmware kernel → ISA definitions: issues tile ops
README.md - Firmware image tool → Kernel op-list SDK: checks image ABI
sw/ctrl/tools/tsr-elf/main.cpp - Profile reporting → Transport API: profiles device
- Device monitor → Transport API: monitors device
- Device debugger → Transport API: debugs control core
- Issue-path verification → ISA definitions: models instructions