How harishsg993010/tiny-npu works
tiny-npu is a learning-oriented, synthesizable NPU project supporting transformer inference and ONNX graph execution. The documented user workflow is to prepare model weights or compile an ONNX model, then execute it on the NPU; this tree exposes the Python preparation, compiler, and golden-reference pieces, but not the RTL implementation described in the README. The graph therefore emphasizes inspectable software behavior and documented hardware capabilities. Simulation testbenches are omitted as verification infrastructure; hardware relationships are documented rather than inferred from unavailable RTL.
Main entry points: llama_infer_golden.py LLaMA Inference · gpt2_infer_golden.py GPT-2 Inference · quant.py Quantization · gpt2_block_ref.py GPT-2 Block
Components 23
Model Preparation
- LLaMA Weights
python/tools/llama_gen_weights.py - HF Weight Export
python/tools/llama_gen_weights_hf.py - Mistral Weights
python/tools/mistral_gen_weights_hf.py - Qwen Weights
python/tools/qwen_gen_weights_hf.py - LLM Memory Layout
python/tools/llama_map.py - DDR Layout
python/tools/ddr_map.py
LLM References
- GPT-2 Inference
python/golden/gpt2_infer_golden.py - LLaMA Inference
python/golden/llama_infer_golden.py - GPT-2 Block
python/golden/gpt2_block_ref.py - INT8 GEMM
python/golden/gemm_ref.py - Fixed Softmax
python/golden/softmax_ref.py - LayerNorm
python/golden/layernorm_ref.py - GELU
python/golden/gelu_ref.py - RMSNorm
python/golden/rmsnorm_ref.py - RoPE
python/golden/rope_ref.py - SiLU
python/golden/silu_ref.py - Quantization
python/golden/quant.py
Graph Compilation
- ONNX Compiler
python/onnx_compiler/compile.py - Graph ISA
include/graph_isa.h - DDR Graph Interface
include/ddr_graph.h
NPU Hardware
- NPU Execution
Other components
- Model Provider
- ONNX Model
Connections 37
- Model Provider → HF Weight Export: provides weights
python/tools/llama_gen_weights_hf.py - Model Provider → Mistral Weights: provides weights
python/tools/mistral_gen_weights_hf.py - Model Provider → Qwen Weights: provides weights
python/tools/qwen_gen_weights_hf.py - LLaMA Weights → LLM Memory Layout: uses layout
python/tools/llama_gen_weights.py - HF Weight Export → LLM Memory Layout: uses layout
python/tools/llama_gen_weights_hf.py - Mistral Weights → LLM Memory Layout: uses layout
python/tools/mistral_gen_weights_hf.py - Qwen Weights → LLM Memory Layout: uses layout
python/tools/qwen_gen_weights_hf.py - HF Weight Export → LLaMA Inference: runs golden inference
python/tools/llama_gen_weights_hf.py - Mistral Weights → LLaMA Inference: runs golden inference
python/tools/mistral_gen_weights_hf.py - Qwen Weights → LLaMA Inference: runs golden inference
python/tools/qwen_gen_weights_hf.py - GPT-2 Inference → DDR Layout: uses layout
python/golden/gpt2_infer_golden.py - GPT-2 Inference → INT8 GEMM: calls
python/golden/gpt2_infer_golden.py - GPT-2 Inference → Fixed Softmax: calls
python/golden/gpt2_infer_golden.py - GPT-2 Inference → LayerNorm: calls
python/golden/gpt2_infer_golden.py - GPT-2 Inference → GELU: calls
python/golden/gpt2_infer_golden.py - GPT-2 Inference → Quantization: uses arithmetic
python/golden/gpt2_infer_golden.py - GPT-2 Block → INT8 GEMM: calls
python/golden/gpt2_block_ref.py - GPT-2 Block → Fixed Softmax: calls
python/golden/gpt2_block_ref.py - GPT-2 Block → LayerNorm: calls
python/golden/gpt2_block_ref.py - GPT-2 Block → GELU: calls
python/golden/gpt2_block_ref.py - LLaMA Inference → INT8 GEMM: calls
python/golden/llama_infer_golden.py - LLaMA Inference → Fixed Softmax: calls
python/golden/llama_infer_golden.py - LLaMA Inference → RMSNorm: calls
python/golden/llama_infer_golden.py - LLaMA Inference → RoPE: calls
python/golden/llama_infer_golden.py - LLaMA Inference → SiLU: calls
python/golden/llama_infer_golden.py - LLaMA Inference → Quantization: uses arithmetic
python/golden/llama_infer_golden.py - INT8 GEMM → Quantization: uses arithmetic
python/golden/gemm_ref.py - Fixed Softmax → Quantization: uses arithmetic
python/golden/softmax_ref.py - LayerNorm → Quantization: uses arithmetic
python/golden/layernorm_ref.py - RMSNorm → Quantization: uses arithmetic
python/golden/rmsnorm_ref.py - GELU → Quantization: uses arithmetic
python/golden/gelu_ref.py - ONNX Model → ONNX Compiler: compiles
README.md - ONNX Compiler → Graph ISA: emits instructions
- Graph ISA → NPU Execution: executes graph
README.md - DDR Graph Interface → NPU Execution: defines interface
- LLaMA Inference → NPU Execution: models inference
README.md - ONNX Compiler → NPU Execution: targets graph mode
README.md