# harishsg993010/tiny-npu architecture

> Architecture diagram of the GitHub repository harishsg993010/tiny-npu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harishsg993010/tiny-npu
- Repository: https://github.com/harishsg993010/tiny-npu

## Overview

tiny-npu is a learning-oriented, synthesizable NPU project supporting transformer inference and ONNX graph execution. The documented user workflow is to prepare model weights or compile an ONNX model, then execute it on the NPU; this tree exposes the Python preparation, compiler, and golden-reference pieces, but not the RTL implementation described in the README. The graph therefore emphasizes inspectable software behavior and documented hardware capabilities. Simulation testbenches are omitted as verification infrastructure; hardware relationships are documented rather than inferred from unavailable RTL.

Main entry points: [`python/golden/llama_infer_golden.py`](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/llama_infer_golden.py) (LLaMA Inference), [`python/golden/gpt2_infer_golden.py`](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gpt2_infer_golden.py) (GPT-2 Inference), [`python/golden/quant.py`](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/quant.py) (Quantization), [`python/golden/gpt2_block_ref.py`](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gpt2_block_ref.py) (GPT-2 Block)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Model Preparation"]
  node_llama_gen["LLaMA Weights"]
  node_hf_gen["HF Weight Export"]
  node_mistral_gen["Mistral Weights"]
  node_qwen_gen["Qwen Weights"]
  node_llama_map["LLM Memory Layout<br/>[llama_map.py]"]
  node_ddr_map["DDR Layout<br/>[ddr_map.py]"]
end

subgraph group_llm["LLM References"]
  node_gpt2_ref["GPT-2 Inference"]
  node_llama_ref["LLaMA Inference"]
  node_gpt2_block["GPT-2 Block<br/>[gpt2_block_ref.py]"]
  node_gemm["INT8 GEMM<br/>[gemm_ref.py]"]
  node_softmax["Fixed Softmax<br/>[softmax_ref.py]"]
  node_layernorm["LayerNorm<br/>[layernorm_ref.py]"]
  node_gelu["GELU<br/>[gelu_ref.py]"]
  node_rmsnorm["RMSNorm<br/>[rmsnorm_ref.py]"]
  node_rope["RoPE<br/>[rope_ref.py]"]
  node_silu["SiLU<br/>[silu_ref.py]"]
  node_quant["Quantization<br/>[quant.py]"]
end

subgraph group_graph["Graph Compilation"]
  node_onnx_compile["ONNX Compiler<br/>[compile.py]"]
  node_graph_isa["Graph ISA<br/>[graph_isa.h]"]
  node_ddr_isa["DDR Graph Interface<br/>[ddr_graph.h]"]
end

subgraph group_hw["NPU Hardware"]
  node_npu{{"NPU Execution"}}
end

node_model_author(("Model Provider"))
node_onnx_model["ONNX Model"]

node_model_author -->|"provides weights"| node_hf_gen
node_model_author -->|"provides weights"| node_mistral_gen
node_model_author -->|"provides weights"| node_qwen_gen
node_llama_gen -->|"uses layout"| node_llama_map
node_hf_gen -->|"uses layout"| node_llama_map
node_mistral_gen -->|"uses layout"| node_llama_map
node_qwen_gen -->|"uses layout"| node_llama_map
node_hf_gen -->|"runs golden inference"| node_llama_ref
node_mistral_gen -->|"runs golden inference"| node_llama_ref
node_qwen_gen -->|"runs golden inference"| node_llama_ref
node_gpt2_ref -->|"uses layout"| node_ddr_map
node_gpt2_ref -->|"calls"| node_gemm
node_gpt2_ref -->|"calls"| node_softmax
node_gpt2_ref -->|"calls"| node_layernorm
node_gpt2_ref -->|"calls"| node_gelu
node_gpt2_ref -->|"uses arithmetic"| node_quant
node_gpt2_block -->|"calls"| node_gemm
node_gpt2_block -->|"calls"| node_softmax
node_gpt2_block -->|"calls"| node_layernorm
node_gpt2_block -->|"calls"| node_gelu
node_llama_ref -->|"calls"| node_gemm
node_llama_ref -->|"calls"| node_softmax
node_llama_ref -->|"calls"| node_rmsnorm
node_llama_ref -->|"calls"| node_rope
node_llama_ref -->|"calls"| node_silu
node_llama_ref -->|"uses arithmetic"| node_quant
node_gemm -->|"uses arithmetic"| node_quant
node_softmax -->|"uses arithmetic"| node_quant
node_layernorm -->|"uses arithmetic"| node_quant
node_rmsnorm -->|"uses arithmetic"| node_quant
node_gelu -->|"uses arithmetic"| node_quant
node_onnx_model -->|"compiles"| node_onnx_compile
node_onnx_compile -.->|"emits instructions"| node_graph_isa
node_graph_isa -->|"executes graph"| node_npu
node_ddr_isa -.->|"defines interface"| node_npu
node_llama_ref -->|"models inference"| node_npu
node_onnx_compile -->|"targets graph mode"| node_npu

click node_llama_gen "https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/llama_gen_weights.py"
click node_hf_gen "https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/llama_gen_weights_hf.py"
click node_mistral_gen "https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/mistral_gen_weights_hf.py"
click node_qwen_gen "https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/qwen_gen_weights_hf.py"
click node_llama_map "https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/llama_map.py"
click node_ddr_map "https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/ddr_map.py"
click node_gpt2_ref "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gpt2_infer_golden.py"
click node_llama_ref "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/llama_infer_golden.py"
click node_gpt2_block "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gpt2_block_ref.py"
click node_gemm "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gemm_ref.py"
click node_softmax "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/softmax_ref.py"
click node_layernorm "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/layernorm_ref.py"
click node_gelu "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gelu_ref.py"
click node_rmsnorm "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/rmsnorm_ref.py"
click node_rope "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/rope_ref.py"
click node_silu "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/silu_ref.py"
click node_quant "https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/quant.py"
click node_onnx_compile "https://github.com/harishsg993010/tiny-npu/blob/main/python/onnx_compiler/compile.py"
click node_graph_isa "https://github.com/harishsg993010/tiny-npu/blob/main/include/graph_isa.h"
click node_ddr_isa "https://github.com/harishsg993010/tiny-npu/blob/main/include/ddr_graph.h"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_llama_gen,node_hf_gen,node_mistral_gen,node_qwen_gen,node_llama_map,node_ddr_map toneBlue
class node_gpt2_ref,node_llama_ref,node_gpt2_block,node_gemm,node_softmax,node_layernorm,node_gelu,node_rmsnorm,node_rope,node_silu,node_quant toneAmber
class node_onnx_compile,node_graph_isa,node_ddr_isa toneMint
class node_npu toneRose
class node_model_author,node_onnx_model toneIndigo
```

## Components

### Model Preparation

- [LLaMA Weights](https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/llama_gen_weights.py): `python/tools/llama_gen_weights.py`
- [HF Weight Export](https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/llama_gen_weights_hf.py): `python/tools/llama_gen_weights_hf.py`
- [Mistral Weights](https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/mistral_gen_weights_hf.py): `python/tools/mistral_gen_weights_hf.py`
- [Qwen Weights](https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/qwen_gen_weights_hf.py): `python/tools/qwen_gen_weights_hf.py`
- [LLM Memory Layout](https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/llama_map.py): `python/tools/llama_map.py`
- [DDR Layout](https://github.com/harishsg993010/tiny-npu/blob/main/python/tools/ddr_map.py): `python/tools/ddr_map.py`

### LLM References

- [GPT-2 Inference](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gpt2_infer_golden.py): `python/golden/gpt2_infer_golden.py`
- [LLaMA Inference](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/llama_infer_golden.py): `python/golden/llama_infer_golden.py`
- [GPT-2 Block](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gpt2_block_ref.py): `python/golden/gpt2_block_ref.py`
- [INT8 GEMM](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gemm_ref.py): `python/golden/gemm_ref.py`
- [Fixed Softmax](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/softmax_ref.py): `python/golden/softmax_ref.py`
- [LayerNorm](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/layernorm_ref.py): `python/golden/layernorm_ref.py`
- [GELU](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/gelu_ref.py): `python/golden/gelu_ref.py`
- [RMSNorm](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/rmsnorm_ref.py): `python/golden/rmsnorm_ref.py`
- [RoPE](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/rope_ref.py): `python/golden/rope_ref.py`
- [SiLU](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/silu_ref.py): `python/golden/silu_ref.py`
- [Quantization](https://github.com/harishsg993010/tiny-npu/blob/main/python/golden/quant.py): `python/golden/quant.py`

### Graph Compilation

- [ONNX Compiler](https://github.com/harishsg993010/tiny-npu/blob/main/python/onnx_compiler/compile.py): `python/onnx_compiler/compile.py`
- [Graph ISA](https://github.com/harishsg993010/tiny-npu/blob/main/include/graph_isa.h): `include/graph_isa.h`
- [DDR Graph Interface](https://github.com/harishsg993010/tiny-npu/blob/main/include/ddr_graph.h): `include/ddr_graph.h`

### NPU Hardware

- **NPU Execution**

### Other components

- **Model Provider**
- **ONNX Model**

## Connections

- Model Provider → HF Weight Export: provides weights (evidence: `python/tools/llama_gen_weights_hf.py`)
- Model Provider → Mistral Weights: provides weights (evidence: `python/tools/mistral_gen_weights_hf.py`)
- Model Provider → Qwen Weights: provides weights (evidence: `python/tools/qwen_gen_weights_hf.py`)
- LLaMA Weights → LLM Memory Layout: uses layout (evidence: `python/tools/llama_gen_weights.py`)
- HF Weight Export → LLM Memory Layout: uses layout (evidence: `python/tools/llama_gen_weights_hf.py`)
- Mistral Weights → LLM Memory Layout: uses layout (evidence: `python/tools/mistral_gen_weights_hf.py`)
- Qwen Weights → LLM Memory Layout: uses layout (evidence: `python/tools/qwen_gen_weights_hf.py`)
- HF Weight Export → LLaMA Inference: runs golden inference (evidence: `python/tools/llama_gen_weights_hf.py`)
- Mistral Weights → LLaMA Inference: runs golden inference (evidence: `python/tools/mistral_gen_weights_hf.py`)
- Qwen Weights → LLaMA Inference: runs golden inference (evidence: `python/tools/qwen_gen_weights_hf.py`)
- GPT-2 Inference → DDR Layout: uses layout (evidence: `python/golden/gpt2_infer_golden.py`)
- GPT-2 Inference → INT8 GEMM: calls (evidence: `python/golden/gpt2_infer_golden.py`)
- GPT-2 Inference → Fixed Softmax: calls (evidence: `python/golden/gpt2_infer_golden.py`)
- GPT-2 Inference → LayerNorm: calls (evidence: `python/golden/gpt2_infer_golden.py`)
- GPT-2 Inference → GELU: calls (evidence: `python/golden/gpt2_infer_golden.py`)
- GPT-2 Inference → Quantization: uses arithmetic (evidence: `python/golden/gpt2_infer_golden.py`)
- GPT-2 Block → INT8 GEMM: calls (evidence: `python/golden/gpt2_block_ref.py`)
- GPT-2 Block → Fixed Softmax: calls (evidence: `python/golden/gpt2_block_ref.py`)
- GPT-2 Block → LayerNorm: calls (evidence: `python/golden/gpt2_block_ref.py`)
- GPT-2 Block → GELU: calls (evidence: `python/golden/gpt2_block_ref.py`)
- LLaMA Inference → INT8 GEMM: calls (evidence: `python/golden/llama_infer_golden.py`)
- LLaMA Inference → Fixed Softmax: calls (evidence: `python/golden/llama_infer_golden.py`)
- LLaMA Inference → RMSNorm: calls (evidence: `python/golden/llama_infer_golden.py`)
- LLaMA Inference → RoPE: calls (evidence: `python/golden/llama_infer_golden.py`)
- LLaMA Inference → SiLU: calls (evidence: `python/golden/llama_infer_golden.py`)
- LLaMA Inference → Quantization: uses arithmetic (evidence: `python/golden/llama_infer_golden.py`)
- INT8 GEMM → Quantization: uses arithmetic (evidence: `python/golden/gemm_ref.py`)
- Fixed Softmax → Quantization: uses arithmetic (evidence: `python/golden/softmax_ref.py`)
- LayerNorm → Quantization: uses arithmetic (evidence: `python/golden/layernorm_ref.py`)
- RMSNorm → Quantization: uses arithmetic (evidence: `python/golden/rmsnorm_ref.py`)
- GELU → Quantization: uses arithmetic (evidence: `python/golden/gelu_ref.py`)
- ONNX Model → ONNX Compiler: compiles (evidence: `README.md`)
- ONNX Compiler → Graph ISA: emits instructions
- Graph ISA → NPU Execution: executes graph (evidence: `README.md`)
- DDR Graph Interface → NPU Execution: defines interface
- LLaMA Inference → NPU Execution: models inference (evidence: `README.md`)
- ONNX Compiler → NPU Execution: targets graph mode (evidence: `README.md`)
