# moonshotai/nano-kpu architecture

> Architecture diagram of the GitHub repository moonshotai/nano-kpu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/moonshotai/nano-kpu
- Repository: https://github.com/moonshotai/nano-kpu

## Overview

nano-kpu is a hybrid-attention MoE inference chip, with a functional-simulation and synthesis-measurement flow. Its principal workflow is a host RUN command plus a memory image: the chip fetches token and weight data, runs token-by-token transformer inference, and writes logits, argmax results, and status to external memory before returning DONE. The model stages include KDA, MLA, attention-residual mixing, and sigmoid-routed top-2 MoE with a shared expert. The evaluation flow compares chip output with the float32 reference and reports correctness, throughput, area, and timing. Some RTL wiring is inferred from documented capabilities and module names; unsampled source limits finer-grained call relationships.

Main entry points: [`rtl/msh_chip_top.v`](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_chip_top.v) (Chip controller), [`harness/evaluate.py`](https://github.com/moonshotai/nano-kpu/blob/main/harness/evaluate.py) (Evaluation runner), [`rtl/msh_mem.v`](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_mem.v) (Memory interface), [`rtl/msh_vec.v`](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_vec.v) (Vector operations)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Interface and memory"]
  node_chip["Chip controller<br/>[msh_chip_top.v]"]
  node_fetch["Fetch engine<br/>[msh_fetch.v]"]
  node_memory["Memory interface<br/>[msh_mem.v]"]
  node_image["Packed image<br/>[memmap.py]"]
  node_sramrom["Memory macros"]
end

subgraph group_datapath["Inference datapath"]
  node_sequencer["Token sequencer<br/>[msh_seq.v]"]
  node_vector["Vector operations<br/>[msh_vec.v]"]
  node_gemv["Quantized GEMV<br/>[msh_gemv.v]"]
  node_kda["KDA attention<br/>[msh_kda.v]"]
  node_mla["MLA attention<br/>[msh_mla.v]"]
  node_moe["MoE routing<br/>[msh_deq.v]"]
  node_logits["Logit generation<br/>[msh_lg.v]"]
  node_roms[("Nonlinearity tables")]
end

subgraph group_evaluation["Model and evaluation"]
  node_fixedmodel["Fixed-point model<br/>[fxmodel.py]"]
  node_reference["Float reference<br/>[model.py]"]
  node_weights["Weight generation<br/>[gen_weights.py]"]
  node_evaluate["Evaluation runner<br/>[evaluate.py]"]
  node_scoring["Correctness metrics<br/>[scoring.py]"]
end

subgraph group_measurement["Synthesis measurement"]
  node_synthesis["Area and timing"]
end

node_host(("Host command"))
node_dram["DRAM model<br/>[tb_main.cpp]"]

node_host -->|"issues RUN"| node_chip
node_chip -->|"dispatches"| node_fetch
node_fetch -->|"requests data"| node_memory
node_memory -->|"reads and writes"| node_dram
node_dram -->|"returns reads"| node_memory
node_image -->|"loads image"| node_dram
node_memory -->|"uses macros"| node_sramrom
node_sequencer -->|"requests tokens"| node_fetch
node_chip -->|"controls decode"| node_sequencer
node_chip -->|"dispatches layer"| node_kda
node_chip -->|"dispatches layer"| node_mla
node_chip -->|"dispatches experts"| node_moe
node_kda -->|"computes projections"| node_gemv
node_mla -->|"computes projections"| node_gemv
node_moe -->|"computes experts"| node_gemv
node_kda -->|"uses operations"| node_vector
node_mla -->|"uses operations"| node_vector
node_moe -->|"uses operations"| node_vector
node_vector -->|"looks up functions"| node_roms
node_chip -->|"produces output"| node_logits
node_logits -->|"writes results"| node_memory
node_evaluate -->|"generates tensors"| node_weights
node_evaluate -->|"packs inputs"| node_image
node_evaluate -->|"computes golden output"| node_reference
node_evaluate -->|"runs simulation"| node_chip
node_evaluate -->|"compares outputs"| node_scoring
node_evaluate -->|"measures design"| node_synthesis
node_fixedmodel -->|"loads tables"| node_roms
node_reference -->|"consumes tensors"| node_weights
node_scoring -->|"compares against"| node_reference

click node_chip "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_chip_top.v"
click node_fetch "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_fetch.v"
click node_memory "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_mem.v"
click node_dram "https://github.com/moonshotai/nano-kpu/blob/main/harness/tb/tb_main.cpp"
click node_image "https://github.com/moonshotai/nano-kpu/blob/main/harness/memmap.py"
click node_sequencer "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_seq.v"
click node_vector "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_vec.v"
click node_gemv "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_gemv.v"
click node_kda "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_kda.v"
click node_mla "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_mla.v"
click node_moe "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_deq.v"
click node_logits "https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_lg.v"
click node_roms "https://github.com/moonshotai/nano-kpu/tree/main/rtl/roms"
click node_sramrom "https://github.com/moonshotai/nano-kpu/tree/main/harness/macros"
click node_fixedmodel "https://github.com/moonshotai/nano-kpu/blob/main/rtl/selfmodel/fxmodel.py"
click node_reference "https://github.com/moonshotai/nano-kpu/blob/main/reference/model.py"
click node_weights "https://github.com/moonshotai/nano-kpu/blob/main/reference/gen_weights.py"
click node_evaluate "https://github.com/moonshotai/nano-kpu/blob/main/harness/evaluate.py"
click node_scoring "https://github.com/moonshotai/nano-kpu/blob/main/harness/scoring.py"
click node_synthesis "https://github.com/moonshotai/nano-kpu/tree/main/harness"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chip,node_fetch,node_memory,node_image,node_sramrom toneBlue
class node_sequencer,node_vector,node_gemv,node_kda,node_mla,node_moe,node_logits,node_roms toneAmber
class node_fixedmodel,node_reference,node_weights,node_evaluate,node_scoring toneMint
class node_synthesis toneRose
class node_host,node_dram toneIndigo
```

## Components

### Interface and memory

- [Chip controller](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_chip_top.v): `rtl/msh_chip_top.v`
- [Fetch engine](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_fetch.v): `rtl/msh_fetch.v`
- [Memory interface](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_mem.v): `rtl/msh_mem.v`
- [Packed image](https://github.com/moonshotai/nano-kpu/blob/main/harness/memmap.py): `harness/memmap.py`
- [Memory macros](https://github.com/moonshotai/nano-kpu/tree/main/harness/macros): `harness/macros`

### Inference datapath

- [Token sequencer](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_seq.v): `rtl/msh_seq.v`
- [Vector operations](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_vec.v): `rtl/msh_vec.v`
- [Quantized GEMV](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_gemv.v): `rtl/msh_gemv.v`
- [KDA attention](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_kda.v): `rtl/msh_kda.v`
- [MLA attention](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_mla.v): `rtl/msh_mla.v`
- [MoE routing](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_deq.v): `rtl/msh_deq.v`
- [Logit generation](https://github.com/moonshotai/nano-kpu/blob/main/rtl/msh_lg.v): `rtl/msh_lg.v`
- [Nonlinearity tables](https://github.com/moonshotai/nano-kpu/tree/main/rtl/roms): `rtl/roms`

### Model and evaluation

- [Fixed-point model](https://github.com/moonshotai/nano-kpu/blob/main/rtl/selfmodel/fxmodel.py): `rtl/selfmodel/fxmodel.py`
- [Float reference](https://github.com/moonshotai/nano-kpu/blob/main/reference/model.py): `reference/model.py`
- [Weight generation](https://github.com/moonshotai/nano-kpu/blob/main/reference/gen_weights.py): `reference/gen_weights.py`
- [Evaluation runner](https://github.com/moonshotai/nano-kpu/blob/main/harness/evaluate.py): `harness/evaluate.py`
- [Correctness metrics](https://github.com/moonshotai/nano-kpu/blob/main/harness/scoring.py): `harness/scoring.py`

### Synthesis measurement

- [Area and timing](https://github.com/moonshotai/nano-kpu/tree/main/harness): `harness`

### Other components

- **Host command**
- [DRAM model](https://github.com/moonshotai/nano-kpu/blob/main/harness/tb/tb_main.cpp): `harness/tb/tb_main.cpp`

## Connections

- Host command → Chip controller: issues RUN
- Chip controller → Fetch engine: dispatches
- Fetch engine → Memory interface: requests data
- Memory interface → DRAM model: reads and writes
- DRAM model → Memory interface: returns reads
- Packed image → DRAM model: loads image
- Memory interface → Memory macros: uses macros
- Token sequencer → Fetch engine: requests tokens
- Chip controller → Token sequencer: controls decode
- Chip controller → KDA attention: dispatches layer
- Chip controller → MLA attention: dispatches layer
- Chip controller → MoE routing: dispatches experts
- KDA attention → Quantized GEMV: computes projections
- MLA attention → Quantized GEMV: computes projections
- MoE routing → Quantized GEMV: computes experts
- KDA attention → Vector operations: uses operations
- MLA attention → Vector operations: uses operations
- MoE routing → Vector operations: uses operations
- Vector operations → Nonlinearity tables: looks up functions
- Chip controller → Logit generation: produces output
- Logit generation → Memory interface: writes results
- Evaluation runner → Weight generation: generates tensors
- Evaluation runner → Packed image: packs inputs
- Evaluation runner → Float reference: computes golden output
- Evaluation runner → Chip controller: runs simulation
- Evaluation runner → Correctness metrics: compares outputs
- Evaluation runner → Area and timing: measures design
- Fixed-point model → Nonlinearity tables: loads tables
- Float reference → Weight generation: consumes tensors
- Correctness metrics → Float reference: compares against