# fguzman82/gategpt architecture

> Architecture diagram of the GitHub repository fguzman82/gategpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fguzman82/gategpt
- Repository: https://github.com/fguzman82/gategpt

## Overview

1. **What this repository is**
   - `gateGPT` is a complete FPGA implementation of a tiny character-level GPT, not a software model.
   - The target is a Xilinx Virtex-5 XC5VLX110T on the XUPV5 / ML509 board, and the design is written in Verilog-2001.

2. **Top-level runtime boundary**
   - The board top is `board/xupv5_microgpt_top.v`.
   - It connects three major regions:
     - the inference core (`core/microgpt_core.v`)
     - the user interface/peripherals (`lcd_hd44780.v`, `rotary_throttle.v`, `tok_meter.v`, `name_generator.v`)
     - board-specific clocking and pin constraints (`board/xupv5_microgpt.ucf`)
   - At runtime, the core produces one next character token at a time; the board logic displays the current generated name and lets the user change speed and sampling temperature.

3. **Core architecture: microcode-controlled datapath**
   - The core is not a monolithic FSM. It is a **microcode sequencer**.
   - `core/ucode_rom.vh` contains the micro-ops that define the transformer execution schedule.
   - `core/coremap.vh` maps those micro-ops to actuator modules.
   - The sequencer fetches one micro-op, starts exactly one datapath actuator, and waits for its `done` signal before advancing.

4. **Main compute blocks inside `core/`**
   - `embed.v`: token and position embedding lookup.
   - `norm.v`: RMSNorm.
   - `matvec.v`: dense linear algebra / projection engine.
   - `attn.v`: causal attention for the current position over cached context.
   - `vecop.v`: elementwise vector ops such as residual add and ReLU.
   - `sampler.v`: temperature-based softmax sampling or greedy selection.
   - Supporting math blocks include `exp.v`, `udiv.v`, `isqrt.v`, `grom.v`, `wrom.v`, `vmem.v`, and `vmem2.v`.

5. **Data movement and memory boundaries**
   - The main working memory is a **true dual-port scratchpad BRAM** implemented by `vmem` / `vmem2`.
   - That same memory also stores the persistent **KV cache**, which is what makes incremental decoding fast.
   - Weight constants live in wide ROMs (`wrom`, `grom`) and are read-only during inference.
   - The core computes only the new token’s K/V each step and reuses the cached context instead of recomputing the entire window.

6. **Model shape and numeric format**
   - The model is tiny and fixed: one transformer block, 4 heads, head dim 6, embedding size 24, MLP hidden size 96, context length 16, vocabulary size 27.
   - All arithmetic is signed **Q5.11 fixed point**.
   - The fixed-point behavior is defined by the Python reference in `tools/fixedpoint.py`, and the RTL is intended to be bit-exact to it.

7. **Python-side training/export pipeline**
   - `data/names.txt` is the training corpus.
   - `tools/train.py` trains the model in Python/PyTorch.
   - `tools/export.py` converts trained weights into hardware-ready artifacts in `generated/` and also emits parameter headers such as `core/core_params.vh`.
   - `tools/ucode_asm.py` assembles the microcode program into `generated/ucode.hex` and `core/coremap.vh`.
   - `tools/check_quant.py`, `tools/model.py`, and `tools/dump_*.py` support quantization checking and model inspection.

8. **Generated artifacts are part of the hardware interface**
   - The RTL consumes precomputed constants from `generated/`:
     - weight ROM contents like `wq_t.hex`, `wk_t.hex`, `wv_t.hex`, `wo_t.hex`, `fc1_t.hex`, `fc2_t.hex`, `lm_t.hex`
     - embeddings like `tok_embed.hex` and `pos_embed.hex`
     - microcode in `ucode.hex`
     - test vectors for simulation
   - These artifacts are effectively the boundary between the Python training/export flow and the RTL inference flow.

9. **Verification architecture**
   - `sim/` contains iSim testbenches for both individual units and the full core:
     - `tb_mathops.v`, `tb_exp.v`, `tb_norm.v`, `tb_matvec.v`, `tb_attn.v`
     - `tb_core.v` for end-to-end verification
   - `sim/isim_run.tcl` drives the simulator and compares hardware behavior against the golden reference.
   - This repo treats bit-exact simulation as the correctness gate before generating a board bitstream.

10. **Board-facing user experience**
   - `board/name_generator.v` orchestrates the token generation loop on the FPGA board.
   - `rotary_throttle.v` lets the user change either generation rate or temperature.
   - `tok_meter.v` measures and displays throughput.
   - `lcd_hd44780.v` drives the character LCD that shows the current generated name and settings.

11. **Build and deployment flow**
   - `build_board_ise_project.tcl` and `run_board_bitgen.tcl` support the Xilinx ISE 14.7 toolchain.
   - The target flow is classic FPGA synthesis/place-and-route/bitgen for the Virtex-5 board.
   - The top-level hardware artifact is the board bitstream, while the intermediate Python-generated hex/VH files are required inputs to synthesis.

12. **Important architectural constraint**
   - The design is resource-driven: DSPs are the main compute bottleneck, while BRAM is used sparingly for the scratchpad/KV cache.
   - The architecture is therefore centered on reusing a small set of modular actuators under microcode control, rather than instantiating a large fully parallel transformer datapath.

13. **How to draw the system diagram**
   - Put `tools/` and `data/` on the left as the offline training/export pipeline.
   - Put `generated/` as the artifact boundary feeding `core/`.
   - Place `core/microgpt_core.v` in the center with the sequencer, shared scratchpad, weight ROMs, and actuators.
   - Put `board/` on the right as the runtime interface to LCD, rotary input, and throughput display.
   - Add `sim/` as the verification path that validates the same generated artifacts and RTL before hardware deployment.

Main entry points: [`core/ucode_rom.vh`](https://github.com/fguzman82/gategpt/blob/main/core/ucode_rom.vh) (Ucode ROM), [`core/microgpt_core.v`](https://github.com/fguzman82/gategpt/blob/main/core/microgpt_core.v) (Core), [`generated`](https://github.com/fguzman82/gategpt/tree/main/generated) (Generated ROMs), [`board/name_generator.v`](https://github.com/fguzman82/gategpt/blob/main/board/name_generator.v) (Name gen)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_offline["Offline pipeline"]
  node_tools_train["Train<br/>[train.py]"]
  node_tools_export["Export<br/>[export.py]"]
  node_tools_ucode["Ucode asm<br/>microcode assembler<br/>[ucode_asm.py]"]
  node_tools_fixed["Fixed point<br/>numeric reference<br/>[fixedpoint.py]"]
end

subgraph group_artifacts["Artifacts"]
  node_generated_artifacts[("Generated ROMs<br/>hex/vh artifacts")]
end

subgraph group_core["FPGA core"]
  node_core_top["Core<br/>microcode sequencer<br/>[microgpt_core.v]"]
  node_core_ucode["Ucode ROM<br/>micro-op schedule<br/>[ucode_rom.vh]"]
  node_core_map["Coremap<br/>actuator map<br/>[coremap.vh]"]
  node_core_mem[("Scratchpad<br/>dual-port BRAM<br/>[vmem.v]")]
  node_core_wrom[("Weight ROMs<br/>constant storage<br/>[wrom.v]")]
  node_core_embed["Embed<br/>[embed.v]"]
  node_core_norm["Norm<br/>[norm.v]"]
  node_core_matvec["Matvec<br/>linear algebra<br/>[matvec.v]"]
  node_core_attn["Attention<br/>[attn.v]"]
  node_core_vecop["Vecops<br/>elementwise ops<br/>[vecop.v]"]
  node_core_sampler["Sampler<br/>[sampler.v]"]
end

subgraph group_board["Board runtime"]
  node_board_top["Board top<br/>FPGA top module"]
  node_board_name["Name gen<br/>runtime controller<br/>[name_generator.v]"]
  node_board_lcd["LCD<br/>HD44780 driver<br/>[lcd_hd44780.v]"]
  node_board_throttle["Throttle<br/>rotary input<br/>[rotary_throttle.v]"]
  node_board_meter["Meter<br/>throughput display<br/>[tok_meter.v]"]
end

subgraph group_sim["Verification"]
  node_sim_core["Core TB<br/>end-to-end testbench<br/>[tb_core.v]"]
  node_sim_units["Unit TBs<br/>module testbenches<br/>[tb_matvec.v]"]
  node_sim_run["Sim flow<br/>isim driver<br/>[isim_run.tcl]"]
end

node_tools_train -->|"trained weights"| node_tools_export
node_tools_fixed -->|"numeric rules"| node_tools_export
node_tools_export -->|"hex/vh"| node_generated_artifacts
node_tools_ucode -->|"ucode"| node_generated_artifacts
node_generated_artifacts -->|"inputs"| node_core_top
node_generated_artifacts -->|"golden data"| node_sim_run
node_core_top -->|"fetch"| node_core_ucode
node_core_top -->|"dispatch"| node_core_map
node_core_top -->|"scratchpad"| node_core_mem
node_core_top -->|"weights"| node_core_wrom
node_core_ucode -->|"step"| node_core_embed
node_core_ucode -->|"step"| node_core_norm
node_core_ucode -->|"step"| node_core_matvec
node_core_ucode -->|"step"| node_core_attn
node_core_ucode -->|"step"| node_core_vecop
node_core_ucode -->|"step"| node_core_sampler
node_core_embed -->|"tokens"| node_core_mem
node_core_norm -->|"activations"| node_core_mem
node_core_matvec -->|"projections"| node_core_wrom
node_core_attn -->|"kv cache"| node_core_mem
node_core_vecop -->|"residuals"| node_core_mem
node_core_sampler -->|"token"| node_board_name
node_board_top -->|"inference"| node_core_top
node_board_top -->|"control"| node_board_name
node_board_name -->|"display"| node_board_lcd
node_board_throttle -->|"user input"| node_board_name
node_board_name -->|"rate"| node_board_meter
node_sim_core -->|"vectors"| node_generated_artifacts
node_sim_units -->|"vectors"| node_generated_artifacts
node_sim_run -->|"run"| node_sim_core
node_sim_run -->|"run"| node_sim_units

click node_tools_train "https://github.com/fguzman82/gategpt/blob/main/tools/train.py"
click node_tools_export "https://github.com/fguzman82/gategpt/blob/main/tools/export.py"
click node_tools_ucode "https://github.com/fguzman82/gategpt/blob/main/tools/ucode_asm.py"
click node_tools_fixed "https://github.com/fguzman82/gategpt/blob/main/tools/fixedpoint.py"
click node_generated_artifacts "https://github.com/fguzman82/gategpt/tree/main/generated"
click node_core_top "https://github.com/fguzman82/gategpt/blob/main/core/microgpt_core.v"
click node_core_ucode "https://github.com/fguzman82/gategpt/blob/main/core/ucode_rom.vh"
click node_core_map "https://github.com/fguzman82/gategpt/blob/main/core/coremap.vh"
click node_core_mem "https://github.com/fguzman82/gategpt/blob/main/core/vmem.v"
click node_core_wrom "https://github.com/fguzman82/gategpt/blob/main/core/wrom.v"
click node_core_embed "https://github.com/fguzman82/gategpt/blob/main/core/embed.v"
click node_core_norm "https://github.com/fguzman82/gategpt/blob/main/core/norm.v"
click node_core_matvec "https://github.com/fguzman82/gategpt/blob/main/core/matvec.v"
click node_core_attn "https://github.com/fguzman82/gategpt/blob/main/core/attn.v"
click node_core_vecop "https://github.com/fguzman82/gategpt/blob/main/core/vecop.v"
click node_core_sampler "https://github.com/fguzman82/gategpt/blob/main/core/sampler.v"
click node_board_top "https://github.com/fguzman82/gategpt/blob/main/board/xupv5_microgpt_top.v"
click node_board_name "https://github.com/fguzman82/gategpt/blob/main/board/name_generator.v"
click node_board_lcd "https://github.com/fguzman82/gategpt/blob/main/board/lcd_hd44780.v"
click node_board_throttle "https://github.com/fguzman82/gategpt/blob/main/board/rotary_throttle.v"
click node_board_meter "https://github.com/fguzman82/gategpt/blob/main/board/tok_meter.v"
click node_sim_core "https://github.com/fguzman82/gategpt/blob/main/sim/tb_core.v"
click node_sim_units "https://github.com/fguzman82/gategpt/blob/main/sim/tb_matvec.v"
click node_sim_run "https://github.com/fguzman82/gategpt/blob/main/sim/isim_run.tcl"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tools_train,node_tools_export,node_tools_ucode,node_tools_fixed toneBlue
class node_generated_artifacts toneAmber
class node_core_top,node_core_ucode,node_core_map,node_core_mem,node_core_wrom,node_core_embed,node_core_norm,node_core_matvec,node_core_attn,node_core_vecop,node_core_sampler toneMint
class node_board_top,node_board_name,node_board_lcd,node_board_throttle,node_board_meter toneRose
class node_sim_core,node_sim_units,node_sim_run toneIndigo
```

## Components

### Offline pipeline

Python training, export, and microcode assembly

- [Train](https://github.com/fguzman82/gategpt/blob/main/tools/train.py): PyTorch training · Trains the tiny character model from the names corpus · `tools/train.py`
- [Export](https://github.com/fguzman82/gategpt/blob/main/tools/export.py): artifact export · Quantizes weights and emits hardware-ready hex and headers · `tools/export.py`
- [Ucode asm](https://github.com/fguzman82/gategpt/blob/main/tools/ucode_asm.py): microcode assembler · Assembles the sequencer program and coremap · `tools/ucode_asm.py`
- [Fixed point](https://github.com/fguzman82/gategpt/blob/main/tools/fixedpoint.py): numeric reference · Defines the Q5.11 reference behavior used by RTL and export · `tools/fixedpoint.py`

### Artifacts

Generated hardware inputs and golden vectors

- [Generated ROMs](https://github.com/fguzman82/gategpt/tree/main/generated): hex/vh artifacts · Weights, embeddings, microcode, and test vectors consumed by RTL · `generated`

### FPGA core

Microcode-controlled transformer datapath

- [Core](https://github.com/fguzman82/gategpt/blob/main/core/microgpt_core.v): microcode sequencer · Top-level inference engine for one token step at a time · `core/microgpt_core.v`
- [Ucode ROM](https://github.com/fguzman82/gategpt/blob/main/core/ucode_rom.vh): micro-op schedule · Stores the execution schedule for each transformer step · `core/ucode_rom.vh`
- [Coremap](https://github.com/fguzman82/gategpt/blob/main/core/coremap.vh): actuator map · Maps micro-ops to datapath actuator modules · `core/coremap.vh`
- [Scratchpad](https://github.com/fguzman82/gategpt/blob/main/core/vmem.v): dual-port BRAM · Working memory and incremental KV cache · `core/vmem.v`
- [Weight ROMs](https://github.com/fguzman82/gategpt/blob/main/core/wrom.v): constant storage · Read-only model weights and embedding tables · `core/wrom.v`
- [Embed](https://github.com/fguzman82/gategpt/blob/main/core/embed.v): token/pos embed · Looks up token and position embeddings · `core/embed.v`
- [Norm](https://github.com/fguzman82/gategpt/blob/main/core/norm.v): RMSNorm · Normalizes activations before projections · `core/norm.v`
- [Matvec](https://github.com/fguzman82/gategpt/blob/main/core/matvec.v): linear algebra · Dense projection and MLP matrix-vector engine · `core/matvec.v`
- [Attention](https://github.com/fguzman82/gategpt/blob/main/core/attn.v): causal attention · Computes current-token attention over cached context · `core/attn.v`
- [Vecops](https://github.com/fguzman82/gategpt/blob/main/core/vecop.v): elementwise ops · Residual add and activation-style vector operations · `core/vecop.v`
- [Sampler](https://github.com/fguzman82/gategpt/blob/main/core/sampler.v): token sampler · Temperature or greedy next-token selection · `core/sampler.v`

### Board runtime

FPGA top-level and user interface peripherals

- [Board top](https://github.com/fguzman82/gategpt/blob/main/board/xupv5_microgpt_top.v): FPGA top module · Connects core, UI peripherals, and board pins · `board/xupv5_microgpt_top.v`
- [Name gen](https://github.com/fguzman82/gategpt/blob/main/board/name_generator.v): runtime controller · Runs the generation loop and user-facing output · `board/name_generator.v`
- [LCD](https://github.com/fguzman82/gategpt/blob/main/board/lcd_hd44780.v): HD44780 driver · Drives the character display for the generated name · `board/lcd_hd44780.v`
- [Throttle](https://github.com/fguzman82/gategpt/blob/main/board/rotary_throttle.v): rotary input · Adjusts generation rate and sampling temperature · `board/rotary_throttle.v`
- [Meter](https://github.com/fguzman82/gategpt/blob/main/board/tok_meter.v): throughput display · Measures and shows token generation rate · `board/tok_meter.v`

### Verification

RTL testbenches and simulator flow

- [Core TB](https://github.com/fguzman82/gategpt/blob/main/sim/tb_core.v): end-to-end testbench · Verifies the full inference core against golden behavior · `sim/tb_core.v`
- [Unit TBs](https://github.com/fguzman82/gategpt/blob/main/sim/tb_matvec.v): module testbenches · Checks math and transformer submodules in isolation · `sim/tb_matvec.v`
- [Sim flow](https://github.com/fguzman82/gategpt/blob/main/sim/isim_run.tcl): isim driver · Runs simulator comparisons against reference vectors · `sim/isim_run.tcl`

## Connections

- Train → Export: trained weights · Training produces model parameters for export
- Fixed point → Export: numeric rules · Export uses the fixed-point reference for quantization
- Export → Generated ROMs: hex/vh · Writes ROM images and parameter headers
- Ucode asm → Generated ROMs: ucode · Assembles micro-op program and related maps
- Generated ROMs → Core: inputs · Hardware consumes precomputed constants and test vectors
- Generated ROMs → Sim flow: golden data · Simulation compares RTL against exported vectors
- Core → Ucode ROM: fetch · Sequencer reads the next micro-op
- Core → Coremap: dispatch · Micro-ops select actuator modules
- Core → Scratchpad: scratchpad · Core reads and writes working state and KV cache
- Core → Weight ROMs: weights · Core streams read-only model constants
- Ucode ROM → Embed: step · Embedding is one actuator in the microcode schedule
- Ucode ROM → Norm: step · Normalization is one actuator in the microcode schedule
- Ucode ROM → Matvec: step · Projection and MLP are scheduled as actuators
- Ucode ROM → Attention: step · Attention is scheduled as an actuator
- Ucode ROM → Vecops: step · Vector ops are scheduled as actuators
- Ucode ROM → Sampler: step · Sampling is the final actuator before output
- Embed → Scratchpad: tokens · Writes embedded state into the shared scratchpad
- Norm → Scratchpad: activations · Consumes and updates intermediate vectors
- Matvec → Weight ROMs: projections · Reads linear layer weights and embedding tables
- Attention → Scratchpad: kv cache · Reads cached context and appends new K/V
- Vecops → Scratchpad: residuals · Updates the shared working vectors
- Sampler → Name gen: token · Produces the next character for the board runtime
- Board top → Core: inference · Top-level board wrapper instantiates the core
- Board top → Name gen: control · Board logic orchestrates generation timing and UI state
- Name gen → LCD: display · Sends the current generated name and settings to LCD
- Throttle → Name gen: user input · User changes speed or temperature
- Name gen → Meter: rate · Feeds throughput measurements for display
- Core TB → Generated ROMs: vectors · Uses exported constants and test files for end-to-end checks
- Unit TBs → Generated ROMs: vectors · Unit testbenches compare module outputs against golden data
- Sim flow → Core TB: run · Simulation flow drives the core testbench
- Sim flow → Unit TBs: run · Simulation flow drives the unit testbenches