How fguzman82/gategpt works
1. What this repository is
- gateGPT is a complete FPGA implementation of a tiny character-level GPT, not a software model.
- The target is a Xilinx Virtex-5 XC5VLX110T on the XUPV5 / ML509 board, and the design is written in Verilog-2001.
2. Top-level runtime boundary
- The board top is board/xupv5_microgpt_top.v.
- It connects three major regions:
- the inference core (core/microgpt_core.v)
- the user interface/peripherals (lcd_hd44780.v, rotary_throttle.v, tok_meter.v, name_generator.v)
- board-specific clocking and pin constraints (board/xupv5_microgpt.ucf)
- At runtime, the core produces one next character token at a time; the board logic displays the current generated name and lets the user change speed and sampling temperature.
3. Core architecture: microcode-controlled datapath
- The core is not a monolithic FSM. It is a microcode sequencer.
- core/ucode_rom.vh contains the micro-ops that define the transformer execution schedule.
- core/coremap.vh maps those micro-ops to actuator modules.
- The sequencer fetches one micro-op, starts exactly one datapath actuator, and waits for its done signal before advancing.
4. Main compute blocks inside core/
- embed.v: token and position embedding lookup.
- norm.v: RMSNorm.
- matvec.v: dense linear algebra / projection engine.
- attn.v: causal attention for the current position over cached context.
- vecop.v: elementwise vector ops such as residual add and ReLU.
- sampler.v: temperature-based softmax sampling or greedy selection.
- Supporting math blocks include exp.v, udiv.v, isqrt.v, grom.v, wrom.v, vmem.v, and vmem2.v.
5. Data movement and memory boundaries
- The main working memory is a true dual-port scratchpad BRAM implemented by vmem / vmem2.
- That same memory also stores the persistent KV cache, which is what makes incremental decoding fast.
- Weight constants live in wide ROMs (wrom, grom) and are read-only during inference.
- The core computes only the new token’s K/V each step and reuses the cached context instead of recomputing the entire window.
6. Model shape and numeric format
- The model is tiny and fixed: one transformer block, 4 heads, head dim 6, embedding size 24, MLP hidden size 96, context length 16, vocabulary size 27.
- All arithmetic is signed Q5.11 fixed point.
- The fixed-point behavior is defined by the Python reference in tools/fixedpoint.py, and the RTL is intended to be bit-exact to it.
7. Python-side training/export pipeline
- data/names.txt is the training corpus.
- tools/train.py trains the model in Python/PyTorch.
- tools/export.py converts trained weights into hardware-ready artifacts in generated/ and also emits parameter headers such as core/core_params.vh.
- tools/ucode_asm.py assembles the microcode program into generated/ucode.hex and core/coremap.vh.
- tools/check_quant.py, tools/model.py, and tools/dump_*.py support quantization checking and model inspection.
8. Generated artifacts are part of the hardware interface
- The RTL consumes precomputed constants from generated/:
- weight ROM contents like wq_t.hex, wk_t.hex, wv_t.hex, wo_t.hex, fc1_t.hex, fc2_t.hex, lm_t.hex
- embeddings like tok_embed.hex and pos_embed.hex
- microcode in ucode.hex
- test vectors for simulation
- These artifacts are effectively the boundary between the Python training/export flow and the RTL inference flow.
9. Verification architecture
- sim/ contains iSim testbenches for both individual units and the full core:
- tb_mathops.v, tb_exp.v, tb_norm.v, tb_matvec.v, tb_attn.v
- tb_core.v for end-to-end verification
- sim/isim_run.tcl drives the simulator and compares hardware behavior against the golden reference.
- This repo treats bit-exact simulation as the correctness gate before generating a board bitstream.
10. Board-facing user experience
- board/name_generator.v orchestrates the token generation loop on the FPGA board.
- rotary_throttle.v lets the user change either generation rate or temperature.
- tok_meter.v measures and displays throughput.
- lcd_hd44780.v drives the character LCD that shows the current generated name and settings.
11. Build and deployment flow
- build_board_ise_project.tcl and run_board_bitgen.tcl support the Xilinx ISE 14.7 toolchain.
- The target flow is classic FPGA synthesis/place-and-route/bitgen for the Virtex-5 board.
- The top-level hardware artifact is the board bitstream, while the intermediate Python-generated hex/VH files are required inputs to synthesis.
12. Important architectural constraint
- The design is resource-driven: DSPs are the main compute bottleneck, while BRAM is used sparingly for the scratchpad/KV cache.
- The architecture is therefore centered on reusing a small set of modular actuators under microcode control, rather than instantiating a large fully parallel transformer datapath.
13. How to draw the system diagram
- Put tools/ and data/ on the left as the offline training/export pipeline.
- Put generated/ as the artifact boundary feeding core/.
- Place core/microgpt_core.v in the center with the sequencer, shared scratchpad, weight ROMs, and actuators.
- Put board/ on the right as the runtime interface to LCD, rotary input, and throughput display.
- Add sim/ as the verification path that validates the same generated artifacts and RTL before hardware deployment.
Main entry points: ucode_rom.vh Ucode ROM · microgpt_core.v Core · generated Generated ROMs · name_generator.v Name gen
Components 24
Offline pipeline
Python training, export, and microcode assembly
- Train · Trains the tiny character model from the names corpus
tools/train.py - Export · Quantizes weights and emits hardware-ready hex and headers
tools/export.py - Ucode asm · Assembles the sequencer program and coremap
tools/ucode_asm.py - Fixed point · Defines the Q5.11 reference behavior used by RTL and export
tools/fixedpoint.py
Artifacts
Generated hardware inputs and golden vectors
- Generated ROMs · Weights, embeddings, microcode, and test vectors consumed by RTL
generated
FPGA core
Microcode-controlled transformer datapath
- Core · Top-level inference engine for one token step at a time
core/microgpt_core.v - Ucode ROM · Stores the execution schedule for each transformer step
core/ucode_rom.vh - Coremap · Maps micro-ops to datapath actuator modules
core/coremap.vh - Scratchpad · Working memory and incremental KV cache
core/vmem.v - Weight ROMs · Read-only model weights and embedding tables
core/wrom.v - Embed · Looks up token and position embeddings
core/embed.v - Norm · Normalizes activations before projections
core/norm.v - Matvec · Dense projection and MLP matrix-vector engine
core/matvec.v - Attention · Computes current-token attention over cached context
core/attn.v - Vecops · Residual add and activation-style vector operations
core/vecop.v - Sampler · Temperature or greedy next-token selection
core/sampler.v
Board runtime
FPGA top-level and user interface peripherals
- Board top · Connects core, UI peripherals, and board pins
board/xupv5_microgpt_top.v - Name gen · Runs the generation loop and user-facing output
board/name_generator.v - LCD · Drives the character display for the generated name
board/lcd_hd44780.v - Throttle · Adjusts generation rate and sampling temperature
board/rotary_throttle.v - Meter · Measures and shows token generation rate
board/tok_meter.v
Connections 31
- Train → Export: trained weights · Training produces model parameters for export
- Fixed point → Export: numeric rules · Export uses the fixed-point reference for quantization
- Export → Generated ROMs: hex/vh · Writes ROM images and parameter headers
- Ucode asm → Generated ROMs: ucode · Assembles micro-op program and related maps
- Generated ROMs → Core: inputs · Hardware consumes precomputed constants and test vectors
- Generated ROMs → Sim flow: golden data · Simulation compares RTL against exported vectors
- Core → Ucode ROM: fetch · Sequencer reads the next micro-op
- Core → Coremap: dispatch · Micro-ops select actuator modules
- Core → Scratchpad: scratchpad · Core reads and writes working state and KV cache
- Core → Weight ROMs: weights · Core streams read-only model constants
- Ucode ROM → Embed: step · Embedding is one actuator in the microcode schedule
- Ucode ROM → Norm: step · Normalization is one actuator in the microcode schedule
- Ucode ROM → Matvec: step · Projection and MLP are scheduled as actuators
- Ucode ROM → Attention: step · Attention is scheduled as an actuator
- Ucode ROM → Vecops: step · Vector ops are scheduled as actuators
- Ucode ROM → Sampler: step · Sampling is the final actuator before output
- Embed → Scratchpad: tokens · Writes embedded state into the shared scratchpad
- Norm → Scratchpad: activations · Consumes and updates intermediate vectors
- Matvec → Weight ROMs: projections · Reads linear layer weights and embedding tables
- Attention → Scratchpad: kv cache · Reads cached context and appends new K/V
- Vecops → Scratchpad: residuals · Updates the shared working vectors
- Sampler → Name gen: token · Produces the next character for the board runtime
- Board top → Core: inference · Top-level board wrapper instantiates the core
- Board top → Name gen: control · Board logic orchestrates generation timing and UI state
- Name gen → LCD: display · Sends the current generated name and settings to LCD
- Throttle → Name gen: user input · User changes speed or temperature
- Name gen → Meter: rate · Feeds throughput measurements for display
- Core TB → Generated ROMs: vectors · Uses exported constants and test files for end-to-end checks
- Unit TBs → Generated ROMs: vectors · Unit testbenches compare module outputs against golden data
- Sim flow → Core TB: run · Simulation flow drives the core testbench
- Sim flow → Unit TBs: run · Simulation flow drives the unit testbenches