# sayatabdikul/llm_accelerator architecture

> Architecture diagram of the GitHub repository sayatabdikul/llm_accelerator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sayatabdikul/llm_accelerator
- Repository: https://github.com/sayatabdikul/llm_accelerator

## Overview

TACCEL is a research LLM inference accelerator with a Python model-to-instruction toolchain and a SystemVerilog execution target. The principal workflow takes a nanoGPT model through frontend graph construction, quantization and W8A16 compilation, then assembles a program bundle for host simulation or RTL co-simulation. Separate runtime paths support reference evaluation and speculative decoding. The README documents additional hardware capabilities, but the supplied tree omits the shared RTL source, so the hardware is represented only by the visible program runner; detailed RTL wiring is not inferred.

Main entry points: [`software/taccel/compiler/codegen.py`](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/codegen.py) (Instruction Codegen), [`software/taccel/runtime/host_runner.py`](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/host_runner.py) (Host Runner), [`software/taccel/quantizer/quantize.py`](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/quantizer/quantize.py) (Weight Quantization), [`software/taccel/assembler/assembler.py`](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/assembler/assembler.py) (Program Bundle)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Model Frontends"]
  node_nano["nanoGPT Frontend<br/>[nanogpt_adapter.py]"]
  node_deit["DeiT Frontend"]
  node_graph["Graph Extraction<br/>[graph_extract.py]"]
  node_ir["Compiler IR<br/>[ir.py]"]
end

subgraph group_compile["Compiler Toolchain"]
  node_config["Model Configuration<br/>[model_config.py]"]
  node_tiler["Tiling<br/>[tiler.py]"]
  node_allocator["Memory Allocation<br/>[memory_alloc.py]"]
  node_codegen["Instruction Codegen<br/>[codegen.py]"]
  node_w8emit["W8A16 Lowering<br/>[matmul.py]"]
  node_isa["Instruction Set<br/>[instructions.py]"]
  node_assembler["Assembler<br/>[assembler.py]"]
  node_bundle["Program Bundle<br/>[assembler.py]"]
end

subgraph group_quant["Quantization"]
  node_quantizer["Weight Quantization<br/>[quantize.py]"]
  node_calibration["Calibration<br/>[calibrate.py]"]
end

subgraph group_execute["Execution"]
  node_golden["Golden Simulator<br/>[simulator.py]"]
  node_memory[("Simulated Memory<br/>[memory.py]")]
  node_host["Host Runner<br/>[host_runner.py]"]
  node_rtlrunner["RTL Program Runner<br/>[run_program.cpp]"]
end

subgraph group_evaluate["Runtime Evaluation"]
  node_fp32ref["FP32 Reference<br/>[fp32_reference.py]"]
  node_fakequantref["Quantized Reference"]
  node_ppl["Perplexity Evaluation<br/>[gpt2_perplexity.py]"]
  node_spec["Speculative Decode<br/>[speculative.py]"]
end

node_user(("Model User"))
node_checkpoint["Model Checkpoint"]

node_user -->|"selects model"| node_nano
node_checkpoint -->|"provides weights"| node_nano
node_checkpoint -->|"provides weights"| node_quantizer
node_nano -->|"builds graph"| node_graph
node_deit -.->|"builds graph"| node_graph
node_graph -->|"produces IR"| node_ir
node_calibration -->|"supplies calibration"| node_quantizer
node_quantizer -->|"provides quantized weights"| node_codegen
node_ir -->|"lowers graph"| node_codegen
node_config -->|"configures compilation"| node_codegen
node_codegen -->|"uses tiling"| node_tiler
node_codegen -->|"allocates buffers"| node_allocator
node_codegen -->|"dispatches lowering"| node_w8emit
node_w8emit -->|"emits instructions"| node_codegen
node_codegen -->|"creates instructions"| node_isa
node_isa -->|"encodes instructions"| node_assembler
node_assembler -->|"materializes bundle"| node_bundle
node_bundle -->|"runs program"| node_host
node_host -->|"executes instructions"| node_golden
node_golden -->|"reads and writes"| node_memory
node_bundle -->|"loads program"| node_rtlrunner
node_user -->|"requests reference"| node_fp32ref
node_checkpoint -->|"provides weights"| node_fp32ref
node_checkpoint -->|"provides weights"| node_fakequantref
node_quantizer -->|"shares quantization"| node_fakequantref
node_host -->|"supplies inference output"| node_ppl
node_fp32ref -->|"supplies reference output"| node_ppl
node_host -->|"supports decode"| node_spec

click node_nano "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/frontend/nanogpt_adapter.py"
click node_deit "https://github.com/sayatabdikul/llm_accelerator/tree/main/software/taccel/compiler/frontend"
click node_graph "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/graph_extract.py"
click node_ir "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/ir.py"
click node_quantizer "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/quantizer/quantize.py"
click node_calibration "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/quantizer/calibrate.py"
click node_config "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/model_config.py"
click node_tiler "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/tiler.py"
click node_allocator "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/memory_alloc.py"
click node_codegen "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/codegen.py"
click node_w8emit "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/w8a16_emit/matmul.py"
click node_isa "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/isa/instructions.py"
click node_assembler "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/assembler/assembler.py"
click node_bundle "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/assembler/assembler.py"
click node_golden "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/golden_model/simulator.py"
click node_memory "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/golden_model/memory.py"
click node_host "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/host_runner.py"
click node_rtlrunner "https://github.com/sayatabdikul/llm_accelerator/blob/main/rtl/verilator/run_program.cpp"
click node_fp32ref "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/fp32_reference.py"
click node_fakequantref "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/fake_quant_reference.py"
click node_ppl "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/gpt2_perplexity.py"
click node_spec "https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/speculative.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_nano,node_deit,node_graph,node_ir,node_user toneBlue
class node_config,node_tiler,node_allocator,node_codegen,node_w8emit,node_isa,node_assembler,node_bundle toneAmber
class node_quantizer,node_calibration toneMint
class node_golden,node_memory,node_host,node_rtlrunner toneRose
class node_fp32ref,node_fakequantref,node_ppl,node_spec,node_checkpoint toneIndigo
```

## Components

### Model Frontends

- [nanoGPT Frontend](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/frontend/nanogpt_adapter.py): `software/taccel/compiler/frontend/nanogpt_adapter.py`
- [DeiT Frontend](https://github.com/sayatabdikul/llm_accelerator/tree/main/software/taccel/compiler/frontend): `software/taccel/compiler/frontend`
- [Graph Extraction](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/graph_extract.py): `software/taccel/compiler/graph_extract.py`
- [Compiler IR](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/ir.py): `software/taccel/compiler/ir.py`

### Compiler Toolchain

- [Model Configuration](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/model_config.py): `software/taccel/compiler/model_config.py`
- [Tiling](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/tiler.py): `software/taccel/compiler/tiler.py`
- [Memory Allocation](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/memory_alloc.py): `software/taccel/compiler/memory_alloc.py`
- [Instruction Codegen](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/codegen.py): `software/taccel/compiler/codegen.py`
- [W8A16 Lowering](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/compiler/w8a16_emit/matmul.py): `software/taccel/compiler/w8a16_emit/matmul.py`
- [Instruction Set](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/isa/instructions.py): `software/taccel/isa/instructions.py`
- [Assembler](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/assembler/assembler.py): `software/taccel/assembler/assembler.py`
- [Program Bundle](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/assembler/assembler.py): `software/taccel/assembler/assembler.py`

### Quantization

- [Weight Quantization](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/quantizer/quantize.py): `software/taccel/quantizer/quantize.py`
- [Calibration](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/quantizer/calibrate.py): `software/taccel/quantizer/calibrate.py`

### Execution

- [Golden Simulator](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/golden_model/simulator.py): `software/taccel/golden_model/simulator.py`
- [Simulated Memory](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/golden_model/memory.py): `software/taccel/golden_model/memory.py`
- [Host Runner](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/host_runner.py): `software/taccel/runtime/host_runner.py`
- [RTL Program Runner](https://github.com/sayatabdikul/llm_accelerator/blob/main/rtl/verilator/run_program.cpp): `rtl/verilator/run_program.cpp`

### Runtime Evaluation

- [FP32 Reference](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/fp32_reference.py): `software/taccel/runtime/fp32_reference.py`
- [Quantized Reference](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/fake_quant_reference.py): `software/taccel/runtime/fake_quant_reference.py`
- [Perplexity Evaluation](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/gpt2_perplexity.py): `software/taccel/runtime/gpt2_perplexity.py`
- [Speculative Decode](https://github.com/sayatabdikul/llm_accelerator/blob/main/software/taccel/runtime/speculative.py): `software/taccel/runtime/speculative.py`

### Other components

- **Model User**
- **Model Checkpoint**

## Connections

- Model User → nanoGPT Frontend: selects model
- Model Checkpoint → nanoGPT Frontend: provides weights
- Model Checkpoint → Weight Quantization: provides weights
- nanoGPT Frontend → Graph Extraction: builds graph
- DeiT Frontend → Graph Extraction: builds graph
- Graph Extraction → Compiler IR: produces IR
- Calibration → Weight Quantization: supplies calibration
- Weight Quantization → Instruction Codegen: provides quantized weights
- Compiler IR → Instruction Codegen: lowers graph
- Model Configuration → Instruction Codegen: configures compilation
- Instruction Codegen → Tiling: uses tiling
- Instruction Codegen → Memory Allocation: allocates buffers
- Instruction Codegen → W8A16 Lowering: dispatches lowering
- W8A16 Lowering → Instruction Codegen: emits instructions
- Instruction Codegen → Instruction Set: creates instructions
- Instruction Set → Assembler: encodes instructions
- Assembler → Program Bundle: materializes bundle
- Program Bundle → Host Runner: runs program
- Host Runner → Golden Simulator: executes instructions
- Golden Simulator → Simulated Memory: reads and writes
- Program Bundle → RTL Program Runner: loads program
- Model User → FP32 Reference: requests reference
- Model Checkpoint → FP32 Reference: provides weights
- Model Checkpoint → Quantized Reference: provides weights
- Weight Quantization → Quantized Reference: shares quantization
- Host Runner → Perplexity Evaluation: supplies inference output
- FP32 Reference → Perplexity Evaluation: supplies reference output
- Host Runner → Speculative Decode: supports decode