How sayatabdikul/llm_accelerator works
TACCEL is a research LLM inference accelerator with a Python model-to-instruction toolchain and a SystemVerilog execution target. The principal workflow takes a nanoGPT model through frontend graph construction, quantization and W8A16 compilation, then assembles a program bundle for host simulation or RTL co-simulation. Separate runtime paths support reference evaluation and speculative decoding. The README documents additional hardware capabilities, but the supplied tree omits the shared RTL source, so the hardware is represented only by the visible program runner; detailed RTL wiring is not inferred.
Main entry points: codegen.py Instruction Codegen · host_runner.py Host Runner · quantize.py Weight Quantization · assembler.py Program Bundle
Components 24
Model Frontends
- nanoGPT Frontend
software/taccel/compiler/frontend/nanogpt_adapter.py - DeiT Frontend
software/taccel/compiler/frontend - Graph Extraction
software/taccel/compiler/graph_extract.py - Compiler IR
software/taccel/compiler/ir.py
Compiler Toolchain
- Model Configuration
software/taccel/compiler/model_config.py - Tiling
software/taccel/compiler/tiler.py - Memory Allocation
software/taccel/compiler/memory_alloc.py - Instruction Codegen
software/taccel/compiler/codegen.py - W8A16 Lowering
software/taccel/compiler/w8a16_emit/matmul.py - Instruction Set
software/taccel/isa/instructions.py - Assembler
software/taccel/assembler/assembler.py - Program Bundle
software/taccel/assembler/assembler.py
Quantization
- Weight Quantization
software/taccel/quantizer/quantize.py - Calibration
software/taccel/quantizer/calibrate.py
Execution
- Golden Simulator
software/taccel/golden_model/simulator.py - Simulated Memory
software/taccel/golden_model/memory.py - Host Runner
software/taccel/runtime/host_runner.py - RTL Program Runner
rtl/verilator/run_program.cpp
Runtime Evaluation
- FP32 Reference
software/taccel/runtime/fp32_reference.py - Quantized Reference
software/taccel/runtime/fake_quant_reference.py - Perplexity Evaluation
software/taccel/runtime/gpt2_perplexity.py - Speculative Decode
software/taccel/runtime/speculative.py
Other components
- Model User
- Model Checkpoint
Connections 28
- Model User → nanoGPT Frontend: selects model
- Model Checkpoint → nanoGPT Frontend: provides weights
- Model Checkpoint → Weight Quantization: provides weights
- nanoGPT Frontend → Graph Extraction: builds graph
- DeiT Frontend → Graph Extraction: builds graph
- Graph Extraction → Compiler IR: produces IR
- Calibration → Weight Quantization: supplies calibration
- Weight Quantization → Instruction Codegen: provides quantized weights
- Compiler IR → Instruction Codegen: lowers graph
- Model Configuration → Instruction Codegen: configures compilation
- Instruction Codegen → Tiling: uses tiling
- Instruction Codegen → Memory Allocation: allocates buffers
- Instruction Codegen → W8A16 Lowering: dispatches lowering
- W8A16 Lowering → Instruction Codegen: emits instructions
- Instruction Codegen → Instruction Set: creates instructions
- Instruction Set → Assembler: encodes instructions
- Assembler → Program Bundle: materializes bundle
- Program Bundle → Host Runner: runs program
- Host Runner → Golden Simulator: executes instructions
- Golden Simulator → Simulated Memory: reads and writes
- Program Bundle → RTL Program Runner: loads program
- Model User → FP32 Reference: requests reference
- Model Checkpoint → FP32 Reference: provides weights
- Model Checkpoint → Quantized Reference: provides weights
- Weight Quantization → Quantized Reference: shares quantization
- Host Runner → Perplexity Evaluation: supplies inference output
- FP32 Reference → Perplexity Evaluation: supplies reference output
- Host Runner → Speculative Decode: supports decode