How sigmanticai/apex-inference-chip works
APEX is an open, verification-first LLM-inference tile, with a software golden pipeline for decoder-layer inference and hardware-oriented job descriptions. The principal software workflow prepares model weights, streams token steps through the fixed-point golden model, and can emit traces for replay; the README describes the corresponding on-tile attention, KV-cache compression, and feed-forward stages. Verification generators and walker tooling form a separate branch around that golden reference. The supplied tree omits most RTL and some runtime code, so those implementation links are not expanded here.
Main entry points: transformer.py Decoder Layer · attention.py Attention Chain · run_tinynpu.py Token Runner · gen_layer_trace.py Layer Trace Spec
Components 16
Golden Runtime
- Token Runner
run_tinynpu.py - Decoder Layer
golden/apex_golden/transformer.py - W4 Weight Installer
eval/kv_eval/w4_weights_mlx.py
Numeric Primitives
- Attention Chain
golden/apex_golden/attention.py - Compute Contracts
golden/apex_golden/compute.py - KV Codec
golden/apex_golden/cq_codec.py - FP Helpers
golden/apex_golden/fp.py - Weight Codec
golden/apex_golden/weight_codec.py
Walker Tooling
- Walker Format
verif/seq_walker/seq_walker_fmt.py - Layer Trace Spec
verif/seq_walker/gen_layer_trace.py - Walker Golden Compiler
verif/walkgold/gen_walk_golden.py
Verification
- Layer Vector Generator
verif/layer/gen_layer_vectors.py - Walker Trace Golden
verif/top/l3/walker_composite_golden.py
Other components
- Qwen Weights
- Token Output
- Golden Vectors
Connections 18
- Qwen Weights → Token Runner: loads weights
run_tinynpu.py - Token Runner → Decoder Layer: runs decode
run_tinynpu.py - Decoder Layer → Attention Chain: calls attention
golden/apex_golden/transformer.py - Decoder Layer → Compute Contracts: uses primitives
golden/apex_golden/transformer.py - Decoder Layer → FP Helpers: uses rounding
golden/apex_golden/transformer.py - Decoder Layer → Weight Codec: grades weights
golden/apex_golden/transformer.py - Attention Chain → KV Codec: compresses KV
golden/apex_golden/attention.py - Attention Chain → Compute Contracts: uses primitives
golden/apex_golden/attention.py - Attention Chain → FP Helpers: uses rounding
golden/apex_golden/attention.py - Token Runner → Token Output: emits tokens
run_tinynpu.py - W4 Weight Installer → Weight Codec: uses codec
eval/kv_eval/w4_weights_mlx.py - Layer Trace Spec → Walker Format: builds descriptors
verif/seq_walker/gen_layer_trace.py - Layer Trace Spec → Decoder Layer: checks stages
verif/seq_walker/gen_layer_trace.py - Walker Golden Compiler → Layer Trace Spec: uses emitter
verif/walkgold/gen_walk_golden.py - Walker Golden Compiler → Walker Format: packs descriptors
verif/walkgold/gen_walk_golden.py - Layer Vector Generator → Decoder Layer: checks layer
verif/layer/gen_layer_vectors.py - Layer Vector Generator → Golden Vectors: emits vectors
verif/layer/gen_layer_vectors.py - Walker Trace Golden → Attention Chain: uses reference
verif/top/l3/walker_composite_golden.py