# sigmanticai/apex-inference-chip architecture

> Architecture diagram of the GitHub repository sigmanticai/apex-inference-chip, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sigmanticai/apex-inference-chip
- Repository: https://github.com/sigmanticai/apex-inference-chip

## Overview

APEX is an open, verification-first LLM-inference tile, with a software golden pipeline for decoder-layer inference and hardware-oriented job descriptions. The principal software workflow prepares model weights, streams token steps through the fixed-point golden model, and can emit traces for replay; the README describes the corresponding on-tile attention, KV-cache compression, and feed-forward stages. Verification generators and walker tooling form a separate branch around that golden reference. The supplied tree omits most RTL and some runtime code, so those implementation links are not expanded here.

Main entry points: [`golden/apex_golden/transformer.py`](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/transformer.py) (Decoder Layer), [`golden/apex_golden/attention.py`](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/attention.py) (Attention Chain), [`run_tinynpu.py`](https://github.com/sigmanticai/apex-inference-chip/blob/main/run_tinynpu.py) (Token Runner), [`verif/seq_walker/gen_layer_trace.py`](https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/seq_walker/gen_layer_trace.py) (Layer Trace Spec)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Golden Runtime"]
  node_runner["Token Runner<br/>[run_tinynpu.py]"]
  node_transformer["Decoder Layer<br/>[transformer.py]"]
  node_weightprep["W4 Weight Installer<br/>[w4_weights_mlx.py]"]
end

subgraph group_numeric["Numeric Primitives"]
  node_attention["Attention Chain<br/>[attention.py]"]
  node_compute["Compute Contracts<br/>[compute.py]"]
  node_codec["KV Codec<br/>[cq_codec.py]"]
  node_fp["FP Helpers<br/>[fp.py]"]
  node_weightcodec["Weight Codec<br/>[weight_codec.py]"]
end

subgraph group_walk["Walker Tooling"]
  node_fmt["Walker Format<br/>[seq_walker_fmt.py]"]
  node_layertrace["Layer Trace Spec<br/>[gen_layer_trace.py]"]
  node_walkgold["Walker Golden Compiler<br/>[gen_walk_golden.py]"]
end

subgraph group_verify["Verification"]
  node_layerverify["Layer Vector Generator"]
  node_tracegold["Walker Trace Golden"]
end

node_checkpoint["Qwen Weights"]
node_tokenresult["Token Output"]
node_vectors["Golden Vectors"]

node_checkpoint -->|"loads weights"| node_runner
node_runner -->|"runs decode"| node_transformer
node_transformer -->|"calls attention"| node_attention
node_transformer -->|"uses primitives"| node_compute
node_transformer -->|"uses rounding"| node_fp
node_transformer -->|"grades weights"| node_weightcodec
node_attention -->|"compresses KV"| node_codec
node_attention -->|"uses primitives"| node_compute
node_attention -->|"uses rounding"| node_fp
node_runner -->|"emits tokens"| node_tokenresult
node_weightprep -->|"uses codec"| node_weightcodec
node_layertrace -->|"builds descriptors"| node_fmt
node_layertrace -->|"checks stages"| node_transformer
node_walkgold -->|"uses emitter"| node_layertrace
node_walkgold -->|"packs descriptors"| node_fmt
node_layerverify -->|"checks layer"| node_transformer
node_layerverify -->|"emits vectors"| node_vectors
node_tracegold -->|"uses reference"| node_attention

click node_runner "https://github.com/sigmanticai/apex-inference-chip/blob/main/run_tinynpu.py"
click node_transformer "https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/transformer.py"
click node_attention "https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/attention.py"
click node_compute "https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/compute.py"
click node_codec "https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/cq_codec.py"
click node_fp "https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/fp.py"
click node_weightcodec "https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/weight_codec.py"
click node_weightprep "https://github.com/sigmanticai/apex-inference-chip/blob/main/eval/kv_eval/w4_weights_mlx.py"
click node_fmt "https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/seq_walker/seq_walker_fmt.py"
click node_layertrace "https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/seq_walker/gen_layer_trace.py"
click node_walkgold "https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/walkgold/gen_walk_golden.py"
click node_layerverify "https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/layer/gen_layer_vectors.py"
click node_tracegold "https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/top/l3/walker_composite_golden.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_transformer,node_weightprep toneBlue
class node_attention,node_compute,node_codec,node_fp,node_weightcodec toneAmber
class node_fmt,node_layertrace,node_walkgold toneMint
class node_layerverify,node_tracegold toneRose
class node_checkpoint,node_tokenresult,node_vectors toneIndigo
```

## Components

### Golden Runtime

- [Token Runner](https://github.com/sigmanticai/apex-inference-chip/blob/main/run_tinynpu.py): `run_tinynpu.py`
- [Decoder Layer](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/transformer.py): `golden/apex_golden/transformer.py`
- [W4 Weight Installer](https://github.com/sigmanticai/apex-inference-chip/blob/main/eval/kv_eval/w4_weights_mlx.py): `eval/kv_eval/w4_weights_mlx.py`

### Numeric Primitives

- [Attention Chain](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/attention.py): `golden/apex_golden/attention.py`
- [Compute Contracts](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/compute.py): `golden/apex_golden/compute.py`
- [KV Codec](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/cq_codec.py): `golden/apex_golden/cq_codec.py`
- [FP Helpers](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/fp.py): `golden/apex_golden/fp.py`
- [Weight Codec](https://github.com/sigmanticai/apex-inference-chip/blob/main/golden/apex_golden/weight_codec.py): `golden/apex_golden/weight_codec.py`

### Walker Tooling

- [Walker Format](https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/seq_walker/seq_walker_fmt.py): `verif/seq_walker/seq_walker_fmt.py`
- [Layer Trace Spec](https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/seq_walker/gen_layer_trace.py): `verif/seq_walker/gen_layer_trace.py`
- [Walker Golden Compiler](https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/walkgold/gen_walk_golden.py): `verif/walkgold/gen_walk_golden.py`

### Verification

- [Layer Vector Generator](https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/layer/gen_layer_vectors.py): `verif/layer/gen_layer_vectors.py`
- [Walker Trace Golden](https://github.com/sigmanticai/apex-inference-chip/blob/main/verif/top/l3/walker_composite_golden.py): `verif/top/l3/walker_composite_golden.py`

### Other components

- **Qwen Weights**
- **Token Output**
- **Golden Vectors**

## Connections

- Qwen Weights → Token Runner: loads weights (evidence: `run_tinynpu.py`)
- Token Runner → Decoder Layer: runs decode (evidence: `run_tinynpu.py`)
- Decoder Layer → Attention Chain: calls attention (evidence: `golden/apex_golden/transformer.py`)
- Decoder Layer → Compute Contracts: uses primitives (evidence: `golden/apex_golden/transformer.py`)
- Decoder Layer → FP Helpers: uses rounding (evidence: `golden/apex_golden/transformer.py`)
- Decoder Layer → Weight Codec: grades weights (evidence: `golden/apex_golden/transformer.py`)
- Attention Chain → KV Codec: compresses KV (evidence: `golden/apex_golden/attention.py`)
- Attention Chain → Compute Contracts: uses primitives (evidence: `golden/apex_golden/attention.py`)
- Attention Chain → FP Helpers: uses rounding (evidence: `golden/apex_golden/attention.py`)
- Token Runner → Token Output: emits tokens (evidence: `run_tinynpu.py`)
- W4 Weight Installer → Weight Codec: uses codec (evidence: `eval/kv_eval/w4_weights_mlx.py`)
- Layer Trace Spec → Walker Format: builds descriptors (evidence: `verif/seq_walker/gen_layer_trace.py`)
- Layer Trace Spec → Decoder Layer: checks stages (evidence: `verif/seq_walker/gen_layer_trace.py`)
- Walker Golden Compiler → Layer Trace Spec: uses emitter (evidence: `verif/walkgold/gen_walk_golden.py`)
- Walker Golden Compiler → Walker Format: packs descriptors (evidence: `verif/walkgold/gen_walk_golden.py`)
- Layer Vector Generator → Decoder Layer: checks layer (evidence: `verif/layer/gen_layer_vectors.py`)
- Layer Vector Generator → Golden Vectors: emits vectors (evidence: `verif/layer/gen_layer_vectors.py`)
- Walker Trace Golden → Attention Chain: uses reference (evidence: `verif/top/l3/walker_composite_golden.py`)
