# gazbg/poormansllama architecture

> Architecture diagram of the GitHub repository gazbg/poormansllama, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gazbg/poormansllama
- Repository: https://github.com/gazbg/poormansllama

## Overview

This repository is a research record and tooling collection for optimizing llama.cpp on memory-constrained GPUs, not a standalone server implementation in the supplied tree. Its practical workflow is to run controlled server or kernel experiments, capture results and traces, then analyze or cross-check them. The graph preserves the major sampled research branches and the SOURCE INDEX core modules; the vendored runtime is described by the README but its implementation is not included in the supplied tree, so runtime internals are not mapped.

Main entry points: [`third_party/llama.cpp`](https://github.com/gazbg/poormansllama/tree/main/third_party/llama.cpp) (Vendored llama.cpp), [`tools/dense/win-arm.py`](https://github.com/gazbg/poormansllama/blob/main/tools/dense/win-arm.py) (Dense server arm), [`tools/check-tg-predict.py`](https://github.com/gazbg/poormansllama/blob/main/tools/check-tg-predict.py) (Prediction verdict), [`tools/server_vision.py`](https://github.com/gazbg/poormansllama/blob/main/tools/server_vision.py) (Vision harness)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiments["Experiment runners"]
  node_dense_arm["Dense server arm<br/>[win-arm.py]"]
  node_dense_plan["Dense planning<br/>[dense_plan.py]"]
  node_dense_probe["Dense HTTP probe<br/>[dense_probe.py]"]
  node_tg_arms["TG arm runner<br/>[run-tg-arms.py]"]
end

subgraph group_analysis["Trace analysis"]
  node_pregate["Expert pre-gate<br/>[analyze-pregate.py]"]
  node_predict_verdict["Prediction verdict"]
  node_d1_replay["MTP replay<br/>[tg59_d1.py]"]
  node_route_analysis["Route overlap<br/>[analyze.py]"]
  node_retention_check["Retention cross-check"]
  node_predictor_screen["Predictor screening"]
end

subgraph group_server["Server studies"]
  node_review_tables["LM review tables"]
  node_canonical_tables["Canonical tables"]
  node_server_mt["Server harness<br/>[server_mt.py]"]
  node_vision["Vision harness<br/>[server_vision.py]"]
end

subgraph group_kernels["Kernel probes"]
  node_expert_bench["CPU expert bench"]
  node_seqsplit["Sequence split probe<br/>[seqsplit_probe.cpp]"]
  node_stream_bench["CPU stream bench<br/>[bench_stream.cpp]"]
  node_mmq_check["CUDA MMQ check"]
end

node_researcher(("Researcher"))
node_llama_runtime["Vendored llama.cpp<br/>[llama.cpp]"]

node_researcher -->|"runs experiments"| node_dense_arm
node_researcher -->|"runs experiments"| node_server_mt
node_researcher -->|"runs experiments"| node_vision
node_researcher -->|"analyzes traces"| node_route_analysis
node_dense_arm -->|"sends probes"| node_dense_probe
node_vision -->|"uses harness"| node_server_mt
node_review_tables -->|"reads tables"| node_canonical_tables
node_retention_check -->|"reuses analyzer"| node_route_analysis
node_predict_verdict -->|"uses replay"| node_d1_replay
node_expert_bench -->|"uses GGML APIs"| node_llama_runtime
node_seqsplit -->|"uses llama APIs"| node_llama_runtime
node_stream_bench -->|"uses GGML APIs"| node_llama_runtime
node_mmq_check -->|"uses CUDA APIs"| node_llama_runtime
node_pregate -->|"reads GGUF code"| node_llama_runtime
node_dense_plan -.->|"plans arms"| node_dense_arm
node_predictor_screen -.->|"screens predictors"| node_predict_verdict
node_tg_arms -.->|"produces traces"| node_predict_verdict

click node_dense_arm "https://github.com/gazbg/poormansllama/blob/main/tools/dense/win-arm.py"
click node_dense_plan "https://github.com/gazbg/poormansllama/blob/main/tools/dense_plan.py"
click node_dense_probe "https://github.com/gazbg/poormansllama/blob/main/tools/dense_probe.py"
click node_pregate "https://github.com/gazbg/poormansllama/blob/main/tools/analyze-pregate.py"
click node_predict_verdict "https://github.com/gazbg/poormansllama/blob/main/tools/check-tg-predict.py"
click node_d1_replay "https://github.com/gazbg/poormansllama/blob/main/tools/tg59_d1.py"
click node_route_analysis "https://github.com/gazbg/poormansllama/blob/main/tools/route_overlap/analyze.py"
click node_retention_check "https://github.com/gazbg/poormansllama/blob/main/tools/route_overlap/crosscheck-retention.py"
click node_review_tables "https://github.com/gazbg/poormansllama/blob/main/tools/server_lm_review_tables.py"
click node_canonical_tables "https://github.com/gazbg/poormansllama/blob/main/tools/server_canon_tables.py"
click node_server_mt "https://github.com/gazbg/poormansllama/blob/main/tools/server_mt.py"
click node_vision "https://github.com/gazbg/poormansllama/blob/main/tools/server_vision.py"
click node_expert_bench "https://github.com/gazbg/poormansllama/blob/main/tools/expert-cpu-bench.cpp"
click node_seqsplit "https://github.com/gazbg/poormansllama/blob/main/tools/pascal/seqsplit_probe.cpp"
click node_stream_bench "https://github.com/gazbg/poormansllama/blob/main/tools/pascal/bench_stream.cpp"
click node_mmq_check "https://github.com/gazbg/poormansllama/blob/main/tools/pascal/check_mmq_remainder.cpp"
click node_predictor_screen "https://github.com/gazbg/poormansllama/blob/main/tools/predictor-screen-gplr.py"
click node_tg_arms "https://github.com/gazbg/poormansllama/blob/main/tools/run-tg-arms.py"
click node_llama_runtime "https://github.com/gazbg/poormansllama/tree/main/third_party/llama.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dense_arm,node_dense_plan,node_dense_probe,node_tg_arms toneBlue
class node_pregate,node_predict_verdict,node_d1_replay,node_route_analysis,node_retention_check,node_predictor_screen toneAmber
class node_review_tables,node_canonical_tables,node_server_mt,node_vision toneMint
class node_expert_bench,node_seqsplit,node_stream_bench,node_mmq_check toneRose
class node_researcher toneIndigo
class node_llama_runtime toneTeal
```

## Components

### Experiment runners

- [Dense server arm](https://github.com/gazbg/poormansllama/blob/main/tools/dense/win-arm.py): `tools/dense/win-arm.py`
- [Dense planning](https://github.com/gazbg/poormansllama/blob/main/tools/dense_plan.py): `tools/dense_plan.py`
- [Dense HTTP probe](https://github.com/gazbg/poormansllama/blob/main/tools/dense_probe.py): `tools/dense_probe.py`
- [TG arm runner](https://github.com/gazbg/poormansllama/blob/main/tools/run-tg-arms.py): `tools/run-tg-arms.py`

### Trace analysis

- [Expert pre-gate](https://github.com/gazbg/poormansllama/blob/main/tools/analyze-pregate.py): `tools/analyze-pregate.py`
- [Prediction verdict](https://github.com/gazbg/poormansllama/blob/main/tools/check-tg-predict.py): `tools/check-tg-predict.py`
- [MTP replay](https://github.com/gazbg/poormansllama/blob/main/tools/tg59_d1.py): `tools/tg59_d1.py`
- [Route overlap](https://github.com/gazbg/poormansllama/blob/main/tools/route_overlap/analyze.py): `tools/route_overlap/analyze.py`
- [Retention cross-check](https://github.com/gazbg/poormansllama/blob/main/tools/route_overlap/crosscheck-retention.py): `tools/route_overlap/crosscheck-retention.py`
- [Predictor screening](https://github.com/gazbg/poormansllama/blob/main/tools/predictor-screen-gplr.py): `tools/predictor-screen-gplr.py`

### Server studies

- [LM review tables](https://github.com/gazbg/poormansllama/blob/main/tools/server_lm_review_tables.py): `tools/server_lm_review_tables.py`
- [Canonical tables](https://github.com/gazbg/poormansllama/blob/main/tools/server_canon_tables.py): `tools/server_canon_tables.py`
- [Server harness](https://github.com/gazbg/poormansllama/blob/main/tools/server_mt.py): `tools/server_mt.py`
- [Vision harness](https://github.com/gazbg/poormansllama/blob/main/tools/server_vision.py): `tools/server_vision.py`

### Kernel probes

- [CPU expert bench](https://github.com/gazbg/poormansllama/blob/main/tools/expert-cpu-bench.cpp): `tools/expert-cpu-bench.cpp`
- [Sequence split probe](https://github.com/gazbg/poormansllama/blob/main/tools/pascal/seqsplit_probe.cpp): `tools/pascal/seqsplit_probe.cpp`
- [CPU stream bench](https://github.com/gazbg/poormansllama/blob/main/tools/pascal/bench_stream.cpp): `tools/pascal/bench_stream.cpp`
- [CUDA MMQ check](https://github.com/gazbg/poormansllama/blob/main/tools/pascal/check_mmq_remainder.cpp): `tools/pascal/check_mmq_remainder.cpp`

### Other components

- **Researcher**
- [Vendored llama.cpp](https://github.com/gazbg/poormansllama/tree/main/third_party/llama.cpp): `third_party/llama.cpp`

## Connections

- Researcher → Dense server arm: runs experiments
- Researcher → Server harness: runs experiments
- Researcher → Vision harness: runs experiments
- Researcher → Route overlap: analyzes traces
- Dense server arm → Dense HTTP probe: sends probes (evidence: `tools/dense/win-arm.py`)
- Vision harness → Server harness: uses harness (evidence: `tools/server_vision.py`)
- LM review tables → Canonical tables: reads tables (evidence: `tools/server_lm_review_tables.py`)
- Retention cross-check → Route overlap: reuses analyzer (evidence: `tools/route_overlap/crosscheck-retention.py`)
- Prediction verdict → MTP replay: uses replay (evidence: `tools/check-tg-predict.py`)
- CPU expert bench → Vendored llama.cpp: uses GGML APIs (evidence: `tools/expert-cpu-bench.cpp`)
- Sequence split probe → Vendored llama.cpp: uses llama APIs (evidence: `tools/pascal/seqsplit_probe.cpp`)
- CPU stream bench → Vendored llama.cpp: uses GGML APIs (evidence: `tools/pascal/bench_stream.cpp`)
- CUDA MMQ check → Vendored llama.cpp: uses CUDA APIs (evidence: `tools/pascal/check_mmq_remainder.cpp`)
- Expert pre-gate → Vendored llama.cpp: reads GGUF code (evidence: `tools/analyze-pregate.py`)
- Dense planning → Dense server arm: plans arms
- Predictor screening → Prediction verdict: screens predictors
- TG arm runner → Prediction verdict: produces traces