How hyperionaidev/smolmaxxing works
smolmaxxing is a low-memory LLM experimentation project: it prepares model files and demonstrates chat/inference on constrained devices, with device-specific kernels and performance probes. The README documents expert paging, dense-model flash streaming, and vocabulary-head pruning as major techniques, though their llama.cpp patch implementations are not in the supplied tree. The graph also captures the iPad chat and tiny story-model server, plus model-file tooling and hardware measurement. Relationships are limited to excerpted calls/import references or README statements; unsampled implementations are not treated as inspected.
Main entry points: gguf_split_adb.py ADB GGUF splitter · smolchat.cpp iPad chat · server.c Tiny LLM server · gguf_remote.py Remote GGUF ranges
Components 21
Chat runtimes
- iPad chat
ipad-air/smolchat.cpp - Safari chat server
ipad-air/web.h - Tiny LLM server
tiny-llm-web/server.c - llama2.c inference
tiny-llm-web/server.c
Model preparation
- MoE assembler
tools/gguf_moe_assemble.py - Remote GGUF ranges
tools/gguf_remote.py - Head recutter
tools/gguf-recut/gguf_recut.c - ADB GGUF splitter
tools/gguf_split_adb.py - Range reader
tools/rangecat/rangecat.c - GGUF builder
tools/gguf_build.py - Vocabulary selector
tools/vocab_select.py
Device techniques
- Expert paging
patches - Flash streaming
patches - Vulkan GEMV probe
tools/vkgemv/vkgemv.c - GEMV shader data
tools/vkgemv/gemv.spv.h - Vulkan bandwidth probe
tools/vkprobe/vkprobe.c - Bandwidth shader data
tools/vkprobe/bw.spv.h
Hardware measurement
- Kernel benchmarks
tools/kbench/kbench_tq.c - iPad device probes
ipad-air/kbench.c - Memory bandwidth probe
tools/membw/membw.c
Other components
- Device user
Connections 11
- Device user → iPad chat: chats with
ipad-air/smolchat.cpp - iPad chat → Safari chat server: serves through
ipad-air/smolchat.cpp - Device user → Tiny LLM server: requests generation
tiny-llm-web/server.c - Tiny LLM server → llama2.c inference: invokes inference
tiny-llm-web/server.c - MoE assembler → Remote GGUF ranges: fetches ranges
tools/gguf_moe_assemble.py - ADB GGUF splitter → Remote GGUF ranges: uses range helpers
tools/gguf_split_adb.py - ADB GGUF splitter → Range reader: reads device ranges
tools/gguf_split_adb.py - Vulkan GEMV probe → GEMV shader data: loads shader data
tools/vkgemv/vkgemv.c - Vulkan bandwidth probe → Bandwidth shader data: loads shader data
tools/vkprobe/vkprobe.c - Device user → Expert paging: runs models with
README.md - Device user → Flash streaming: runs models with
README.md