How ngxson/wllama works
wllama is a browser-side WebAssembly binding for llama.cpp. The principal workflow is to obtain a GGUF model from a URL or Hugging Face, download and cache its files, load them into a worker, and expose inference results through a typed OpenAI-compatible API. The worker bridges JavaScript and the native llama.cpp runtime; documented capabilities include chat/completions, embeddings, multimodal input, tool calling, WebGPU, and split-model downloads. Some feature-to-native-runtime details are documented but not visible in the excerpts, so those links are marked unverified.
Main entry points: wllama.ts Wllama API · worker.ts Worker proxy · cache-manager.ts Download cache · wllama-context.h Inference context
Components 21
Public API
- Wllama API
src/wllama.ts - OpenAI types
src/types/oai-compat.ts
Model acquisition
- Hugging Face source
src/huggingface.ts - Model manager
src/model-manager.ts - Download cache
src/cache-manager.ts - Storage backends
src/storage - COS backend
src/storage/cos.ts - OPFS backend
src/storage/opfs.ts
Inference runtime
- Worker proxy
src/worker.ts - Worker runtime code
src/workers-code/generated.ts - Glue protocol
src/glue/glue.ts - Message schemas
src/glue/messages.ts - Wasm module
src/wasm/wllama.js
Native engine
- C++ bindings
cpp/wllama.cpp - Inference context
cpp/wllama-context.h - Native filesystem
cpp/wllama-fs.h - Native glue
cpp/glue.hpp - llama.cpp engine
llama.cpp
Other components
- Application developer
- Hugging Face
- Browser
Connections 22
- Application developer → Wllama API: calls
README.md - Wllama API → OpenAI types: uses types
src/wllama.ts - Wllama API → Hugging Face source: resolves model
src/wllama.ts - Wllama API → Model manager: loads model
src/wllama.ts - Hugging Face source → Hugging Face: fetches metadata
README.md - Model manager → Download cache: downloads files
src/model-manager.ts - Download cache → Hugging Face source: checks file hash
src/cache-manager.ts - Download cache → COS backend: selects backend
src/cache-manager.ts - Download cache → Storage backends: uses interface
src/cache-manager.ts - COS backend → OPFS backend: references backend
src/storage/cos.ts - Wllama API → Worker proxy: dispatches inference
src/wllama.ts - Worker proxy → Glue protocol: serializes messages
src/worker.ts - Glue protocol → Message schemas: uses schemas
src/glue/glue.ts - Worker proxy → Worker runtime code: loads runtime code
src/worker.ts - Browser → Worker proxy: runs worker
README.md - Worker proxy → Wasm module: loads module
- Wasm module → C++ bindings: invokes bindings
- C++ bindings → Inference context: uses context
cpp/wllama.cpp - C++ bindings → Native filesystem: uses filesystem
cpp/wllama.cpp - Inference context → Native glue: uses protocol
cpp/wllama-context.h - Inference context → llama.cpp engine: runs inference
- Wllama API → Inference context: supports inference features