# ngxson/wllama architecture

> Architecture diagram of the GitHub repository ngxson/wllama, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ngxson/wllama
- Repository: https://github.com/ngxson/wllama

## Overview

wllama is a browser-side WebAssembly binding for llama.cpp. The principal workflow is to obtain a GGUF model from a URL or Hugging Face, download and cache its files, load them into a worker, and expose inference results through a typed OpenAI-compatible API. The worker bridges JavaScript and the native llama.cpp runtime; documented capabilities include chat/completions, embeddings, multimodal input, tool calling, WebGPU, and split-model downloads. Some feature-to-native-runtime details are documented but not visible in the excerpts, so those links are marked unverified.

Main entry points: [`src/wllama.ts`](https://github.com/ngxson/wllama/blob/master/src/wllama.ts) (Wllama API), [`src/worker.ts`](https://github.com/ngxson/wllama/blob/master/src/worker.ts) (Worker proxy), [`src/cache-manager.ts`](https://github.com/ngxson/wllama/blob/master/src/cache-manager.ts) (Download cache), [`cpp/wllama-context.h`](https://github.com/ngxson/wllama/blob/master/cpp/wllama-context.h) (Inference context)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Public API"]
  node_api["Wllama API<br/>[wllama.ts]"]
  node_oai["OpenAI types<br/>[oai-compat.ts]"]
end

subgraph group_models["Model acquisition"]
  node_hf["Hugging Face source<br/>[huggingface.ts]"]
  node_model["Model manager<br/>[model-manager.ts]"]
  node_cache["Download cache<br/>[cache-manager.ts]"]
  node_storage[("Storage backends")]
  node_cos[("COS backend<br/>[cos.ts]")]
  node_opfs[("OPFS backend<br/>[opfs.ts]")]
end

subgraph group_execution["Inference runtime"]
  node_worker["Worker proxy<br/>[worker.ts]"]
  node_workerjs["Worker runtime code<br/>[generated.ts]"]
  node_protocol["Glue protocol<br/>[glue.ts]"]
  node_messages["Message schemas<br/>[messages.ts]"]
  node_wasm["Wasm module<br/>[wllama.js]"]
end

subgraph group_native["Native engine"]
  node_nativecpp["C++ bindings<br/>[wllama.cpp]"]
  node_context["Inference context<br/>[wllama-context.h]"]
  node_filesystem["Native filesystem<br/>[wllama-fs.h]"]
  node_gluecpp["Native glue<br/>[glue.hpp]"]
  node_llamacpp["llama.cpp engine<br/>[llama.cpp]"]
end

node_user(("Application developer"))
node_huggingface(("Hugging Face"))
node_browser(("Browser"))

node_user -->|"calls"| node_api
node_api -->|"uses types"| node_oai
node_api -->|"resolves model"| node_hf
node_api -->|"loads model"| node_model
node_hf -->|"fetches metadata"| node_huggingface
node_model -->|"downloads files"| node_cache
node_cache -->|"checks file hash"| node_hf
node_cache -->|"selects backend"| node_cos
node_cache -->|"uses interface"| node_storage
node_cos -->|"references backend"| node_opfs
node_api -->|"dispatches inference"| node_worker
node_worker -->|"serializes messages"| node_protocol
node_protocol -->|"uses schemas"| node_messages
node_worker -->|"loads runtime code"| node_workerjs
node_browser -->|"runs worker"| node_worker
node_worker -.->|"loads module"| node_wasm
node_wasm -.->|"invokes bindings"| node_nativecpp
node_nativecpp -->|"uses context"| node_context
node_nativecpp -->|"uses filesystem"| node_filesystem
node_context -->|"uses protocol"| node_gluecpp
node_context -.->|"runs inference"| node_llamacpp
node_api -.->|"supports inference features"| node_context

click node_api "https://github.com/ngxson/wllama/blob/master/src/wllama.ts"
click node_oai "https://github.com/ngxson/wllama/blob/master/src/types/oai-compat.ts"
click node_hf "https://github.com/ngxson/wllama/blob/master/src/huggingface.ts"
click node_model "https://github.com/ngxson/wllama/blob/master/src/model-manager.ts"
click node_cache "https://github.com/ngxson/wllama/blob/master/src/cache-manager.ts"
click node_storage "https://github.com/ngxson/wllama/tree/master/src/storage"
click node_cos "https://github.com/ngxson/wllama/blob/master/src/storage/cos.ts"
click node_opfs "https://github.com/ngxson/wllama/blob/master/src/storage/opfs.ts"
click node_worker "https://github.com/ngxson/wllama/blob/master/src/worker.ts"
click node_workerjs "https://github.com/ngxson/wllama/blob/master/src/workers-code/generated.ts"
click node_protocol "https://github.com/ngxson/wllama/blob/master/src/glue/glue.ts"
click node_messages "https://github.com/ngxson/wllama/blob/master/src/glue/messages.ts"
click node_wasm "https://github.com/ngxson/wllama/blob/master/src/wasm/wllama.js"
click node_nativecpp "https://github.com/ngxson/wllama/blob/master/cpp/wllama.cpp"
click node_context "https://github.com/ngxson/wllama/blob/master/cpp/wllama-context.h"
click node_filesystem "https://github.com/ngxson/wllama/blob/master/cpp/wllama-fs.h"
click node_gluecpp "https://github.com/ngxson/wllama/blob/master/cpp/glue.hpp"
click node_llamacpp "https://github.com/ngxson/wllama/blob/master/llama.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_oai,node_browser toneBlue
class node_hf,node_model,node_cache,node_storage,node_cos,node_opfs toneAmber
class node_worker,node_workerjs,node_protocol,node_messages,node_wasm toneMint
class node_nativecpp,node_context,node_filesystem,node_gluecpp,node_llamacpp toneRose
class node_user,node_huggingface toneIndigo
```

## Components

### Public API

- [Wllama API](https://github.com/ngxson/wllama/blob/master/src/wllama.ts): `src/wllama.ts`
- [OpenAI types](https://github.com/ngxson/wllama/blob/master/src/types/oai-compat.ts): `src/types/oai-compat.ts`

### Model acquisition

- [Hugging Face source](https://github.com/ngxson/wllama/blob/master/src/huggingface.ts): `src/huggingface.ts`
- [Model manager](https://github.com/ngxson/wllama/blob/master/src/model-manager.ts): `src/model-manager.ts`
- [Download cache](https://github.com/ngxson/wllama/blob/master/src/cache-manager.ts): `src/cache-manager.ts`
- [Storage backends](https://github.com/ngxson/wllama/tree/master/src/storage): `src/storage`
- [COS backend](https://github.com/ngxson/wllama/blob/master/src/storage/cos.ts): `src/storage/cos.ts`
- [OPFS backend](https://github.com/ngxson/wllama/blob/master/src/storage/opfs.ts): `src/storage/opfs.ts`

### Inference runtime

- [Worker proxy](https://github.com/ngxson/wllama/blob/master/src/worker.ts): `src/worker.ts`
- [Worker runtime code](https://github.com/ngxson/wllama/blob/master/src/workers-code/generated.ts): `src/workers-code/generated.ts`
- [Glue protocol](https://github.com/ngxson/wllama/blob/master/src/glue/glue.ts): `src/glue/glue.ts`
- [Message schemas](https://github.com/ngxson/wllama/blob/master/src/glue/messages.ts): `src/glue/messages.ts`
- [Wasm module](https://github.com/ngxson/wllama/blob/master/src/wasm/wllama.js): `src/wasm/wllama.js`

### Native engine

- [C++ bindings](https://github.com/ngxson/wllama/blob/master/cpp/wllama.cpp): `cpp/wllama.cpp`
- [Inference context](https://github.com/ngxson/wllama/blob/master/cpp/wllama-context.h): `cpp/wllama-context.h`
- [Native filesystem](https://github.com/ngxson/wllama/blob/master/cpp/wllama-fs.h): `cpp/wllama-fs.h`
- [Native glue](https://github.com/ngxson/wllama/blob/master/cpp/glue.hpp): `cpp/glue.hpp`
- [llama.cpp engine](https://github.com/ngxson/wllama/blob/master/llama.cpp): `llama.cpp`

### Other components

- **Application developer**
- **Hugging Face**
- **Browser**

## Connections

- Application developer → Wllama API: calls (evidence: `README.md`)
- Wllama API → OpenAI types: uses types (evidence: `src/wllama.ts`)
- Wllama API → Hugging Face source: resolves model (evidence: `src/wllama.ts`)
- Wllama API → Model manager: loads model (evidence: `src/wllama.ts`)
- Hugging Face source → Hugging Face: fetches metadata (evidence: `README.md`)
- Model manager → Download cache: downloads files (evidence: `src/model-manager.ts`)
- Download cache → Hugging Face source: checks file hash (evidence: `src/cache-manager.ts`)
- Download cache → COS backend: selects backend (evidence: `src/cache-manager.ts`)
- Download cache → Storage backends: uses interface (evidence: `src/cache-manager.ts`)
- COS backend → OPFS backend: references backend (evidence: `src/storage/cos.ts`)
- Wllama API → Worker proxy: dispatches inference (evidence: `src/wllama.ts`)
- Worker proxy → Glue protocol: serializes messages (evidence: `src/worker.ts`)
- Glue protocol → Message schemas: uses schemas (evidence: `src/glue/glue.ts`)
- Worker proxy → Worker runtime code: loads runtime code (evidence: `src/worker.ts`)
- Browser → Worker proxy: runs worker (evidence: `README.md`)
- Worker proxy → Wasm module: loads module
- Wasm module → C++ bindings: invokes bindings
- C++ bindings → Inference context: uses context (evidence: `cpp/wllama.cpp`)
- C++ bindings → Native filesystem: uses filesystem (evidence: `cpp/wllama.cpp`)
- Inference context → Native glue: uses protocol (evidence: `cpp/wllama-context.h`)
- Inference context → llama.cpp engine: runs inference
- Wllama API → Inference context: supports inference features
