# mlc-ai/web-llm architecture

> Architecture diagram of the GitHub repository mlc-ai/web-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mlc-ai/web-llm
- Repository: https://github.com/mlc-ai/web-llm

## Overview

WebLLM is a browser-based LLM inference engine. An application creates and loads a model through the engine, then submits chat, completion, or embedding requests and receives results, optionally as a stream. The engine coordinates model assets, tokenization, and inference; worker variants move engine work off the main thread. The README also documents structured generation, model customization, and several browser cache backends. Resumable generation and storage links reflect the sampled engine and session code; other unsampled subsystem wiring is omitted.

Main entry points: [`src/engine.ts`](https://github.com/mlc-ai/web-llm/blob/main/src/engine.ts) (MLCEngine), [`src/resumable/session_store.ts`](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/session_store.ts) (Session Store), [`src/web_worker.ts`](https://github.com/mlc-ai/web-llm/blob/main/src/web_worker.ts) (Worker Handler), [`src/cache_util.ts`](https://github.com/mlc-ai/web-llm/blob/main/src/cache_util.ts) (Model Cache)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Inference API"]
  node_engine["MLCEngine<br/>[engine.ts]"]
  node_protocols["OpenAI Protocols<br/>[index.ts]"]
end

subgraph group_model["Model Runtime"]
  node_chat["Chat Pipeline<br/>[llm_chat.ts]"]
  node_config["Model Config<br/>[config.ts]"]
  node_cache["Model Cache<br/>[cache_util.ts]"]
  node_integrity["Integrity Checks<br/>[integrity.ts]"]
  node_tokenizer["Tokenizer<br/>[cache_util.ts]"]
  node_runtime["WebGPU Runtime"]
end

subgraph group_execution["Worker Execution"]
  node_worker_handler["Worker Handler<br/>[web_worker.ts]"]
  node_worker_client["Worker Client<br/>[web_worker.ts]"]
  node_service_worker["Service Worker Engine<br/>[service_worker.ts]"]
  node_extension_worker["Extension Worker"]
  node_message["Worker Messages<br/>[message.ts]"]
end

subgraph group_resume["Resumable Generation"]
  node_coordinator["Resume Coordinator<br/>[coordinator.ts]"]
  node_session_store["Session Store<br/>[session_store.ts]"]
  node_opfs[("OPFS File Store<br/>[opfs_file_store.ts]")]
  node_replay["Journal Replay<br/>[replay.ts]"]
  node_probe["Resume Probe<br/>[session_probe.ts]"]
  node_checkpoint["Checkpoint Writer"]
end

node_app(("Web Application"))
node_assets["Model Assets"]
node_browser_cache[("Browser Cache")]

node_app -->|"creates and requests"| node_engine
node_engine -->|"accepts and returns"| node_protocols
node_engine -->|"reads model settings"| node_config
node_engine -->|"loads cached assets"| node_cache
node_engine -->|"loads tokenizer"| node_tokenizer
node_engine -->|"verifies assets"| node_integrity
node_engine -->|"runs generation"| node_chat
node_engine -->|"initializes inference"| node_runtime
node_chat -->|"executes model operations"| node_runtime
node_cache -->|"reads and writes"| node_browser_cache
node_cache -->|"fetches model files"| node_assets
node_engine -->|"coordinates resume"| node_coordinator
node_coordinator -->|"manages sessions"| node_session_store
node_session_store -->|"reads and writes"| node_opfs
node_session_store -->|"loads replay state"| node_replay
node_session_store -->|"reads checkpoints"| node_checkpoint
node_probe -->|"probes replay state"| node_replay
node_app -.->|"uses worker engine"| node_worker_client
node_worker_client -->|"sends requests"| node_message
node_message -->|"dispatches requests"| node_worker_handler
node_worker_handler -->|"calls engine"| node_engine
node_service_worker -->|"extends handler"| node_worker_handler
node_extension_worker -->|"extends handler"| node_worker_handler

click node_engine "https://github.com/mlc-ai/web-llm/blob/main/src/engine.ts"
click node_protocols "https://github.com/mlc-ai/web-llm/blob/main/src/openai_api_protocols/index.ts"
click node_chat "https://github.com/mlc-ai/web-llm/blob/main/src/llm_chat.ts"
click node_config "https://github.com/mlc-ai/web-llm/blob/main/src/config.ts"
click node_cache "https://github.com/mlc-ai/web-llm/blob/main/src/cache_util.ts"
click node_integrity "https://github.com/mlc-ai/web-llm/blob/main/src/integrity.ts"
click node_tokenizer "https://github.com/mlc-ai/web-llm/blob/main/src/cache_util.ts"
click node_worker_handler "https://github.com/mlc-ai/web-llm/blob/main/src/web_worker.ts"
click node_worker_client "https://github.com/mlc-ai/web-llm/blob/main/src/web_worker.ts"
click node_service_worker "https://github.com/mlc-ai/web-llm/blob/main/src/service_worker.ts"
click node_extension_worker "https://github.com/mlc-ai/web-llm/blob/main/src/extension_service_worker.ts"
click node_message "https://github.com/mlc-ai/web-llm/blob/main/src/message.ts"
click node_coordinator "https://github.com/mlc-ai/web-llm/blob/main/src/resumable/coordinator.ts"
click node_session_store "https://github.com/mlc-ai/web-llm/blob/main/src/resumable/session_store.ts"
click node_opfs "https://github.com/mlc-ai/web-llm/blob/main/src/resumable/opfs_file_store.ts"
click node_replay "https://github.com/mlc-ai/web-llm/blob/main/src/resumable/replay.ts"
click node_probe "https://github.com/mlc-ai/web-llm/blob/main/src/resumable/session_probe.ts"
click node_checkpoint "https://github.com/mlc-ai/web-llm/blob/main/src/resumable/checkpoint_writer.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_engine,node_protocols toneBlue
class node_chat,node_config,node_cache,node_integrity,node_tokenizer,node_runtime,node_browser_cache toneAmber
class node_worker_handler,node_worker_client,node_service_worker,node_extension_worker,node_message toneMint
class node_coordinator,node_session_store,node_opfs,node_replay,node_probe,node_checkpoint toneRose
class node_app,node_assets toneIndigo
```

## Components

### Inference API

- [MLCEngine](https://github.com/mlc-ai/web-llm/blob/main/src/engine.ts): `src/engine.ts`
- [OpenAI Protocols](https://github.com/mlc-ai/web-llm/blob/main/src/openai_api_protocols/index.ts): `src/openai_api_protocols/index.ts`

### Model Runtime

- [Chat Pipeline](https://github.com/mlc-ai/web-llm/blob/main/src/llm_chat.ts): `src/llm_chat.ts`
- [Model Config](https://github.com/mlc-ai/web-llm/blob/main/src/config.ts): `src/config.ts`
- [Model Cache](https://github.com/mlc-ai/web-llm/blob/main/src/cache_util.ts): `src/cache_util.ts`
- [Integrity Checks](https://github.com/mlc-ai/web-llm/blob/main/src/integrity.ts): `src/integrity.ts`
- [Tokenizer](https://github.com/mlc-ai/web-llm/blob/main/src/cache_util.ts): `src/cache_util.ts`
- **WebGPU Runtime**

### Worker Execution

- [Worker Handler](https://github.com/mlc-ai/web-llm/blob/main/src/web_worker.ts): `src/web_worker.ts`
- [Worker Client](https://github.com/mlc-ai/web-llm/blob/main/src/web_worker.ts): `src/web_worker.ts`
- [Service Worker Engine](https://github.com/mlc-ai/web-llm/blob/main/src/service_worker.ts): `src/service_worker.ts`
- [Extension Worker](https://github.com/mlc-ai/web-llm/blob/main/src/extension_service_worker.ts): `src/extension_service_worker.ts`
- [Worker Messages](https://github.com/mlc-ai/web-llm/blob/main/src/message.ts): `src/message.ts`

### Resumable Generation

- [Resume Coordinator](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/coordinator.ts): `src/resumable/coordinator.ts`
- [Session Store](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/session_store.ts): `src/resumable/session_store.ts`
- [OPFS File Store](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/opfs_file_store.ts): `src/resumable/opfs_file_store.ts`
- [Journal Replay](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/replay.ts): `src/resumable/replay.ts`
- [Resume Probe](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/session_probe.ts): `src/resumable/session_probe.ts`
- [Checkpoint Writer](https://github.com/mlc-ai/web-llm/blob/main/src/resumable/checkpoint_writer.ts): `src/resumable/checkpoint_writer.ts`

### Other components

- **Web Application**
- **Model Assets**
- **Browser Cache**

## Connections

- Web Application → MLCEngine: creates and requests
- MLCEngine → OpenAI Protocols: accepts and returns
- MLCEngine → Model Config: reads model settings
- MLCEngine → Model Cache: loads cached assets
- MLCEngine → Tokenizer: loads tokenizer
- MLCEngine → Integrity Checks: verifies assets
- MLCEngine → Chat Pipeline: runs generation
- MLCEngine → WebGPU Runtime: initializes inference
- Chat Pipeline → WebGPU Runtime: executes model operations
- Model Cache → Browser Cache: reads and writes
- Model Cache → Model Assets: fetches model files
- MLCEngine → Resume Coordinator: coordinates resume
- Resume Coordinator → Session Store: manages sessions
- Session Store → OPFS File Store: reads and writes
- Session Store → Journal Replay: loads replay state
- Session Store → Checkpoint Writer: reads checkpoints
- Resume Probe → Journal Replay: probes replay state
- Web Application → Worker Client: uses worker engine
- Worker Client → Worker Messages: sends requests
- Worker Messages → Worker Handler: dispatches requests
- Worker Handler → MLCEngine: calls engine
- Service Worker Engine → Worker Handler: extends handler
- Extension Worker → Worker Handler: extends handler
