How 0hq/webgpt works
Project type and purpose
• Project type: A client-side, single-page web application (vanilla HTML + JavaScript) that implements a transformer model inference engine using WebGPU.
• Purpose: Educational/proof-of-concept demo (“WebGPT”) showing how to load model weights, tokenize input, run inference on the GPU, and render results entirely in the browser. It also includes offline conversion scripts to prepare PyTorch checkpoints for browser consumption.
File structure
• Top-level:
• index.html – application entry point
• globals.js – constants, WebGPU adapter/device initialization
• tokenizer.js – text ↔ token encoding/decoding
• model.js & condensed/condensed.js – core transformer inference logic (attention, MLP blocks, layer norms)
• visuals.js & instructions.js – UI for input, controls, result rendering
• weights/ – binary weight files and JSON parameter descriptors for GPT-2 and “better_shakespeare” toy model
• other/conversion_scripts/ – Python scripts to convert PyTorch checkpoints into browser-loadable weights
• other/validation/ – JSON test sets and JS to verify correctness
• Patterns:
• Modular JS: separate concerns (tokenization, model math, UI)
• WebGPU pipelines: kernels fused for matmuls & transformer ops
• Offline vs. runtime: conversion scripts run offline; browser loads static binaries via Git LFS
README for additional insights
• Confirms WebGPU usage, performance stats, supported model sizes
• Highlights conversion scripts for custom models
• Roadmap shows where compute kernels and buffers are managed