# nadchif/in-browser-llm-inference architecture

> Architecture diagram of the GitHub repository nadchif/in-browser-llm-inference, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nadchif/in-browser-llm-inference
- Repository: https://github.com/nadchif/in-browser-llm-inference

## Overview

##### Project type and purpose
- This is a single-page, front-end only web application built with React and Vite.
- It hosts and runs large language models (LLMs) entirely in the browser (zero backend) using the Wllama WebAssembly inference engine.
- A Node script (`download-model.cjs`) pulls down model files at build or setup time into `public/models`, which are then served as static assets.

##### File structure
- Top-level files:
– `index.html`, `vite.config.js`: SPA entry point and Vite configuration.
– `package.json`, `package-lock.json`: dependency and script definitions (`npm run dev`, etc.).
– `download-model.cjs`: Node utility to fetch LLM model files.
- `public/models/`: static directory for downloaded model artifacts.
- `src/`: core application code
– `main.jsx`: mounts the React app.
– `App.jsx`: top-level component orchestrating UI and model inference.
– `components/`: UI building blocks (Dropdowns, Buttons, Loader, Markdown renderer, Footer, etc.).
– `lib/wllama.js`: wraps the Wllama WebAssembly module for model loading and inference.
– `assets/`, `index.css`: styling and static assets.

##### Key points from the README
- No server-side API or database—everything runs in-browser.
- Relies on Wllama (WebAssembly) to perform inference on downloaded LLM checkpoints.
- Models come from SmolLm (HuggingFace) or Llama 3.2 (Meta) and live in `public/models`.

##### Main components
a. Static Host / Dev Server
– Vite dev server in development or Vercel/Netlify for production
– Serves `index.html`, JavaScript bundles, CSS, and `public/models/*`
b. Browser Client
– UI Layer: React components (Dropdown, Chat UI, Loader, etc.)
– App Logic: `App.jsx` handles user input, manages state, and calls the inference engine
– Inference Layer: Wllama WebAssembly module (wrapped by `src/lib/wllama.js`)
– Model Loader: fetches model files from the static host at runtime
c. External Model Sources (build time)
– `download-model.cjs` contacts HuggingFace/Meta to download model files into `public/models`
d. Static Assets
– Pre-downloaded model checkpoints stored under `public/models`

##### Relationships and data flow
1. Developer runs `npm run dev`
→ Vite server starts, optionally `download-model.cjs` downloads models
2. Browser requests `index.html` and JS bundles
3. User enters a prompt in the UI
4. `App.jsx` passes text to the Wllama wrapper
5. Wllama loads the model files (via fetch from `public/models`) into WebAssembly
6. Inference runs in-browser, returns tokens/response
7. UI renders the response

##### Architectural patterns and principles
- Separation of concerns: UI components vs. inference engine vs. static model storage
- Single responsibility: each React component handles one piece of UI
- Static hosting: no dynamic server, simple build & deploy pipeline
- Modular inference wrapper in `lib/wllama.js` allows swapping out engines in future

##### Technologies and frameworks to label
- React (JSX, components)
- Vite (build & dev server)
- Wllama (WebAssembly LLM inference)
- Node.js script for model download
- Static hosting (e.g., Vercel/Netlify)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Build-Time Model Download
    subgraph "Build-Time Process" 
        direction TB
        DL["Model Download Script\ndownload-model.cjs"]:::build
        EXTSVC["External Model Sources\n(HuggingFace/Meta)"]:::ext
        STORAGE_BUILD["public/models\n(Model Storage)"]:::storage
        DL -->|"download at setup"| EXTSVC
        EXTSVC -->|"store models in"| STORAGE_BUILD
    end

    %% Static Host / Dev Server
    subgraph "Static Host / Dev Server (Vite/Vercel)" 
        direction TB
        VITE["Vite Config\nvite.config.js"]:::build
        ENTRY["SPA Entry Point\nindex.html"]:::ui
        PKG["Dependency & Script Definitions\npackage.json"]:::build
        STORAGE_SERVE["public/models\n(Static Model Assets)"]:::storage
        VITE --> ENTRY
        VITE --> PKG
        ENTRY --> STORAGE_SERVE
    end

    %% Browser Runtime
    subgraph "Browser Runtime" 
        direction TB
        MAIN["App Bootstrap\nsrc/main.jsx"]:::build
        APP["App Logic & State\nsrc/App.jsx"]:::ui
        subgraph "UI Components" 
            direction TB
            DD["Dropdown\nsrc/components/Dropdown.jsx"]:::ui
            FOOT["Footer\nsrc/components/Footer.jsx"]:::ui
            ICON["IconButton\nsrc/components/IconButton.jsx"]:::ui
            LOADER["Loader\nsrc/components/Loader.jsx"]:::ui
            MD["Markdown Renderer\nsrc/components/Markdown.jsx"]:::ui
            NAV["NavLink\nsrc/components/NavLink.jsx"]:::ui
        end
        LIB["Inference Engine\nWllama Wrapper\nsrc/lib/wllama.js"]:::engine
        STYLE_CSS["Global Styles\nsrc/index.css"]:::build
        ASSETS["Static Assets\ndir: src/assets/"]:::build
        MAIN --> APP
        APP --> DD
        APP --> FOOT
        APP --> ICON
        APP --> LOADER
        APP --> MD
        APP --> NAV
        APP --> LIB
        LIB -->|"fetch models from"| STORAGE_SERVE
        APP -->|"uses styles"| STYLE_CSS
        APP -->|"loads assets"| ASSETS
    end

    %% User Flow
    USER["User"]:::external
    USER -->|"open app"| ENTRY
    USER -->|"enter prompt"| APP
    APP -->|"invoke inference"| LIB
    LIB -->|"run inference"| APP
    APP -->|"render response"| DD

    %% Classes
    classDef ui fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef engine fill:#90EE90,stroke:#333,stroke-width:1px
    classDef storage fill:#FBE7C6,stroke:#333,stroke-width:1px
    classDef ext fill:#FFA500,stroke:#333,stroke-width:1px
    classDef build fill:#FFDEAD,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px

    %% Click Events
    click VITE "https://github.com/nadchif/in-browser-llm-inference/blob/main/vite.config.js"
    click ENTRY "https://github.com/nadchif/in-browser-llm-inference/blob/main/index.html"
    click PKG "https://github.com/nadchif/in-browser-llm-inference/blob/main/package.json"
    click DL "https://github.com/nadchif/in-browser-llm-inference/blob/main/download-model.cjs"
    click STORAGE_BUILD "https://github.com/nadchif/in-browser-llm-inference/tree/main/public/models/"
    click MAIN "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/main.jsx"
    click APP "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/App.jsx"
    click DD "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/components/Dropdown.jsx"
    click FOOT "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/components/Footer.jsx"
    click ICON "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/components/IconButton.jsx"
    click LOADER "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/components/Loader.jsx"
    click MD "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/components/Markdown.jsx"
    click NAV "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/components/NavLink.jsx"
    click LIB "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/lib/wllama.js"
    click STYLE_CSS "https://github.com/nadchif/in-browser-llm-inference/blob/main/src/index.css"
    click ASSETS "https://github.com/nadchif/in-browser-llm-inference/tree/main/src/assets/"
```
