# omarhimada/local-llm-onnx architecture

> Architecture diagram of the GitHub repository omarhimada/local-llm-onnx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/omarhimada/local-llm-onnx
- Repository: https://github.com/omarhimada/local-llm-onnx

## Overview

1. **Overall shape**
   - This is a **local Windows desktop chat application** built with **WPF** (`App.xaml`, `MainWindow.xaml`, `LoadingWindow.xaml`).
   - The app is intentionally **offline-only**: no HTTP APIs, no websocket middle layer, no external inference service. The UI talks directly to local ONNX model runtimes and local assets.

2. **Startup and app shell**
   - `App.xaml.cs` is the entry point, with a dedicated **loading screen** (`LoadingWindow`) used while models and runtime state are prepared.
   - `Initialization/EnsureModelsArePresent.cs` suggests the startup path validates that required model files exist before the main UI becomes usable.

3. **UI layer**
   - `MainWindow` is the primary chat interface.
   - `State/Thinking/FloatingAdorner.cs` implies a visual “thinking” indicator overlay during inference.
   - The UI is likely driven by state objects rather than direct model calls, keeping the window code thin and event-oriented.

4. **Conversation / interaction layer**
   - `Interact/LinearCommunication.cs` is the core orchestration path for turn-by-turn chat.
   - This layer likely assembles user input, context, model prompt, and any retrieved memory before sending a request to the local LLM.
   - The name “LinearCommunication” suggests a sequential prompt-response flow rather than an agent graph or multi-worker architecture.

5. **Model runtime boundary**
   - The repository ships model artifacts under `ONNX/`, including:
     - `gpt-oss-20b` for the main chat model,
     - `all-MiniLM-L6-v2-onnx` for embeddings,
     - `med-gemma-27b` and `phi4`-related support in utility code,
     - Stable Diffusion assets under `ONNX/SD`.
   - This is the main compute boundary: the app does not call a server; it loads local model directories directly from disk.

6. **Chat model abstraction**
   - `State/ModelState.cs` and `Utility/ModelSpecific/*` indicate the app has a layer for **model-specific behavior**.
   - That abstraction is important because different models need different prompt formatting, token handling, or generation settings.
   - The presence of `gpt-oss-20b`, `MedGemma`, and `Phi4` helpers implies the app is designed to support multiple local LLM families, even if one is the default.

7. **Embedding / retrieval memory**
   - `Memory/Discussion.cs`, `Memory/Remember.cs`, and `Memory/MiniEmbedder.cs` form the memory subsystem.
   - `State/EmbedderState.cs` shows embeddings are treated as a first-class runtime resource, separate from the main chat model.
   - The README describes a retrieval-augmented memory goal: store discussions locally, then search similar past items before responding. This is a key architectural flow: **user input → embed/search memory → augment prompt → generate answer**.

8. **Parsing and incremental state**
   - `State/IncrementalParserState.cs` suggests the app can consume partial model output and maintain parser progress over time.
   - This is often used for streaming-like UI updates, structured output detection, or progressive rendering of generated text.

9. **Utility and formatting layer**
   - `Utility/` contains reusable support code for text and output shaping:
     - `Md.cs`, `MdFd.cs`, `Markdown`-related syntax utilities,
     - `Utility/Syntax/*` for highlighting and language registry,
     - `Utility/StringCleaner.cs`, `Base64.cs`, and `Fd.cs` for general helpers.
   - These utilities sit between raw model output and the rendered UI, especially where markdown or code snippets need formatting.

10. **Template and code-generation tooling**
    - `Utility/J2CS/*` is a notable subsystem: it parses **Jinja-like templates** and generates C#-side representations.
    - The presence of `chat_template.jinja` inside `ONNX/gpt-oss-20b` makes this especially relevant: prompt/chat templates appear to be converted or interpreted locally rather than delegated to another toolchain.

11. **Image generation sidecar**
    - `SD/Diffusion.cs`, `DiffusionOptions.cs`, and `LmsScheduler.cs` indicate a separate **stable diffusion inference path** exists in the same app.
    - This is a parallel local-model capability, likely optional and isolated from the chat flow, but sharing the same no-network architecture and ONNX asset pattern.

12. **State and lifecycle boundaries**
    - `State/*` holds runtime state objects for loaded models, embeddings, and parser progress.
    - That separation implies the app is organized around **UI → state → inference/memory subsystems**, rather than letting view code manage model lifecycles directly.
    - For an architecture diagram, the most important boundary is that the **main window does not own the model files or memory logic directly**; those responsibilities are split into initialization, state, interaction, and memory layers.

13. **External dependencies and deployment**
    - The repo is packaged as a desktop app (`OLLM.csproj`, `.slnx`, publish profile), with the model payload stored locally under `ONNX/`.
    - The main operational dependency is **disk availability and local runtime support for ONNX Generative AI**, not any cloud service.
    - The README’s setup note confirms the app expects large model artifacts to be present locally, making model files part of the deployment boundary.

14. **Practical architecture summary**
    - A good diagram would show:
      - **WPF shell** on the left,
      - **interaction/orchestration** in the middle,
      - **local ONNX model runtimes** plus **memory/embedding** services as separate back-end subsystems,
      - and **model artifacts on disk** as the persistent runtime dependency.
    - The dominant data flow is: **user prompt → optional memory retrieval/embedding → prompt assembly → local LLM inference → formatted response in UI**.

Main entry points: [`Interact/LinearCommunication.cs`](https://github.com/omarhimada/local-llm-onnx/blob/master/Interact/LinearCommunication.cs) (Turn flow), [`LoadingWindow.xaml.cs`](https://github.com/omarhimada/local-llm-onnx/blob/master/LoadingWindow.xaml.cs) (Loading screen), [`MainWindow.xaml.cs`](https://github.com/omarhimada/local-llm-onnx/blob/master/MainWindow.xaml.cs) (Main chat), [`State/ModelState.cs`](https://github.com/omarhimada/local-llm-onnx/blob/master/State/ModelState.cs) (Model state)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["WPF shell"]
  node_app["App shell<br/>WPF entry<br/>[App.xaml.cs]"]
  node_loading["Loading screen<br/>WPF window"]
  node_mainwin["Main chat<br/>WPF window<br/>[MainWindow.xaml.cs]"]
  node_thinking(("Thinking overlay<br/>UI adornment<br/>[FloatingAdorner.cs]"))
end

subgraph group_runtime["Conversation runtime"]
  node_initmodels["Model check<br/>startup validation"]
  node_linearcomm["Turn flow<br/>orchestrator"]
end

subgraph group_memory["Memory search"]
  node_discussion["Discussion store<br/>memory record<br/>[Discussion.cs]"]
  node_remember["Recall logic<br/>retrieval<br/>[Remember.cs]"]
  node_miniembed["Mini embedder<br/>embedding wrapper<br/>[MiniEmbedder.cs]"]
end

subgraph group_state["Runtime state"]
  node_modelstate["Model state<br/>runtime state<br/>[ModelState.cs]"]
  node_embedstate["Embed state<br/>runtime state<br/>[EmbedderState.cs]"]
  node_parserstate["Parser state<br/>incremental parse"]
end

subgraph group_models["Local models"]
  node_gptoss[("GPT-OSS<br/>LLM model")]
  node_embedmodel[("Embed model<br/>embedding model")]
  node_medgemma[("MedGemma<br/>LLM model")]
  node_sdpath[("SD assets<br/>diffusion model")]
end

subgraph group_image["Image generation"]
  node_diffusion["Diffusion<br/>image pipeline<br/>[Diffusion.cs]"]
  node_scheduler["LMS scheduler<br/>sampling scheduler<br/>[LmsScheduler.cs]"]
end

subgraph group_utils["Formatting and codegen"]
  node_mdfmt["Markdown fmt<br/>text formatting<br/>[Md.cs]"]
  node_syntax["Syntax tools<br/>highlighting<br/>[Highlighter.cs]"]
  node_j2cs["Jinja parser<br/>template compiler<br/>[Converter.cs]"]
end

node_app -->|"shows startup"| node_loading
node_app -->|"validates models"| node_initmodels
node_loading -->|"prepares"| node_modelstate
node_loading -->|"prepares"| node_embedstate
node_loading -->|"opens"| node_mainwin
node_mainwin -->|"drives chat"| node_linearcomm
node_mainwin -->|"shows status"| node_thinking
node_linearcomm -->|"retrieves memory"| node_remember
node_linearcomm -->|"uses chat model"| node_modelstate
node_linearcomm -->|"tracks output"| node_parserstate
node_remember -->|"queries"| node_discussion
node_remember -->|"embeds query"| node_miniembed
node_miniembed -->|"loads from disk"| node_embedmodel
node_modelstate -->|"binds to"| node_gptoss
node_embedstate -->|"binds to"| node_embedmodel
node_modelstate -.->|"supports"| node_medgemma
node_linearcomm -->|"prompts"| node_gptoss
node_linearcomm -->|"formats response"| node_mdfmt
node_mdfmt -->|"highlights code"| node_syntax
node_gptoss -.->|"uses template"| node_j2cs
node_diffusion -->|"loads assets"| node_sdpath
node_diffusion -->|"uses scheduler"| node_scheduler
node_mainwin -.->|"optional image flow"| node_diffusion

click node_app "https://github.com/omarhimada/local-llm-onnx/blob/master/App.xaml.cs"
click node_loading "https://github.com/omarhimada/local-llm-onnx/blob/master/LoadingWindow.xaml.cs"
click node_mainwin "https://github.com/omarhimada/local-llm-onnx/blob/master/MainWindow.xaml.cs"
click node_thinking "https://github.com/omarhimada/local-llm-onnx/blob/master/State/Thinking/FloatingAdorner.cs"
click node_initmodels "https://github.com/omarhimada/local-llm-onnx/blob/master/Initialization/EnsureModelsArePresent.cs"
click node_linearcomm "https://github.com/omarhimada/local-llm-onnx/blob/master/Interact/LinearCommunication.cs"
click node_discussion "https://github.com/omarhimada/local-llm-onnx/blob/master/Memory/Discussion.cs"
click node_remember "https://github.com/omarhimada/local-llm-onnx/blob/master/Memory/Remember.cs"
click node_miniembed "https://github.com/omarhimada/local-llm-onnx/blob/master/Memory/MiniEmbedder.cs"
click node_modelstate "https://github.com/omarhimada/local-llm-onnx/blob/master/State/ModelState.cs"
click node_embedstate "https://github.com/omarhimada/local-llm-onnx/blob/master/State/EmbedderState.cs"
click node_parserstate "https://github.com/omarhimada/local-llm-onnx/blob/master/State/IncrementalParserState.cs"
click node_gptoss "https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/gpt-oss-20b"
click node_embedmodel "https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/Embed/all-MiniLM-L6-v2-onnx"
click node_medgemma "https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/med-gemma-27b"
click node_sdpath "https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/SD/0nnX00Aammnpdxebr"
click node_diffusion "https://github.com/omarhimada/local-llm-onnx/blob/master/SD/Diffusion.cs"
click node_scheduler "https://github.com/omarhimada/local-llm-onnx/blob/master/SD/LmsScheduler.cs"
click node_mdfmt "https://github.com/omarhimada/local-llm-onnx/blob/master/Utility/Md.cs"
click node_syntax "https://github.com/omarhimada/local-llm-onnx/blob/master/Utility/Syntax/Highlighter.cs"
click node_j2cs "https://github.com/omarhimada/local-llm-onnx/blob/master/Utility/J2CS/Converter.cs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_loading,node_mainwin,node_thinking,node_mdfmt,node_syntax,node_j2cs toneBlue
class node_initmodels,node_linearcomm toneAmber
class node_discussion,node_remember,node_miniembed toneMint
class node_modelstate,node_embedstate,node_parserstate toneRose
class node_gptoss,node_embedmodel,node_medgemma,node_sdpath toneIndigo
class node_diffusion,node_scheduler toneTeal
```

## Components

### WPF shell

Desktop windows and app entry points.

- [App shell](https://github.com/omarhimada/local-llm-onnx/blob/master/App.xaml.cs): WPF entry · Application startup and top-level lifecycle. · `App.xaml.cs`
- [Loading screen](https://github.com/omarhimada/local-llm-onnx/blob/master/LoadingWindow.xaml.cs): WPF window · Startup window shown while models and runtime state initialize. · `LoadingWindow.xaml.cs`
- [Main chat](https://github.com/omarhimada/local-llm-onnx/blob/master/MainWindow.xaml.cs): WPF window · Primary chat interface for user interaction. · `MainWindow.xaml.cs`
- [Thinking overlay](https://github.com/omarhimada/local-llm-onnx/blob/master/State/Thinking/FloatingAdorner.cs): UI adornment · Visual indicator for ongoing inference or processing. · `State/Thinking/FloatingAdorner.cs`

### Conversation runtime

Turn-by-turn chat orchestration and prompt assembly.

- [Model check](https://github.com/omarhimada/local-llm-onnx/blob/master/Initialization/EnsureModelsArePresent.cs): startup validation · Ensures required model files are present before use. · `Initialization/EnsureModelsArePresent.cs`
- [Turn flow](https://github.com/omarhimada/local-llm-onnx/blob/master/Interact/LinearCommunication.cs): orchestrator · Sequential chat orchestration for prompts, context, and responses. · `Interact/LinearCommunication.cs`

### Memory search

Local discussion storage and embedding retrieval.

- [Discussion store](https://github.com/omarhimada/local-llm-onnx/blob/master/Memory/Discussion.cs): memory record · Persistent local conversation and retrieval corpus. · `Memory/Discussion.cs`
- [Recall logic](https://github.com/omarhimada/local-llm-onnx/blob/master/Memory/Remember.cs): retrieval · Searches and selects relevant past discussions. · `Memory/Remember.cs`
- [Mini embedder](https://github.com/omarhimada/local-llm-onnx/blob/master/Memory/MiniEmbedder.cs): embedding wrapper · Embeds text using the local embedding model. · `Memory/MiniEmbedder.cs`

### Runtime state

Loaded model and parser state shared across the app.

- [Model state](https://github.com/omarhimada/local-llm-onnx/blob/master/State/ModelState.cs): runtime state · Tracks loaded chat model and related runtime configuration. · `State/ModelState.cs`
- [Embed state](https://github.com/omarhimada/local-llm-onnx/blob/master/State/EmbedderState.cs): runtime state · Tracks embedding runtime availability and state. · `State/EmbedderState.cs`
- [Parser state](https://github.com/omarhimada/local-llm-onnx/blob/master/State/IncrementalParserState.cs): incremental parse · Maintains progress while consuming partial model output. · `State/IncrementalParserState.cs`

### Local models

On-disk ONNX and model artifacts.

- [GPT-OSS](https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/gpt-oss-20b): LLM model · Main local chat model artifacts and chat template. · `ONNX/gpt-oss-20b`
- [Embed model](https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/Embed/all-MiniLM-L6-v2-onnx): embedding model · Local sentence embedding model used for memory retrieval. · `ONNX/Embed/all-MiniLM-L6-v2-onnx`
- [MedGemma](https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/med-gemma-27b): LLM model · Alternate model family supported by model-specific helpers. · `ONNX/med-gemma-27b`
- [SD assets](https://github.com/omarhimada/local-llm-onnx/tree/master/ONNX/SD/0nnX00Aammnpdxebr): diffusion model · Stable Diffusion model bundle and tokenizer/runtime assets. · `ONNX/SD/0nnX00Aammnpdxebr`

### Image generation

Optional Stable Diffusion inference path.

- [Diffusion](https://github.com/omarhimada/local-llm-onnx/blob/master/SD/Diffusion.cs): image pipeline · Local Stable Diffusion inference path. · `SD/Diffusion.cs`
- [LMS scheduler](https://github.com/omarhimada/local-llm-onnx/blob/master/SD/LmsScheduler.cs): sampling scheduler · Scheduler used by the diffusion pipeline. · `SD/LmsScheduler.cs`

### Formatting and codegen

Shared text utilities and template tooling.

- [Markdown fmt](https://github.com/omarhimada/local-llm-onnx/blob/master/Utility/Md.cs): text formatting · Markdown and response shaping utilities. · `Utility/Md.cs`
- [Syntax tools](https://github.com/omarhimada/local-llm-onnx/blob/master/Utility/Syntax/Highlighter.cs): highlighting · Code highlighting and language registry utilities. · `Utility/Syntax/Highlighter.cs`
- [Jinja parser](https://github.com/omarhimada/local-llm-onnx/blob/master/Utility/J2CS/Converter.cs): template compiler · Parses Jinja-like templates and generates C# representations. · `Utility/J2CS/Converter.cs`

## Connections

- App shell → Loading screen: shows startup · Entry point opens the loading window during initialization.
- App shell → Model check: validates models · Startup checks that local model assets are available.
- Loading screen → Model state: prepares · Loading screen waits while runtime state is initialized.
- Loading screen → Embed state: prepares · Embedding runtime is brought up during startup.
- Loading screen → Main chat: opens · Main window becomes available after initialization completes.
- Main chat → Turn flow: drives chat · UI hands user turns to the conversation orchestrator.
- Main chat → Thinking overlay: shows status · Window displays processing feedback during inference.
- Turn flow → Recall logic: retrieves memory · Conversation flow consults recalled discussions before responding.
- Turn flow → Model state: uses chat model · Orchestration relies on the loaded LLM state.
- Turn flow → Parser state: tracks output · Incremental parsing state supports partial or structured responses.
- Recall logic → Discussion store: queries · Retrieval logic searches stored discussions.
- Recall logic → Mini embedder: embeds query · Relevant search requires text embeddings.
- Mini embedder → Embed model: loads from disk · Embedding wrapper uses the local ONNX embedding model.
- Model state → GPT-OSS: binds to · Main chat state points at the GPT-OSS model bundle.
- Embed state → Embed model: binds to · Embedding state points at the local embedding model bundle.
- Model state → MedGemma: supports · Model-specific behavior can target alternate LLM families.
- Turn flow → GPT-OSS: prompts · Final prompt assembly is sent to the local chat model.
- Turn flow → Markdown fmt: formats response · Generated text is shaped for display in the UI.
- Markdown fmt → Syntax tools: highlights code · Markdown rendering may include syntax-aware code formatting.
- GPT-OSS → Jinja parser: uses template · Chat template tooling supports model prompt formatting.
- Diffusion → SD assets: loads assets · Image generation reads local Stable Diffusion files from disk.
- Diffusion → LMS scheduler: uses scheduler · Diffusion inference depends on LMS scheduling.
- Main chat → Diffusion: optional image flow · Desktop app can invoke the separate local image pipeline.