# signalnine/q27 architecture

> Architecture diagram of the GitHub repository signalnine/q27, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/signalnine/q27
- Repository: https://github.com/signalnine/q27
- Video tour (about a minute): https://gitdiagram.com/signalnine/q27/video

## Overview

Quasar is a model-family-specific inference server and runtime, exposing Anthropic-, OpenAI-, and Codex-shaped interfaces over CUDA and Apple Metal backends. The main workflow is a client request through API prompt/token processing, model execution and sampling, then formatted streaming output; tool-call parsing and constrained decoding are a distinct branch. The README also highlights self-speculative decoding, continuous batching, and prefix reuse. Source excerpts are partial, so several runtime links are documented or shown as unverified rather than asserted as direct calls.

Main entry points: [`src/metal/metal_server.cpp`](https://github.com/signalnine/q27/blob/master/src/metal/metal_server.cpp) (Metal API server), [`src/metal/metal_engine.cpp`](https://github.com/signalnine/q27/blob/master/src/metal/metal_engine.cpp) (Metal engine), [`src/toolconstrain.h`](https://github.com/signalnine/q27/blob/master/src/toolconstrain.h) (Tool constraints), [`src/metal/metal_backend.h`](https://github.com/signalnine/q27/blob/master/src/metal/metal_backend.h) (Metal backend)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and decoding"]
  node_server["Metal API server<br/>[metal_server.cpp]"]
  node_api_common["Prompt and tool parsing<br/>[api_common.h]"]
  node_tokenizer["Tokenizer<br/>[tokenizer.cpp]"]
  node_stream["Stream formatting<br/>[stream_format.h]"]
  node_tool_constraint["Tool constraints<br/>[toolconstrain.h]"]
  node_tool_grammar["Tool grammar<br/>[toolgram.h]"]
end

subgraph group_runtime["Inference runtime"]
  node_engine["Metal engine<br/>[metal_engine.cpp]"]
  node_sampling["Sampling<br/>[sampling.h]"]
  node_speculation["Suffix speculation<br/>[suffixdraft.h]"]
  node_batching["CUDA batching<br/>[conductor.h]"]
  node_loader["Model loader<br/>[loader.cpp]"]
end

subgraph group_state["Reuse and state"]
  node_prefix_cache[("Prefix cache<br/>[prefix_cache.h]")]
  node_snapshots[("Disk snapshots")]
end

subgraph group_accel["Compute backends"]
  node_backend["Metal backend<br/>[metal_backend.h]"]
  node_backend_iface["Backend interface<br/>[backend.h]"]
end

subgraph group_deploy["Serving options"]
  node_runpod["RunPod handler<br/>[handler.py]"]
end

node_client(("API client"))

node_client -->|"sends request"| node_server
node_server -.->|"tokenizes prompts"| node_tokenizer
node_server -.->|"builds prompts"| node_api_common
node_server -.->|"dispatches inference"| node_engine
node_server -->|"constrains decoding"| node_tool_constraint
node_tool_constraint -->|"uses grammar"| node_tool_grammar
node_server -.->|"formats responses"| node_stream
node_server -.->|"uses snapshots"| node_snapshots
node_engine -->|"runs operations"| node_backend
node_backend -->|"implements interface"| node_backend_iface
node_engine -->|"samples tokens"| node_sampling
node_engine -->|"uses drafter"| node_speculation
node_engine -.->|"loads model"| node_loader
node_server -.->|"reuses prefixes"| node_prefix_cache
node_server -.->|"batches requests"| node_batching
node_runpod -.->|"invokes service"| node_server

click node_server "https://github.com/signalnine/q27/blob/master/src/metal/metal_server.cpp"
click node_api_common "https://github.com/signalnine/q27/blob/master/src/api_common.h"
click node_tokenizer "https://github.com/signalnine/q27/blob/master/src/tokenizer.cpp"
click node_stream "https://github.com/signalnine/q27/blob/master/src/metal/stream_format.h"
click node_tool_constraint "https://github.com/signalnine/q27/blob/master/src/toolconstrain.h"
click node_tool_grammar "https://github.com/signalnine/q27/blob/master/src/toolgram.h"
click node_engine "https://github.com/signalnine/q27/blob/master/src/metal/metal_engine.cpp"
click node_sampling "https://github.com/signalnine/q27/blob/master/src/sampling.h"
click node_speculation "https://github.com/signalnine/q27/blob/master/src/suffixdraft.h"
click node_batching "https://github.com/signalnine/q27/blob/master/src/conductor.h"
click node_loader "https://github.com/signalnine/q27/blob/master/src/loader.cpp"
click node_prefix_cache "https://github.com/signalnine/q27/blob/master/src/prefix_cache.h"
click node_snapshots "https://github.com/signalnine/q27/blob/master/src/metal/disk_snapshot_store.h"
click node_backend "https://github.com/signalnine/q27/blob/master/src/metal/metal_backend.h"
click node_backend_iface "https://github.com/signalnine/q27/blob/master/src/backend.h"
click node_runpod "https://github.com/signalnine/q27/blob/master/deploy/runpod-serverless/handler.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_api_common,node_tokenizer,node_stream,node_tool_constraint,node_tool_grammar,node_client toneBlue
class node_engine,node_sampling,node_speculation,node_batching,node_loader toneAmber
class node_prefix_cache,node_snapshots toneMint
class node_backend,node_backend_iface toneRose
class node_runpod toneIndigo
```

## Components

### API and decoding

- [Metal API server](https://github.com/signalnine/q27/blob/master/src/metal/metal_server.cpp): `src/metal/metal_server.cpp`
- [Prompt and tool parsing](https://github.com/signalnine/q27/blob/master/src/api_common.h): `src/api_common.h`
- [Tokenizer](https://github.com/signalnine/q27/blob/master/src/tokenizer.cpp): `src/tokenizer.cpp`
- [Stream formatting](https://github.com/signalnine/q27/blob/master/src/metal/stream_format.h): `src/metal/stream_format.h`
- [Tool constraints](https://github.com/signalnine/q27/blob/master/src/toolconstrain.h): `src/toolconstrain.h`
- [Tool grammar](https://github.com/signalnine/q27/blob/master/src/toolgram.h): `src/toolgram.h`

### Inference runtime

- [Metal engine](https://github.com/signalnine/q27/blob/master/src/metal/metal_engine.cpp): `src/metal/metal_engine.cpp`
- [Sampling](https://github.com/signalnine/q27/blob/master/src/sampling.h): `src/sampling.h`
- [Suffix speculation](https://github.com/signalnine/q27/blob/master/src/suffixdraft.h): `src/suffixdraft.h`
- [CUDA batching](https://github.com/signalnine/q27/blob/master/src/conductor.h): `src/conductor.h`
- [Model loader](https://github.com/signalnine/q27/blob/master/src/loader.cpp): `src/loader.cpp`

### Reuse and state

- [Prefix cache](https://github.com/signalnine/q27/blob/master/src/prefix_cache.h): `src/prefix_cache.h`
- [Disk snapshots](https://github.com/signalnine/q27/blob/master/src/metal/disk_snapshot_store.h): `src/metal/disk_snapshot_store.h`

### Compute backends

- [Metal backend](https://github.com/signalnine/q27/blob/master/src/metal/metal_backend.h): `src/metal/metal_backend.h`
- [Backend interface](https://github.com/signalnine/q27/blob/master/src/backend.h): `src/backend.h`

### Serving options

- [RunPod handler](https://github.com/signalnine/q27/blob/master/deploy/runpod-serverless/handler.py): `deploy/runpod-serverless/handler.py`

### Other components

- **API client**

## Connections

- API client → Metal API server: sends request
- Metal API server → Tokenizer: tokenizes prompts
- Metal API server → Prompt and tool parsing: builds prompts
- Metal API server → Metal engine: dispatches inference
- Metal API server → Tool constraints: constrains decoding (evidence: `src/metal/metal_server.cpp`)
- Tool constraints → Tool grammar: uses grammar (evidence: `src/toolconstrain.h`)
- Metal API server → Stream formatting: formats responses (evidence: `src/metal/metal_server.cpp`)
- Metal API server → Disk snapshots: uses snapshots (evidence: `src/metal/metal_server.cpp`)
- Metal engine → Metal backend: runs operations (evidence: `src/metal/metal_engine.cpp`)
- Metal backend → Backend interface: implements interface (evidence: `src/metal/metal_backend.h`)
- Metal engine → Sampling: samples tokens
- Metal engine → Suffix speculation: uses drafter (evidence: `src/metal/metal_engine.cpp`)
- Metal engine → Model loader: loads model
- Metal API server → Prefix cache: reuses prefixes
- Metal API server → CUDA batching: batches requests
- RunPod handler → Metal API server: invokes service
