# infervisor/plow architecture

> Architecture diagram of the GitHub repository infervisor/plow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/infervisor/plow
- Repository: https://github.com/infervisor/plow

## Overview

Plow is an ahead-of-time inference engine: a model build is compiled into a packet and weights, loaded by the runtime, then served through an OpenAI-compatible API. The graph preserves compiler/packet stages, asset loading, request serving, scheduling, execution backends, and KV prefix sharing. The README documents the end-to-end workflow and speech capabilities; source-index references and excerpts support selected internal links. Other unsampled runtime subsystems are represented at directory level, with uncertain wiring omitted.

Main entry points: [`crates/plowrt/src/serve/engine.rs`](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/engine.rs) (Serving engine), [`crates/plowrt/src/exec/packet_runtime.rs`](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/packet_runtime.rs) (Packet runtime), [`crates/plowrt/src/serve/manager.rs`](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/manager.rs) (Model manager), [`crates/plowc/src/main.rs`](https://github.com/infervisor/plow/blob/main/crates/plowc/src/main.rs) (Model compiler)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_compile["Model compilation"]
  node_compiler["Model compiler<br/>[main.rs]"]
  node_modelgraphs["Model graphs"]
  node_packetgen["Packet generation<br/>[pipeline.rs]"]
  node_packetbuild["Packet builder<br/>[devbuild.rs]"]
end

subgraph group_assets["Assets and loading"]
  node_assetspec["Asset contracts<br/>[packet_pipeline.rs]"]
  node_packetimage["Packet image<br/>[packet_image.rs]"]
  node_devblob["Device blob reader<br/>[devblob.rs]"]
  node_assetserve["Serve assets<br/>[serve.rs]"]
end

subgraph group_serve["Serving and scheduling"]
  node_manager["Model manager<br/>[manager.rs]"]
  node_api["HTTP API<br/>[mux.rs]"]
  node_engine["Serving engine<br/>[engine.rs]"]
  node_scheduler["Request scheduler"]
end

subgraph group_execute["Execution and memory"]
  node_packetruntime["Packet runtime<br/>[packet_runtime.rs]"]
  node_gpu["GPU execution<br/>[gpu.rs]"]
  node_cpu["CPU execution<br/>[engine.rs]"]
  node_hsa["AMD device backend<br/>[hsa.rs]"]
  node_vmm[("KV prefix sharing<br/>[vmm.rs]")]
end

subgraph group_speech["Speech capabilities"]
  node_asr["Speech recognition"]
  node_tts["Speech generation"]
end

node_user(("Client / operator"))

node_user -->|"builds model"| node_compiler
node_compiler -->|"emits packet"| node_packetbuild
node_packetbuild -.->|"packages assets"| node_assetspec
node_user -->|"loads model"| node_manager
node_manager -->|"opens packet"| node_packetimage
node_manager -->|"reads device blob"| node_devblob
node_manager -->|"loads serve settings"| node_assetserve
node_user -->|"sends request"| node_api
node_api -.->|"routes requests"| node_engine
node_engine -->|"schedules work"| node_scheduler
node_engine -->|"dispatches GPU work"| node_gpu
node_engine -->|"uses AMD backend"| node_hsa
node_packetruntime -->|"consumes packet"| node_packetimage
node_packetruntime -->|"reads blob data"| node_devblob
node_packetruntime -->|"dispatches GPU ops"| node_gpu
node_packetruntime -->|"uses CPU engine"| node_cpu
node_engine -->|"uses KV sharing"| node_vmm
node_user -.->|"requests transcription"| node_asr
node_user -.->|"requests speech"| node_tts

click node_compiler "https://github.com/infervisor/plow/blob/main/crates/plowc/src/main.rs"
click node_modelgraphs "https://github.com/infervisor/plow/tree/main/crates/nn-graph/src/models"
click node_packetgen "https://github.com/infervisor/plow/blob/main/crates/devgen/src/pipeline.rs"
click node_packetbuild "https://github.com/infervisor/plow/blob/main/crates/packet/src/devbuild.rs"
click node_assetspec "https://github.com/infervisor/plow/blob/main/crates/plow-asset/src/packet_pipeline.rs"
click node_packetimage "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/asset/packet_image.rs"
click node_devblob "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/asset/devblob.rs"
click node_assetserve "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/asset/serve.rs"
click node_manager "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/manager.rs"
click node_api "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/mux.rs"
click node_engine "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/engine.rs"
click node_scheduler "https://github.com/infervisor/plow/tree/main/crates/plowrt/src/sched"
click node_packetruntime "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/packet_runtime.rs"
click node_gpu "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/gpu.rs"
click node_cpu "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/cpu/engine.rs"
click node_hsa "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/device/hsa.rs"
click node_vmm "https://github.com/infervisor/plow/blob/main/crates/plowrt/src/memory/vmm.rs"
click node_asr "https://github.com/infervisor/plow/tree/main/crates/plowrt/src/asr"
click node_tts "https://github.com/infervisor/plow/tree/main/crates/plowrt/src/tts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_compiler,node_modelgraphs,node_packetgen,node_packetbuild,node_user toneBlue
class node_assetspec,node_packetimage,node_devblob,node_assetserve toneAmber
class node_manager,node_api,node_engine,node_scheduler toneMint
class node_packetruntime,node_gpu,node_cpu,node_hsa,node_vmm toneRose
class node_asr,node_tts toneIndigo
```

## Components

### Model compilation

- [Model compiler](https://github.com/infervisor/plow/blob/main/crates/plowc/src/main.rs): `crates/plowc/src/main.rs`
- [Model graphs](https://github.com/infervisor/plow/tree/main/crates/nn-graph/src/models): `crates/nn-graph/src/models`
- [Packet generation](https://github.com/infervisor/plow/blob/main/crates/devgen/src/pipeline.rs): `crates/devgen/src/pipeline.rs`
- [Packet builder](https://github.com/infervisor/plow/blob/main/crates/packet/src/devbuild.rs): `crates/packet/src/devbuild.rs`

### Assets and loading

- [Asset contracts](https://github.com/infervisor/plow/blob/main/crates/plow-asset/src/packet_pipeline.rs): `crates/plow-asset/src/packet_pipeline.rs`
- [Packet image](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/asset/packet_image.rs): `crates/plowrt/src/asset/packet_image.rs`
- [Device blob reader](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/asset/devblob.rs): `crates/plowrt/src/asset/devblob.rs`
- [Serve assets](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/asset/serve.rs): `crates/plowrt/src/asset/serve.rs`

### Serving and scheduling

- [Model manager](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/manager.rs): `crates/plowrt/src/serve/manager.rs`
- [HTTP API](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/mux.rs): `crates/plowrt/src/serve/mux.rs`
- [Serving engine](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/serve/engine.rs): `crates/plowrt/src/serve/engine.rs`
- [Request scheduler](https://github.com/infervisor/plow/tree/main/crates/plowrt/src/sched): `crates/plowrt/src/sched`

### Execution and memory

- [Packet runtime](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/packet_runtime.rs): `crates/plowrt/src/exec/packet_runtime.rs`
- [GPU execution](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/gpu.rs): `crates/plowrt/src/exec/gpu.rs`
- [CPU execution](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/exec/cpu/engine.rs): `crates/plowrt/src/exec/cpu/engine.rs`
- [AMD device backend](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/device/hsa.rs): `crates/plowrt/src/device/hsa.rs`
- [KV prefix sharing](https://github.com/infervisor/plow/blob/main/crates/plowrt/src/memory/vmm.rs): `crates/plowrt/src/memory/vmm.rs`

### Speech capabilities

- [Speech recognition](https://github.com/infervisor/plow/tree/main/crates/plowrt/src/asr): `crates/plowrt/src/asr`
- [Speech generation](https://github.com/infervisor/plow/tree/main/crates/plowrt/src/tts): `crates/plowrt/src/tts`

### Other components

- **Client / operator**

## Connections

- Client / operator → Model compiler: builds model (evidence: `README.md`)
- Model compiler → Packet builder: emits packet (evidence: `README.md`)
- Packet builder → Asset contracts: packages assets
- Client / operator → Model manager: loads model (evidence: `README.md`)
- Model manager → Packet image: opens packet (evidence: `crates/plowrt/src/serve/manager.rs`)
- Model manager → Device blob reader: reads device blob (evidence: `crates/plowrt/src/serve/manager.rs`)
- Model manager → Serve assets: loads serve settings (evidence: `crates/plowrt/src/serve/manager.rs`)
- Client / operator → HTTP API: sends request (evidence: `README.md`)
- HTTP API → Serving engine: routes requests
- Serving engine → Request scheduler: schedules work (evidence: `crates/plowrt/src/serve/engine.rs`)
- Serving engine → GPU execution: dispatches GPU work (evidence: `crates/plowrt/src/serve/engine.rs`)
- Serving engine → AMD device backend: uses AMD backend (evidence: `crates/plowrt/src/serve/engine.rs`)
- Packet runtime → Packet image: consumes packet (evidence: `crates/plowrt/src/exec/packet_runtime.rs`)
- Packet runtime → Device blob reader: reads blob data (evidence: `crates/plowrt/src/exec/packet_runtime.rs`)
- Packet runtime → GPU execution: dispatches GPU ops (evidence: `crates/plowrt/src/exec/packet_runtime.rs`)
- Packet runtime → CPU execution: uses CPU engine (evidence: `crates/plowrt/src/exec/packet_runtime.rs`)
- Serving engine → KV prefix sharing: uses KV sharing (evidence: `crates/plowrt/src/serve/engine.rs`)
- Client / operator → Speech recognition: requests transcription (evidence: `README.md`)
- Client / operator → Speech generation: requests speech (evidence: `README.md`)