# pie-project/pie architecture

> Architecture diagram of the GitHub repository pie-project/pie, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pie-project/pie
- Repository: https://github.com/pie-project/pie

## Overview

Pie is a programmable inference engine: a client submits an inferlet, which drives model execution and returns results while managing persistent working-set state. The runtime schedules pipeline fires onto device engines; CUDA is one documented backend among several. The README also describes OpenAI-compatible serving, media handoff, and model programs written in Poem. Edges follow README statements or shown module references; unsampled backend and service wiring is intentionally not inferred.

Main entry points: [`crates/engine/src/engine.rs`](https://github.com/pie-project/pie/blob/main/crates/engine/src/engine.rs) (Engine abstraction), [`crates/client/src/client.rs`](https://github.com/pie-project/pie/blob/main/crates/client/src/client.rs) (Pie client), [`crates/runtime/src/inferlet/host/pipeline.rs`](https://github.com/pie-project/pie/blob/main/crates/runtime/src/inferlet/host/pipeline.rs) (Inferlet program), [`crates/runtime/src/pipeline/fire.rs`](https://github.com/pie-project/pie/blob/main/crates/runtime/src/pipeline/fire.rs) (Pipeline fire)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_client["Pie client<br/>[client.rs]"]
  node_compat["API compatibility"]
  node_inferlet["Inferlet program<br/>[pipeline.rs]"]
  node_poem["Poem models<br/>[program.rs]"]
end

subgraph group_execution["Inferlet runtime"]
  node_hostpipeline["Pipeline host<br/>[pipeline.rs]"]
  node_pipeline["Pipeline lifecycle<br/>[pipeline.rs]"]
  node_fire["Pipeline fire<br/>[fire.rs]"]
  node_scheduler["Worker scheduling<br/>[worker.rs]"]
end

subgraph group_serving["Serving and state"]
  node_worker["Worker service<br/>[server.rs]"]
  node_controller["Cluster controller<br/>[program.rs]"]
  node_working[("Working-set state<br/>[working_set.rs]")]
end

subgraph group_engines["Device execution"]
  node_engine["Engine abstraction<br/>[engine.rs]"]
  node_cuda["CUDA program plane<br/>[program.rs]"]
  node_metal["Metal program plane<br/>[program.rs]"]
  node_xla["XLA program plane<br/>[program.rs]"]
  node_gpu["GPU operators<br/>[program.rs]"]
  node_checkpoint["Model checkpoint<br/>[lib.rs]"]
end

node_user(("Client or caller"))

node_user -->|"submits runs"| node_client
node_user -->|"sends API requests"| node_compat
node_client -->|"submits programs"| node_worker
node_compat -->|"serves built-ins"| node_inferlet
node_inferlet -->|"opens pipeline"| node_hostpipeline
node_hostpipeline -->|"creates and closes"| node_pipeline
node_pipeline -->|"delegates fires"| node_fire
node_fire -->|"submits execution"| node_engine
node_scheduler -.->|"schedules work"| node_fire
node_worker -.->|"joins cluster"| node_controller
node_client -->|"owns handles"| node_working
node_inferlet -->|"runs model program"| node_poem
node_engine -.->|"backend implementation"| node_cuda
node_engine -.->|"backend implementation"| node_metal
node_engine -.->|"backend implementation"| node_xla
node_engine -.->|"shares GPU operators"| node_gpu
node_checkpoint -.->|"provides model weights"| node_engine

click node_client "https://github.com/pie-project/pie/blob/main/crates/client/src/client.rs"
click node_compat "https://github.com/pie-project/pie/tree/main/crates/builtins/inferlets"
click node_inferlet "https://github.com/pie-project/pie/blob/main/crates/runtime/src/inferlet/host/pipeline.rs"
click node_poem "https://github.com/pie-project/pie/blob/main/crates/engine/src/program.rs"
click node_hostpipeline "https://github.com/pie-project/pie/blob/main/crates/runtime/src/inferlet/host/pipeline.rs"
click node_pipeline "https://github.com/pie-project/pie/blob/main/crates/runtime/src/pipeline.rs"
click node_fire "https://github.com/pie-project/pie/blob/main/crates/runtime/src/pipeline/fire.rs"
click node_scheduler "https://github.com/pie-project/pie/blob/main/crates/runtime/src/scheduler/worker.rs"
click node_worker "https://github.com/pie-project/pie/blob/main/crates/worker/src/server.rs"
click node_controller "https://github.com/pie-project/pie/blob/main/crates/controller/src/cluster/program.rs"
click node_working "https://github.com/pie-project/pie/blob/main/crates/runtime/src/store/kv/working_set.rs"
click node_engine "https://github.com/pie-project/pie/blob/main/crates/engine/src/engine.rs"
click node_cuda "https://github.com/pie-project/pie/blob/main/crates/engine-cuda/src/program.rs"
click node_metal "https://github.com/pie-project/pie/blob/main/crates/engine-metal/src/program.rs"
click node_xla "https://github.com/pie-project/pie/blob/main/crates/engine-xla/src/program.rs"
click node_gpu "https://github.com/pie-project/pie/blob/main/crates/engine-gpu/src/program.rs"
click node_checkpoint "https://github.com/pie-project/pie/blob/main/crates/checkpoint/src/lib.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_client,node_compat,node_inferlet,node_poem,node_user toneBlue
class node_hostpipeline,node_pipeline,node_fire,node_scheduler toneAmber
class node_worker,node_controller,node_working toneMint
class node_engine,node_cuda,node_metal,node_xla,node_gpu,node_checkpoint toneRose
```

## Components

### User interfaces

- [Pie client](https://github.com/pie-project/pie/blob/main/crates/client/src/client.rs): `crates/client/src/client.rs`
- [API compatibility](https://github.com/pie-project/pie/tree/main/crates/builtins/inferlets): `crates/builtins/inferlets`
- [Inferlet program](https://github.com/pie-project/pie/blob/main/crates/runtime/src/inferlet/host/pipeline.rs): `crates/runtime/src/inferlet/host/pipeline.rs`
- [Poem models](https://github.com/pie-project/pie/blob/main/crates/engine/src/program.rs): `crates/engine/src/program.rs`

### Inferlet runtime

- [Pipeline host](https://github.com/pie-project/pie/blob/main/crates/runtime/src/inferlet/host/pipeline.rs): `crates/runtime/src/inferlet/host/pipeline.rs`
- [Pipeline lifecycle](https://github.com/pie-project/pie/blob/main/crates/runtime/src/pipeline.rs): `crates/runtime/src/pipeline.rs`
- [Pipeline fire](https://github.com/pie-project/pie/blob/main/crates/runtime/src/pipeline/fire.rs): `crates/runtime/src/pipeline/fire.rs`
- [Worker scheduling](https://github.com/pie-project/pie/blob/main/crates/runtime/src/scheduler/worker.rs): `crates/runtime/src/scheduler/worker.rs`

### Serving and state

- [Worker service](https://github.com/pie-project/pie/blob/main/crates/worker/src/server.rs): `crates/worker/src/server.rs`
- [Cluster controller](https://github.com/pie-project/pie/blob/main/crates/controller/src/cluster/program.rs): `crates/controller/src/cluster/program.rs`
- [Working-set state](https://github.com/pie-project/pie/blob/main/crates/runtime/src/store/kv/working_set.rs): `crates/runtime/src/store/kv/working_set.rs`

### Device execution

- [Engine abstraction](https://github.com/pie-project/pie/blob/main/crates/engine/src/engine.rs): `crates/engine/src/engine.rs`
- [CUDA program plane](https://github.com/pie-project/pie/blob/main/crates/engine-cuda/src/program.rs): `crates/engine-cuda/src/program.rs`
- [Metal program plane](https://github.com/pie-project/pie/blob/main/crates/engine-metal/src/program.rs): `crates/engine-metal/src/program.rs`
- [XLA program plane](https://github.com/pie-project/pie/blob/main/crates/engine-xla/src/program.rs): `crates/engine-xla/src/program.rs`
- [GPU operators](https://github.com/pie-project/pie/blob/main/crates/engine-gpu/src/program.rs): `crates/engine-gpu/src/program.rs`
- [Model checkpoint](https://github.com/pie-project/pie/blob/main/crates/checkpoint/src/lib.rs): `crates/checkpoint/src/lib.rs`

### Other components

- **Client or caller**

## Connections

- Client or caller → Pie client: submits runs (evidence: `README.md`)
- Client or caller → API compatibility: sends API requests (evidence: `README.md`)
- Pie client → Worker service: submits programs (evidence: `README.md`)
- API compatibility → Inferlet program: serves built-ins (evidence: `README.md`)
- Inferlet program → Pipeline host: opens pipeline (evidence: `crates/runtime/src/inferlet/host/pipeline.rs`)
- Pipeline host → Pipeline lifecycle: creates and closes (evidence: `crates/runtime/src/inferlet/host/pipeline.rs`)
- Pipeline lifecycle → Pipeline fire: delegates fires (evidence: `crates/runtime/src/pipeline.rs`)
- Pipeline fire → Engine abstraction: submits execution (evidence: `crates/runtime/src/pipeline/fire.rs`)
- Worker scheduling → Pipeline fire: schedules work (evidence: `crates/runtime/src/pipeline/fire.rs`)
- Worker service → Cluster controller: joins cluster
- Pie client → Working-set state: owns handles (evidence: `README.md`)
- Inferlet program → Poem models: runs model program (evidence: `README.md`)
- Engine abstraction → CUDA program plane: backend implementation
- Engine abstraction → Metal program plane: backend implementation
- Engine abstraction → XLA program plane: backend implementation
- Engine abstraction → GPU operators: shares GPU operators
- Model checkpoint → Engine abstraction: provides model weights
