How pie-project/pie works
Pie is a programmable inference engine: a client submits an inferlet, which drives model execution and returns results while managing persistent working-set state. The runtime schedules pipeline fires onto device engines; CUDA is one documented backend among several. The README also describes OpenAI-compatible serving, media handoff, and model programs written in Poem. Edges follow README statements or shown module references; unsampled backend and service wiring is intentionally not inferred.
Main entry points: engine.rs Engine abstraction · client.rs Pie client · pipeline.rs Inferlet program · fire.rs Pipeline fire
Components 18
User interfaces
- Pie client
crates/client/src/client.rs - API compatibility
crates/builtins/inferlets - Inferlet program
crates/runtime/src/inferlet/host/pipeline.rs - Poem models
crates/engine/src/program.rs
Inferlet runtime
- Pipeline host
crates/runtime/src/inferlet/host/pipeline.rs - Pipeline lifecycle
crates/runtime/src/pipeline.rs - Pipeline fire
crates/runtime/src/pipeline/fire.rs - Worker scheduling
crates/runtime/src/scheduler/worker.rs
Serving and state
- Worker service
crates/worker/src/server.rs - Cluster controller
crates/controller/src/cluster/program.rs - Working-set state
crates/runtime/src/store/kv/working_set.rs
Device execution
- Engine abstraction
crates/engine/src/engine.rs - CUDA program plane
crates/engine-cuda/src/program.rs - Metal program plane
crates/engine-metal/src/program.rs - XLA program plane
crates/engine-xla/src/program.rs - GPU operators
crates/engine-gpu/src/program.rs - Model checkpoint
crates/checkpoint/src/lib.rs
Other components
- Client or caller
Connections 17
- Client or caller → Pie client: submits runs
README.md - Client or caller → API compatibility: sends API requests
README.md - Pie client → Worker service: submits programs
README.md - API compatibility → Inferlet program: serves built-ins
README.md - Inferlet program → Pipeline host: opens pipeline
crates/runtime/src/inferlet/host/pipeline.rs - Pipeline host → Pipeline lifecycle: creates and closes
crates/runtime/src/inferlet/host/pipeline.rs - Pipeline lifecycle → Pipeline fire: delegates fires
crates/runtime/src/pipeline.rs - Pipeline fire → Engine abstraction: submits execution
crates/runtime/src/pipeline/fire.rs - Worker scheduling → Pipeline fire: schedules work
crates/runtime/src/pipeline/fire.rs - Worker service → Cluster controller: joins cluster
- Pie client → Working-set state: owns handles
README.md - Inferlet program → Poem models: runs model program
README.md - Engine abstraction → CUDA program plane: backend implementation
- Engine abstraction → Metal program plane: backend implementation
- Engine abstraction → XLA program plane: backend implementation
- Engine abstraction → GPU operators: shares GPU operators
- Model checkpoint → Engine abstraction: provides model weights