# shepherd-agents/shepherd architecture

> Architecture diagram of the GitHub repository shepherd-agents/shepherd, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shepherd-agents/shepherd
- Repository: https://github.com/shepherd-agents/shepherd

## Overview

#### Purpose and entry points
Shepherd is a Python 3.11+ framework for executing agent tasks as durable, inspectable, reversible runs. Tasks declare capability grants in Python signatures; filesystem/Git changes are retained as proposals and explicitly selected, applied, released, or discarded. The public distribution facade and user CLI live in `shepherd/packages/meta/src/shepherd/__init__.py` and `shepherd/packages/meta/src/shepherd/cli/`; runnable examples begin at `examples/quickstart/agent_task.py`.

#### Public programming model and dialect
The core model separates task contracts, scopes, effects, providers, outputs, and runs. `shepherd_core` defines shared types, task discovery, provider contracts, and effect representation (`shepherd/packages/core/src/shepherd_core/run.py`, `shepherd/packages/core/src/shepherd_core/effects/registry.py`). The dialect layer turns decorated functions and workflow steps into executable task envelopes, derives permission plans, validates source, and drives runs (`shepherd/packages/dialect/src/shepherd_dialect/nucleus.py`, `shepherd/packages/dialect/src/shepherd_dialect/run_driver.py`, `shepherd/packages/dialect/src/shepherd_dialect/permission_plan.py`).

#### Runtime lifecycle and execution flow
A run proceeds through explicit lifecycle phases—configure, prepare, context setup, execute, extract, artifact, apply, and cleanup—rather than directly mutating a caller workspace. The runtime builds task/source state, resolves bindings and providers, dispatches effects, emits artifacts/trace events, then materializes retained outputs. Key ownership paths are `shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py`, `shepherd/packages/runtime/src/shepherd_runtime/execution.py`, and `shepherd/packages/runtime/src/shepherd_runtime/task/pipeline.py`.

#### Scopes, effects, persistence, and reversibility
Runtime scopes form the isolation and composition boundary: nested bindings, checkpoints, resumptions, handler stacks, and effect materialization are coordinated under the scope runtime. Durable streams and project persistence preserve run history for inspection/resume, while cache policy can reuse eligible work. See `shepherd/packages/runtime/src/shepherd_runtime/_scope/runtime.py`, `shepherd/packages/runtime/src/shepherd_runtime/effects/handle.py`, and `shepherd/packages/runtime/src/shepherd_runtime/_persistence_manager.py`.

#### Workspace authority and versioned outputs
Workspace-control implements named repository bindings, read-only/read-write authority, run handles, changesets, output publication, and retained-output settlement. It is the policy boundary between an agent’s proposed filesystem result and a user-visible workspace update. Git-facing primitives support canonical identities and repository recovery in `commons-vcs/src/commons_vcs/kernel.py` and `commons-vcs/src/commons_vcs/backends/git.py`; the higher-level authority model is in `shepherd/packages/dialect/src/shepherd_dialect/workspace_control/authority.py` and `shepherd/packages/dialect/src/shepherd_dialect/workspace_control/output_publication.py`.

#### Isolation, providers, and external services
Execution can occur locally or on container devices; container support includes Podman, FUSE overlays, VM extraction, and provider execution (`shepherd/packages/runtime/src/shepherd_runtime/device/container/device.py`, `shepherd/packages/runtime/src/shepherd_runtime/device/container/podman.py`). Native enforcement targets macOS Seatbelt and Linux Landlock through confinement/workspace-control paths. Provider adapters integrate Claude, OpenAI, LiteLLM, and OpenCode (`shepherd/packages/providers/src/shepherd_providers/claude/provider.py`, `shepherd/packages/providers/src/shepherd_providers/openai/provider.py`); MCP support appears in both context and provider boundaries.

#### Ecosystem and parallel substrate work
Reusable context packages supply workspace, session, database, KV-store, app-store, and MCP effect surfaces (`shepherd/packages/contexts/src/shepherd_contexts/workspace/`, `shepherd/packages/contexts/src/shepherd_contexts/mcp/`). Domain packages provide coding and authoring workflows. The repository also contains independent/reference substrate efforts: `vcs-core/packages/core/src/vcs_core/` is a large Git/SQLite-backed versioned-world runtime with CLI/session/materialization layers, while `shepherd2/src/shepherd2/` and `shepherd/packages/kernel-v3-reference/src/shepherd_kernel_v3_reference/` are newer/reference kernel and trace designs rather than the primary packaged runtime path.

Main entry points: [`shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py`](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py) (Lifecycle pipeline), [`shepherd/packages/runtime/src/shepherd_runtime/execution.py`](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/execution.py) (Execution engine), [`shepherd/packages/dialect/src/shepherd_dialect/nucleus.py`](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/nucleus.py) (Dialect nucleus), [`shepherd/packages/meta/src/shepherd/__init__.py`](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/meta/src/shepherd/__init__.py) (Package facade & CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_public["Public Surface"]
  node_facade["Package facade &amp; CLI<br/>Python API / CLI<br/>[__init__.py]"]
  node_quickstart["Quickstart task<br/>example entry point<br/>[agent_task.py]"]
end

subgraph group_model["Task Model &amp; Dialect"]
  node_core["Core contracts<br/>[run.py]"]
  node_effect_registry["Effect registry<br/>effect representation<br/>[registry.py]"]
  node_dialect["Dialect nucleus<br/>task envelope builder<br/>[nucleus.py]"]
  node_permission_plan["Permission planning<br/>capability planner<br/>[permission_plan.py]"]
  node_run_driver["Run driver<br/>dialect execution driver<br/>[run_driver.py]"]
end

subgraph group_runtime["Durable Runtime"]
  node_lifecycle["Lifecycle pipeline<br/>run phase orchestrator<br/>[_pipeline.py]"]
  node_task_pipeline["Task pipeline<br/>task/source preparation<br/>[pipeline.py]"]
  node_execution["Execution engine<br/>binding and provider dispatcher<br/>[execution.py]"]
  node_scope_runtime["Scope runtime<br/>isolation and composition<br/>[runtime.py]"]
  node_effect_handling["Effect handling<br/>effect materialization<br/>[handle.py]"]
  node_persistence[("Run persistence<br/>durable streams and cache")]
end

subgraph group_workspace["Workspace &amp; Versioning"]
  node_authority["Workspace authority<br/>repository access policy<br/>[authority.py]"]
  node_publication["Output publication<br/>retained-output settlement"]
  node_vcs["Git primitives<br/>repository identity and recovery<br/>[kernel.py]"]
  node_git_backend["Git backend<br/>Git implementation<br/>[git.py]"]
end

subgraph group_integrations["Execution Integrations"]
  node_container["Container devices<br/>isolated execution device<br/>[device.py]"]
  node_podman["Podman adapter<br/>container backend<br/>[podman.py]"]
  node_providers["Provider adapters<br/>LLM provider integrations<br/>[provider.py]"]
  node_contexts["Context packages<br/>effect surfaces"]
end

node_quickstart -->|"uses"| node_facade
node_facade -->|"exposes"| node_dialect
node_dialect -->|"builds on"| node_core
node_core -->|"defines effects"| node_effect_registry
node_dialect -->|"derives grants"| node_permission_plan
node_dialect -->|"drives"| node_run_driver
node_run_driver -->|"starts run"| node_lifecycle
node_lifecycle -->|"prepares task"| node_task_pipeline
node_lifecycle -->|"executes phase"| node_execution
node_execution -->|"runs within"| node_scope_runtime
node_scope_runtime -->|"coordinates"| node_effect_handling
node_effect_handling -->|"dispatches"| node_effect_registry
node_lifecycle -->|"records runs"| node_persistence
node_execution -->|"invokes"| node_providers
node_execution -->|"binds contexts"| node_contexts
node_execution -.->|"can isolate on"| node_container
node_container -->|"uses"| node_podman
node_lifecycle -->|"settles outputs through"| node_authority
node_authority -->|"authorizes"| node_publication
node_publication -->|"versions changesets"| node_vcs
node_vcs -->|"implemented by"| node_git_backend

click node_facade "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/meta/src/shepherd/__init__.py"
click node_quickstart "https://github.com/shepherd-agents/shepherd/blob/main/examples/quickstart/agent_task.py"
click node_core "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/core/src/shepherd_core/run.py"
click node_effect_registry "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/core/src/shepherd_core/effects/registry.py"
click node_dialect "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/nucleus.py"
click node_permission_plan "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/permission_plan.py"
click node_run_driver "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/run_driver.py"
click node_lifecycle "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py"
click node_task_pipeline "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/task/pipeline.py"
click node_execution "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/execution.py"
click node_scope_runtime "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_scope/runtime.py"
click node_effect_handling "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/effects/handle.py"
click node_persistence "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_persistence_manager.py"
click node_authority "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/workspace_control/authority.py"
click node_publication "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/workspace_control/output_publication.py"
click node_vcs "https://github.com/shepherd-agents/shepherd/blob/main/commons-vcs/src/commons_vcs/kernel.py"
click node_git_backend "https://github.com/shepherd-agents/shepherd/blob/main/commons-vcs/src/commons_vcs/backends/git.py"
click node_container "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/device/container/device.py"
click node_podman "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/device/container/podman.py"
click node_providers "https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/providers/src/shepherd_providers/claude/provider.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_facade,node_quickstart toneBlue
class node_core,node_effect_registry,node_dialect,node_permission_plan,node_run_driver toneAmber
class node_lifecycle,node_task_pipeline,node_execution,node_scope_runtime,node_effect_handling,node_persistence toneMint
class node_authority,node_publication,node_vcs,node_git_backend toneRose
class node_container,node_podman,node_providers,node_contexts toneIndigo
```

## Components

### Public Surface

- [Package facade & CLI](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/meta/src/shepherd/__init__.py): Python API / CLI · `shepherd/packages/meta/src/shepherd/__init__.py`
- [Quickstart task](https://github.com/shepherd-agents/shepherd/blob/main/examples/quickstart/agent_task.py): example entry point · `examples/quickstart/agent_task.py`

### Task Model & Dialect

- [Core contracts](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/core/src/shepherd_core/run.py): shared task and effect model · Defines task discovery, provider contracts, runs, and effect representations. · `shepherd/packages/core/src/shepherd_core/run.py`
- [Effect registry](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/core/src/shepherd_core/effects/registry.py): effect representation · `shepherd/packages/core/src/shepherd_core/effects/registry.py`
- [Dialect nucleus](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/nucleus.py): task envelope builder · Converts decorated functions and workflow steps into executable tasks. · `shepherd/packages/dialect/src/shepherd_dialect/nucleus.py`
- [Permission planning](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/permission_plan.py): capability planner · `shepherd/packages/dialect/src/shepherd_dialect/permission_plan.py`
- [Run driver](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/run_driver.py): dialect execution driver · `shepherd/packages/dialect/src/shepherd_dialect/run_driver.py`

### Durable Runtime

- [Lifecycle pipeline](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py): run phase orchestrator · Coordinates configure through cleanup without directly mutating the caller workspace. · `shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py`
- [Task pipeline](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/task/pipeline.py): task/source preparation · `shepherd/packages/runtime/src/shepherd_runtime/task/pipeline.py`
- [Execution engine](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/execution.py): binding and provider dispatcher · `shepherd/packages/runtime/src/shepherd_runtime/execution.py`
- [Scope runtime](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_scope/runtime.py): isolation and composition · Coordinates nested bindings, checkpoints, resumptions, and handler stacks. · `shepherd/packages/runtime/src/shepherd_runtime/_scope/runtime.py`
- [Effect handling](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/effects/handle.py): effect materialization · `shepherd/packages/runtime/src/shepherd_runtime/effects/handle.py`
- [Run persistence](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/_persistence_manager.py): durable streams and cache · Preserves history for inspection and resume while applying cache policy. · `shepherd/packages/runtime/src/shepherd_runtime/_persistence_manager.py`

### Workspace & Versioning

- [Workspace authority](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/workspace_control/authority.py): repository access policy · Controls named bindings, access modes, run handles, and changesets. · `shepherd/packages/dialect/src/shepherd_dialect/workspace_control/authority.py`
- [Output publication](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/dialect/src/shepherd_dialect/workspace_control/output_publication.py): retained-output settlement · `shepherd/packages/dialect/src/shepherd_dialect/workspace_control/output_publication.py`
- [Git primitives](https://github.com/shepherd-agents/shepherd/blob/main/commons-vcs/src/commons_vcs/kernel.py): repository identity and recovery · `commons-vcs/src/commons_vcs/kernel.py`
- [Git backend](https://github.com/shepherd-agents/shepherd/blob/main/commons-vcs/src/commons_vcs/backends/git.py): Git implementation · `commons-vcs/src/commons_vcs/backends/git.py`

### Execution Integrations

- [Container devices](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/device/container/device.py): isolated execution device · Supports Podman, FUSE overlays, VM extraction, and provider execution. · `shepherd/packages/runtime/src/shepherd_runtime/device/container/device.py`
- [Podman adapter](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/runtime/src/shepherd_runtime/device/container/podman.py): container backend · `shepherd/packages/runtime/src/shepherd_runtime/device/container/podman.py`
- [Provider adapters](https://github.com/shepherd-agents/shepherd/blob/main/shepherd/packages/providers/src/shepherd_providers/claude/provider.py): LLM provider integrations · Integrates Claude, OpenAI, LiteLLM, and OpenCode; MCP crosses provider boundaries. · `shepherd/packages/providers/src/shepherd_providers/claude/provider.py`
- **Context packages**: effect surfaces · Supplies workspace, session, database, KV-store, app-store, and MCP contexts.

## Connections

- Quickstart task → Package facade & CLI: uses
- Package facade & CLI → Dialect nucleus: exposes
- Dialect nucleus → Core contracts: builds on
- Core contracts → Effect registry: defines effects
- Dialect nucleus → Permission planning: derives grants
- Dialect nucleus → Run driver: drives
- Run driver → Lifecycle pipeline: starts run
- Lifecycle pipeline → Task pipeline: prepares task
- Lifecycle pipeline → Execution engine: executes phase
- Execution engine → Scope runtime: runs within
- Scope runtime → Effect handling: coordinates
- Effect handling → Effect registry: dispatches
- Lifecycle pipeline → Run persistence: records runs
- Execution engine → Provider adapters: invokes
- Execution engine → Context packages: binds contexts
- Execution engine → Container devices: can isolate on
- Container devices → Podman adapter: uses
- Lifecycle pipeline → Workspace authority: settles outputs through
- Workspace authority → Output publication: authorizes
- Output publication → Git primitives: versions changesets
- Git primitives → Git backend: implemented by