shepherd-agents/shepherd

Purpose and entry points

Shepherd is a Python 3.11+ framework for executing agent tasks as durable, inspectable, reversible runs. Tasks declare capability grants in Python signatures; filesystem/Git changes are retained as proposals and explicitly selected, applied, released, or discarded. The public distribution facade and user CLI live in shepherd/packages/meta/src/shepherd/__init__.py and shepherd/packages/meta/src/shepherd/cli/; runnable examples begin at examples/quickstart/agent_task.py.

Public programming model and dialect

The core model separates task contracts, scopes, effects, providers, outputs, and runs. shepherd_core defines shared types, task discovery, provider contracts, and effect representation (shepherd/packages/core/src/shepherd_core/run.py, shepherd/packages/core/src/shepherd_core/effects/registry.py). The dialect layer turns decorated functions and workflow steps into executable task envelopes, derives permission plans, validates source, and drives runs (shepherd/packages/dialect/src/shepherd_dialect/nucleus.py, shepherd/packages/dialect/src/shepherd_dialect/run_driver.py, shepherd/packages/dialect/src/shepherd_dialect/permission_plan.py).

Runtime lifecycle and execution flow

A run proceeds through explicit lifecycle phases—configure, prepare, context setup, execute, extract, artifact, apply, and cleanup—rather than directly mutating a caller workspace. The runtime builds task/source state, resolves bindings and providers, dispatches effects, emits artifacts/trace events, then materializes retained outputs. Key ownership paths are shepherd/packages/runtime/src/shepherd_runtime/_lifecycle/_pipeline.py, shepherd/packages/runtime/src/shepherd_runtime/execution.py, and shepherd/packages/runtime/src/shepherd_runtime/task/pipeline.py.

Scopes, effects, persistence, and reversibility

Runtime scopes form the isolation and composition boundary: nested bindings, checkpoints, resumptions, handler stacks, and effect materialization are coordinated under the scope runtime. Durable streams and project persistence preserve run history for inspection/resume, while cache policy can reuse eligible work. See shepherd/packages/runtime/src/shepherd_runtime/_scope/runtime.py, shepherd/packages/runtime/src/shepherd_runtime/effects/handle.py, and shepherd/packages/runtime/src/shepherd_runtime/_persistence_manager.py.

Workspace authority and versioned outputs

Workspace-control implements named repository bindings, read-only/read-write authority, run handles, changesets, output publication, and retained-output settlement. It is the policy boundary between an agent’s proposed filesystem result and a user-visible workspace update. Git-facing primitives support canonical identities and repository recovery in commons-vcs/src/commons_vcs/kernel.py and commons-vcs/src/commons_vcs/backends/git.py; the higher-level authority model is in shepherd/packages/dialect/src/shepherd_dialect/workspace_control/authority.py and shepherd/packages/dialect/src/shepherd_dialect/workspace_control/output_publication.py.

Isolation, providers, and external services

Execution can occur locally or on container devices; container support includes Podman, FUSE overlays, VM extraction, and provider execution (shepherd/packages/runtime/src/shepherd_runtime/device/container/device.py, shepherd/packages/runtime/src/shepherd_runtime/device/container/podman.py). Native enforcement targets macOS Seatbelt and Linux Landlock through confinement/workspace-control paths. Provider adapters integrate Claude, OpenAI, LiteLLM, and OpenCode (shepherd/packages/providers/src/shepherd_providers/claude/provider.py, shepherd/packages/providers/src/shepherd_providers/openai/provider.py); MCP support appears in both context and provider boundaries.

Ecosystem and parallel substrate work

Reusable context packages supply workspace, session, database, KV-store, app-store, and MCP effect surfaces (shepherd/packages/contexts/src/shepherd_contexts/workspace/, shepherd/packages/contexts/src/shepherd_contexts/mcp/). Domain packages provide coding and authoring workflows. The repository also contains independent/reference substrate efforts: vcs-core/packages/core/src/vcs_core/ is a large Git/SQLite-backed versioned-world runtime with CLI/session/materialization layers, while shepherd2/src/shepherd2/ and shepherd/packages/kernel-v3-reference/src/shepherd_kernel_v3_reference/ are newer/reference kernel and trace designs rather than the primary packaged runtime path.