# redwoodjs/agent-ci architecture

> Architecture diagram of the GitHub repository redwoodjs/agent-ci, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/redwoodjs/agent-ci
- Repository: https://github.com/redwoodjs/agent-ci

## Overview

1. **What this repo is**
   - Agent CI is a local execution platform for GitHub Actions workflows.
   - The architecture centers on replacing GitHub’s server-side workflow API with a local emulator, while still running the official GitHub Actions runner binary inside Docker or macOS VMs.

2. **Top-level shape: a monorepo with three product surfaces**
   - **CLI/runtime** for executing workflows locally.
   - **Website/docs** for marketing and user guidance.
   - **Distribution/packaging** for npm packages, native binaries, and release artifacts.

3. **Primary execution path: `packages/cli`**
   - This is the main TypeScript implementation used by `npx @redwoodjs/agent-ci`.
   - It handles command parsing, workflow discovery, job scheduling, runner lifecycle, local Docker orchestration, output rendering, run-state management, and retry/abort flows.
   - It is the user-facing orchestration layer that turns a workflow file into one or more runnable jobs.

4. **Workflow understanding and planning**
   - `packages/cli/src/workflow` parses GitHub Actions YAML, resolves reusable workflows, computes matrix expansion, classifies jobs by resource type, and creates execution plans.
   - `packages/cli/src/runner` then turns those plans into concrete runtime actions, including job result collection, workspace setup, dirty-sha handling, step wrapping, and job synchronization.

5. **Local container execution boundary: Docker**
   - Most jobs run in Docker containers derived from the official `ghcr.io/actions/actions-runner` image.
   - The CLI connects to Docker through a configurable daemon host, mounts the repo working tree, and binds host caches directly into containers for near-zero-cost reuse.
   - Docker is also the boundary for service containers, cache behavior, shutdown, and remote-daemon support.

6. **GitHub Actions emulation boundary: DTU / mock server**
   - The repo emulates the GitHub-side API surface needed by the official runner: workflow/job routing, artifacts, cache, and GitHub-facing endpoints.
   - That emulation is implemented in `packages/dtu-github-actions`, which contains server routes for GitHub, DTU-specific control, cache, artifacts, and action tarball delivery.
   - The emulator state is persisted in an internal store and exercised heavily through tests and fixtures.

7. **Rust rewrite path: `crates/agent-ci-core`, `crates/agent-ci-runtime`, `crates/agent-ci`**
   - The Rust side mirrors the TypeScript architecture for parity testing and eventual native execution.
   - `agent-ci-core` contains workflow expressions, matrices, reusable workflow logic, planning, and shared state/event modeling.
   - `agent-ci-runtime` contains execution backends: Docker, DTU, macOS VM orchestration, runner timeline handling, workspace management, and image handling.
   - `agent-ci` is the Rust CLI entry point and command layer.

8. **macOS job execution**
   - Jobs targeting `runs-on: macos-*` are routed to a separate macOS VM path rather than Docker.
   - The runtime uses Tart plus SSH on Apple Silicon hosts, with per-job VM provisioning, runner binary transfer, and teardown after completion.
   - This is a distinct execution lane with its own concurrency limit and image mapping.

9. **Caching and prewarm behavior**
   - A major architectural feature is local bind-mounted caching instead of networked GitHub cache round-trips.
   - The system can optionally “prewarm through” a selected step to avoid parallel cold installs colliding on shared `node_modules`.
   - This is a workflow-level optimization that affects job scheduling and initial execution order.

10. **Pause/retry/abort model**
   - Failed runs do not immediately destroy the container in pause mode.
   - The paused container preserves filesystem state, installed tools, and intermediate artifacts so only the failed step needs retrying.
   - That means the lifecycle includes a long-lived paused state, not just “start run / end run”.

11. **Inputs and secrets boundary**
   - Repo-local configuration comes from `.env.agent-ci`, shell environment variables, and explicit CLI flags.
   - Secrets are resolved differently from workflow variables: secrets can come from environment or token flags, while workflow vars come only from explicit `--var`/`--var-file` inputs.
   - This separation is important for the architecture diagram because it affects how configuration enters the runtime.

12. **Packaging and distribution**
   - `packages/agent-ci-*` are platform-specific npm wrappers for native binaries.
   - `install.sh`, `packaging/homebrew`, and the release workflows support cross-platform distribution.
   - The README indicates the published npm path currently favors TypeScript, while the Rust binary is available from source and is being prepared for native packaging.

13. **Support systems: website, docs, experiments**
   - `apps/website` is a separate web app for the public site and docs/marketing content.
   - `experiments/` contains evaluation harnesses, benchmarks, and proof-of-concept flows used to validate behaviors like retries, cache performance, and agent skill quality.
   - These are not part of the runtime path, but they support product validation and release messaging.

14. **Testing and compatibility infrastructure**
   - The repo relies on many workflow fixture files, smoke tests, and GitHub Actions CI workflows to validate compatibility across runner behaviors.
   - There are dedicated fixtures for matrix jobs, reusable workflows, service containers, symlink handling, remote workflow fetches, and failure modes.
   - For an architecture diagram, these tests represent the validation layer around the core orchestration engine rather than another runtime subsystem.

Main entry points: [`packages/cli/src/runner`](https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/runner) (Runner runtime), [`crates/agent-ci-runtime/src`](https://github.com/redwoodjs/agent-ci/tree/main/crates/agent-ci-runtime/src) (Runtime backends), [`packages/cli/src/cli.ts`](https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/cli.ts) (CLI entry), [`packages/dtu-github-actions/src/server`](https://github.com/redwoodjs/agent-ci/tree/main/packages/dtu-github-actions/src/server) (DTU server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_ts["TypeScript runtime"]
  node_n_cli_entry["CLI entry<br/>ts command layer<br/>[cli.ts]"]
  node_n_cli_config["Config<br/>[config.ts]"]
  node_n_cli_workflow["Workflow plan<br/>ts planning"]
  node_n_cli_scheduler["Job scheduler<br/>ts scheduling<br/>[job-scheduler.ts]"]
  node_n_cli_runner["Runner runtime<br/>ts execution"]
  node_n_cli_docker[("Docker bridge<br/>ts container runtime")]
  node_n_cli_macos[("macOS lane<br/>ts vm runtime")]
  node_n_cli_output["Run UI<br/>ts reporting"]
  node_n_dtu[("DTU server<br/>ts emulator")]
  node_n_dtu_routes["API routes<br/>ts http routes"]
  node_n_dtu_store[("State store<br/>ts persistence<br/>[store.ts]")]
end

subgraph group_g_rust["Rust rewrite"]
  node_n_rust_cli["Rust CLI<br/>rust command layer<br/>[main.rs]"]
  node_n_rust_core["Core planner<br/>rust planning"]
  node_n_rust_runtime["Runtime backends<br/>rust execution"]
end

subgraph group_g_product["Site and distribution"]
  node_n_website["Website<br/>web app"]
  node_n_packaging["Packaging"]
end

subgraph group_g_validation["Validation layer"]
  node_n_experiments["Experiments<br/>evaluation harness"]
  node_n_ci_workflows["CI workflows<br/>gha validation<br/>[workflows]"]
end

node_n_cli_entry -->|"reads"| node_n_cli_config
node_n_cli_entry -->|"plans"| node_n_cli_workflow
node_n_cli_workflow -->|"feeds"| node_n_cli_scheduler
node_n_cli_scheduler -->|"dispatches"| node_n_cli_runner
node_n_cli_runner -->|"uses"| node_n_cli_docker
node_n_cli_runner -->|"routes"| node_n_cli_macos
node_n_cli_runner -->|"reports"| node_n_cli_output
node_n_cli_docker -->|"emulates"| node_n_dtu
node_n_dtu -->|"serves"| node_n_dtu_routes
node_n_dtu -->|"persists"| node_n_dtu_store
node_n_rust_cli -->|"uses"| node_n_rust_core
node_n_rust_cli -->|"executes"| node_n_rust_runtime
node_n_rust_runtime -->|"consumes"| node_n_rust_core
node_n_rust_runtime -.->|"parallels"| node_n_cli_docker
node_n_rust_runtime -.->|"parallels"| node_n_cli_macos
node_n_website -->|"documents"| node_n_cli_entry
node_n_packaging -->|"ships"| node_n_rust_cli
node_n_packaging -.->|"also ships"| node_n_cli_entry
node_n_experiments -->|"validates"| node_n_cli_runner
node_n_ci_workflows -->|"smokes"| node_n_cli_runner
node_n_ci_workflows -->|"exercises"| node_n_dtu

click node_n_cli_entry "https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/cli.ts"
click node_n_cli_config "https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/config.ts"
click node_n_cli_workflow "https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/workflow"
click node_n_cli_scheduler "https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/workflow/job-scheduler.ts"
click node_n_cli_runner "https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/runner"
click node_n_cli_docker "https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/docker"
click node_n_cli_macos "https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/runner/macos-vm"
click node_n_cli_output "https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/output"
click node_n_dtu "https://github.com/redwoodjs/agent-ci/tree/main/packages/dtu-github-actions/src/server"
click node_n_dtu_routes "https://github.com/redwoodjs/agent-ci/tree/main/packages/dtu-github-actions/src/server/routes"
click node_n_dtu_store "https://github.com/redwoodjs/agent-ci/blob/main/packages/dtu-github-actions/src/server/store.ts"
click node_n_rust_cli "https://github.com/redwoodjs/agent-ci/blob/main/crates/agent-ci/src/main.rs"
click node_n_rust_core "https://github.com/redwoodjs/agent-ci/tree/main/crates/agent-ci-core/src"
click node_n_rust_runtime "https://github.com/redwoodjs/agent-ci/tree/main/crates/agent-ci-runtime/src"
click node_n_website "https://github.com/redwoodjs/agent-ci/tree/main/apps/website/src"
click node_n_packaging "https://github.com/redwoodjs/agent-ci/tree/main/packages/agent-ci-linux-x64"
click node_n_experiments "https://github.com/redwoodjs/agent-ci/tree/main/experiments"
click node_n_ci_workflows "https://github.com/redwoodjs/agent-ci/blob/main/.github/workflows"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_cli_entry,node_n_cli_config,node_n_cli_workflow,node_n_cli_scheduler,node_n_cli_runner,node_n_cli_docker,node_n_cli_macos,node_n_cli_output,node_n_dtu,node_n_dtu_routes,node_n_dtu_store toneBlue
class node_n_rust_cli,node_n_rust_core,node_n_rust_runtime toneAmber
class node_n_website,node_n_packaging toneMint
class node_n_experiments,node_n_ci_workflows toneRose
```

## Components

### TypeScript runtime

Primary local workflow execution path in packages/cli and DTU emulation.

- [CLI entry](https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/cli.ts): ts command layer · User-facing command parsing, run orchestration, retries, and abort/clean flows. · `packages/cli/src/cli.ts`
- [Config](https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/config.ts): ts config · Loads repo-local env, shell environment, flags, vars, and secrets boundaries. · `packages/cli/src/config.ts`
- [Workflow plan](https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/workflow): ts planning · Parses workflow YAML, resolves reusable workflows, expands matrices, and classifies jobs. · `packages/cli/src/workflow`
- [Job scheduler](https://github.com/redwoodjs/agent-ci/blob/main/packages/cli/src/workflow/job-scheduler.ts): ts scheduling · Orders jobs and coordinates execution constraints, including prewarm-through behavior. · `packages/cli/src/workflow/job-scheduler.ts`
- [Runner runtime](https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/runner): ts execution · Turns plans into job execution, workspace setup, dirty-sha handling, and synchronization. · `packages/cli/src/runner`
- [Docker bridge](https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/docker): ts container runtime · Connects to Docker daemon, configures containers, service containers, images, and shutdown. · `packages/cli/src/docker`
- [macOS lane](https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/runner/macos-vm): ts vm runtime · Routes macOS jobs to Tart/SSH-backed VM execution with image mapping and teardown. · `packages/cli/src/runner/macos-vm`
- [Run UI](https://github.com/redwoodjs/agent-ci/tree/main/packages/cli/src/output): ts reporting · Renders run state, logs, tree views, concurrency, and final job/result output. · `packages/cli/src/output`
- [DTU server](https://github.com/redwoodjs/agent-ci/tree/main/packages/dtu-github-actions/src/server): ts emulator · Emulates GitHub-side workflow, cache, artifact, and control endpoints for the runner. · `packages/dtu-github-actions/src/server`
- [API routes](https://github.com/redwoodjs/agent-ci/tree/main/packages/dtu-github-actions/src/server/routes): ts http routes · GitHub, DTU control, cache, artifacts, and action tarball route handlers. · `packages/dtu-github-actions/src/server/routes`
- [State store](https://github.com/redwoodjs/agent-ci/blob/main/packages/dtu-github-actions/src/server/store.ts): ts persistence · Internal state used by the mock GitHub Actions server and its tests/fixtures. · `packages/dtu-github-actions/src/server/store.ts`

### Rust rewrite

Native parity path split across core planning, runtime backends, and CLI entrypoint.

- [Rust CLI](https://github.com/redwoodjs/agent-ci/blob/main/crates/agent-ci/src/main.rs): rust command layer · Native entrypoint and command layer for local run orchestration. · `crates/agent-ci/src/main.rs`
- [Core planner](https://github.com/redwoodjs/agent-ci/tree/main/crates/agent-ci-core/src): rust planning · Workflow expressions, matrices, reusable workflows, state, and plan modeling. · `crates/agent-ci-core/src`
- [Runtime backends](https://github.com/redwoodjs/agent-ci/tree/main/crates/agent-ci-runtime/src): rust execution · Execution backends for Docker, DTU, macOS VM orchestration, workspace, and runner timeline. · `crates/agent-ci-runtime/src`

### Site and distribution

Public website, docs, packaging, and release artifacts.

- [Website](https://github.com/redwoodjs/agent-ci/tree/main/apps/website/src): web app · Public site, docs, compatibility pages, and marketing content. · `apps/website/src`
- [Packaging](https://github.com/redwoodjs/agent-ci/tree/main/packages/agent-ci-linux-x64): release packaging · Native npm wrappers, install script, and Homebrew formula template for distribution. · `packages/agent-ci-linux-x64`

### Validation layer

Workflows, fixtures, and experiments used to verify compatibility and behavior.

- [Experiments](https://github.com/redwoodjs/agent-ci/tree/main/experiments): evaluation harness · Benchmarks and proof-of-concept flows for cache, retries, and skill evaluation. · `experiments`
- [CI workflows](https://github.com/redwoodjs/agent-ci/blob/main/.github/workflows): gha validation · Smoke and release workflows that exercise runner compatibility across scenarios. · `.github/workflows`

## Connections

- CLI entry → Config: reads · Command execution starts by resolving environment, vars, and secrets.
- CLI entry → Workflow plan: plans · Turns a workflow file into an execution plan.
- Workflow plan → Job scheduler: feeds · Expanded jobs and reusable workflows are scheduled for execution.
- Job scheduler → Runner runtime: dispatches · Creates runnable jobs and coordinates execution order.
- Runner runtime → Docker bridge: uses · Most jobs execute in Docker containers with mounts and cache bindings.
- Runner runtime → macOS lane: routes · macOS-targeted jobs follow the VM execution lane.
- Runner runtime → Run UI: reports · Run and step state are rendered back to the user.
- Docker bridge → DTU server: emulates · Docker jobs interact with the local GitHub Actions emulator.
- DTU server → API routes: serves · Server routes expose GitHub, cache, artifact, and DTU endpoints.
- DTU server → State store: persists · Emulator state is stored for runs and fixtures.
- Rust CLI → Core planner: uses · Native CLI delegates planning and workflow semantics to core.
- Rust CLI → Runtime backends: executes · CLI layer drives the native runtime backends.
- Runtime backends → Core planner: consumes · Runtime executes plans and state from the shared core model.
- Runtime backends → Docker bridge: parallels · Rust backend mirrors the TypeScript Docker execution path.
- Runtime backends → macOS lane: parallels · Rust backend mirrors the TypeScript macOS VM lane.
- Website → CLI entry: documents · Site content explains local workflow execution and usage.
- Packaging → Rust CLI: ships · Distribution packages native binaries and platform wrappers.
- Packaging → CLI entry: also ships · Current npm path still favors the TypeScript CLI surface.
- Experiments → Runner runtime: validates · Benchmarks and eval harnesses exercise retry, cache, and agent behavior.
- CI workflows → Runner runtime: smokes · GitHub Actions workflows validate compatibility and failure modes.
- CI workflows → DTU server: exercises · Smoke workflows depend on the mock server and artifact/cache behavior.