# woodpecker-ci/woodpecker architecture

> Architecture diagram of the GitHub repository woodpecker-ci/woodpecker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/woodpecker-ci/woodpecker
- Repository: https://github.com/woodpecker-ci/woodpecker

## Overview

##### Project Type & Purpose
- Woodpecker is a full-stack, extensible CI/CD engine written in Go, composed of a server (API, business logic, persistence, queueing, pub/sub), an agent for executing pipeline steps on various backends, a CLI for scripting/automation, and a web UI for human interaction.

##### High-Level File Structure Analysis
- Top-level “cmd/…” folders: entry points for `woodpecker-server`, `woodpecker-agent`, `woodpecker-cli`.
- `server/`: REST & gRPC API handlers, router, services layer, store (database + migrations), pub/sub, queue, cron scheduler, model definitions.
- `agent/`: gRPC client, pipeline runner, tracer, state management.
- `cli/`: commands for admin/org/repo/pipeline management, setup, linting, execution, update.
- `pipeline/`: core pipeline library (frontend YAML compiler & linter → DAG → types → backend executors for Docker/Kubernetes/local/dummy) plus logging, tracer, RPC definitions.
- `web/`: Vue.js single-page app consuming server’s REST API.
- `shared/`, `woodpecker-go/`: shared utilities and Go client libraries.

##### README Insights
- CI/CD engine runs with SQLite by default (configurable).
- Components interact over REST, gRPC, and pub/sub.
- Agents connect to server for work via gRPC.
- Supports plugins, multiple SCM forges, extensible backends.

a. Main Components
- Client Tier:
- Web UI (Vue.js SPA)
- CLI (Go binary)
- woodpecker-go client library
- Server Tier:
- API Gateway (HTTP + gRPC endpoints in `server/api` & `server/grpc`)
- Business Services (configuration, encryption, environment, permissions, registry, secret)
- Queue & Pub/Sub (in-memory or persistent FIFO queue, pub/sub topic for logs/events)
- Scheduler/Cron subsystem
- Pipeline Orchestrator (`server/pipeline`, sharing core with `pipeline/`)
- Data Tier:
- Relational DB via `server/store/datastore` (SQLite, MySQL, Postgres) + migrations
- Execution Tier:
- Agent(s) (gRPC clients under `agent/`)
- Pipeline Backend Executors (Docker, Kubernetes, Local, Dummy under `pipeline/backend`)
- External Services:
- SCM Forge Webhooks (GitHub, GitLab, Bitbucket, Gitea, etc. via `server/forge/*`)
- Metrics/Monitoring (Prometheus endpoint)
- Container Registry integration
- Plugin registry (optional)

b. Relationships & Data Flows
- User → Web UI / CLI calls → Server API
- Server API → Services → Store (DB reads/writes)
- Server enqueues pipeline jobs → Queue → Agent polls
- Agent fetches job → runs via pipeline/backend → streams logs back → Server pub/sub → stored + forwarded to UI/CLI
- SCM webhook → Server API webhook handler → triggers pipeline creation
- Cron scheduler in server → enqueues scheduled pipeline jobs
- Metrics server exposes Prometheus metrics

c. Architectural Patterns & Principles
- Layered Architecture: API → Services → Store
- Event-Driven through Queue & Pub/Sub
- Plugin-Based Extensibility in pipeline backends and server forges
- Shared library (`pipeline/`) for core pipeline logic used by server and agent
- Micro-services boundary between Server and Agent via gRPC

d. Key Technologies & Frameworks
- Go language, gRPC, Protobuf
- HTTP router & middleware (`server/router`)
- XORM for DB migrations (`server/store/datastore/xorm.go`)
- Vue.js + Vite for Web UI
- Cobra/Viper (implied in CLI flags/config)
- Docker/Kubernetes clients in pipeline backend

## Diagram (Mermaid)

```mermaid
flowchart TD
  subgraph "Client Tier"
    WebUI["Web UI (Vue.js SPA)"]:::client
    CLI["CLI (Go Binary)"]:::client
    GoClient["Go Client Library"]:::client
  end

  subgraph "Server Process"
    subgraph "API Layer"
      Router["HTTP Router & Middleware"]:::server
      HTTPAPI["HTTP API Endpoints"]:::server
      GRPCAPI["gRPC Endpoints"]:::server
    end
    Services["Business Services"]:::server
    Cron["Cron Scheduler"]:::server
    Orchestrator["Pipeline Orchestrator"]:::server
    SharedPipeline["Shared Core Pipeline Library"]:::server
  end

  subgraph "Data & Messaging"
    DB["Relational Database"]:::data
    Queue["Queue Subsystem"]:::data
    PubSub["Pub/Sub Subsystem"]:::data
  end

  subgraph "Execution Tier"
    Agent["Agent (gRPC Client & Runner)"]:::exec
    BackendDocker["Docker Executor"]:::exec
    BackendK8s["Kubernetes Executor"]:::exec
    BackendLocal["Local Executor"]:::exec
    BackendDummy["Dummy Executor"]:::exec
  end

  subgraph "External Systems"
    SCM["SCM Forge Webhooks"]:::external
    Registry["Container Registry"]:::external
    Metrics["Prometheus Metrics Endpoint"]:::external
    PluginRegistry["Plugin Registry"]:::external
  end

  WebUI -->|HTTP| HTTPAPI
  CLI -->|CLI/API| HTTPAPI
  GoClient -->|Go Client| HTTPAPI

  SCM -->|Webhook HTTP| HTTPAPI

  HTTPAPI -->|routes| Router
  GRPCAPI -->|routes| Router
  Router -->|calls| Services

  Services -->|SQL| DB
  Services -->|enqueue| Queue
  Services -->|publish| PubSub
  Services -->|interact| Registry
  Services -->|metrics| Metrics

  Cron -->|schedule| Queue

  Queue -->|gRPC Poll| Agent
  Agent -->|RPC run| Orchestrator
  Agent -->|uses| SharedPipeline
  Orchestrator -->|uses| SharedPipeline

  Orchestrator -->|exec via| BackendDocker
  Orchestrator -->|exec via| BackendK8s
  Orchestrator -->|exec via| BackendLocal
  Orchestrator -->|exec via| BackendDummy

  Agent -->|streams logs| PubSub

  PubSub -->|subscribe| WebUI
  PubSub -->|subscribe| CLI
  PubSub -->|subscribe| HTTPAPI

  click WebUI "https://github.com/woodpecker-ci/woodpecker/tree/main/web/"
  click CLI "https://github.com/woodpecker-ci/woodpecker/tree/main/cli/"
  click GoClient "https://github.com/woodpecker-ci/woodpecker/tree/main/woodpecker-go/woodpecker/"
  click HTTPAPI "https://github.com/woodpecker-ci/woodpecker/tree/main/server/api/"
  click GRPCAPI "https://github.com/woodpecker-ci/woodpecker/tree/main/server/grpc/"
  click Router "https://github.com/woodpecker-ci/woodpecker/tree/main/server/router/"
  click Services "https://github.com/woodpecker-ci/woodpecker/tree/main/server/services/"
  click Queue "https://github.com/woodpecker-ci/woodpecker/tree/main/server/queue/"
  click PubSub "https://github.com/woodpecker-ci/woodpecker/tree/main/server/pubsub/"
  click Cron "https://github.com/woodpecker-ci/woodpecker/tree/main/server/cron/"
  click Orchestrator "https://github.com/woodpecker-ci/woodpecker/tree/main/server/pipeline/"
  click SharedPipeline "https://github.com/woodpecker-ci/woodpecker/tree/main/pipeline/"
  click BackendDocker "https://github.com/woodpecker-ci/woodpecker/tree/main/pipeline/backend/docker/"
  click BackendK8s "https://github.com/woodpecker-ci/woodpecker/tree/main/pipeline/backend/kubernetes/"
  click BackendLocal "https://github.com/woodpecker-ci/woodpecker/tree/main/pipeline/backend/local/"
  click BackendDummy "https://github.com/woodpecker-ci/woodpecker/tree/main/pipeline/backend/dummy/"
  click DB "https://github.com/woodpecker-ci/woodpecker/tree/main/server/store/datastore/"
  click Agent "https://github.com/woodpecker-ci/woodpecker/tree/main/agent/"
  click SCM "https://github.com/woodpecker-ci/woodpecker/tree/main/server/forge/"
  click Registry "https://github.com/woodpecker-ci/woodpecker/tree/main/server/services/registry/"
  click Metrics "https://github.com/woodpecker-ci/woodpecker/blob/main/cmd/server/metrics_server.go"
  click PluginRegistry "https://github.com/woodpecker-ci/woodpecker/tree/main/pipeline/backend/common/"

  classDef client fill:#dae8fc,stroke:#6c8ebf,color:#1f4e79
  classDef server fill:#d5e8d4,stroke:#82b366,color:#2e7d32
  classDef data fill:#fff2cc,stroke:#d6b656,color:#b35900
  classDef exec fill:#f8cecc,stroke:#b85450,color:#7f0000
  classDef external fill:#e1e1e1,stroke:#aaaaaa,color:#555555
```
