How woodpecker-ci/woodpecker works
Project Type & Purpose
• Woodpecker is a full-stack, extensible CI/CD engine written in Go, composed of a server (API, business logic, persistence, queueing, pub/sub), an agent for executing pipeline steps on various backends, a CLI for scripting/automation, and a web UI for human interaction.
High-Level File Structure Analysis
• Top-level “cmd/…” folders: entry points for woodpecker-server, woodpecker-agent, woodpecker-cli.
• server/: REST & gRPC API handlers, router, services layer, store (database + migrations), pub/sub, queue, cron scheduler, model definitions.
• agent/: gRPC client, pipeline runner, tracer, state management.
• cli/: commands for admin/org/repo/pipeline management, setup, linting, execution, update.
• pipeline/: core pipeline library (frontend YAML compiler & linter → DAG → types → backend executors for Docker/Kubernetes/local/dummy) plus logging, tracer, RPC definitions.
• web/: Vue.js single-page app consuming server’s REST API.
• shared/, woodpecker-go/: shared utilities and Go client libraries.
README Insights
• CI/CD engine runs with SQLite by default (configurable).
• Components interact over REST, gRPC, and pub/sub.
• Agents connect to server for work via gRPC.
• Supports plugins, multiple SCM forges, extensible backends.
a. Main Components
• Client Tier:
• Web UI (Vue.js SPA)
• CLI (Go binary)
• woodpecker-go client library
• Server Tier:
• API Gateway (HTTP + gRPC endpoints in server/api & server/grpc)
• Business Services (configuration, encryption, environment, permissions, registry, secret)
• Queue & Pub/Sub (in-memory or persistent FIFO queue, pub/sub topic for logs/events)
• Scheduler/Cron subsystem
• Pipeline Orchestrator (server/pipeline, sharing core with pipeline/)
• Data Tier:
• Relational DB via server/store/datastore (SQLite, MySQL, Postgres) + migrations
• Execution Tier:
• Agent(s) (gRPC clients under agent/)
• Pipeline Backend Executors (Docker, Kubernetes, Local, Dummy under pipeline/backend)
• External Services:
• SCM Forge Webhooks (GitHub, GitLab, Bitbucket, Gitea, etc. via server/forge/*)
• Metrics/Monitoring (Prometheus endpoint)
• Container Registry integration
• Plugin registry (optional)
b. Relationships & Data Flows
• User → Web UI / CLI calls → Server API
• Server API → Services → Store (DB reads/writes)
• Server enqueues pipeline jobs → Queue → Agent polls
• Agent fetches job → runs via pipeline/backend → streams logs back → Server pub/sub → stored + forwarded to UI/CLI
• SCM webhook → Server API webhook handler → triggers pipeline creation
• Cron scheduler in server → enqueues scheduled pipeline jobs
• Metrics server exposes Prometheus metrics
c. Architectural Patterns & Principles
• Layered Architecture: API → Services → Store
• Event-Driven through Queue & Pub/Sub
• Plugin-Based Extensibility in pipeline backends and server forges
• Shared library (pipeline/) for core pipeline logic used by server and agent
• Micro-services boundary between Server and Agent via gRPC
d. Key Technologies & Frameworks
• Go language, gRPC, Protobuf
• HTTP router & middleware (server/router)
• XORM for DB migrations (server/store/datastore/xorm.go)
• Vue.js + Vite for Web UI
• Cobra/Viper (implied in CLI flags/config)
• Docker/Kubernetes clients in pipeline backend