How microsoft/azure-pipelines-agent works
Project type and purpose
• This is the cross‐platform Azure Pipelines Agent: a .NET Core C# application that connects to Azure DevOps/TFS, polls for build or release jobs, and executes pipeline tasks on Windows, Linux, and macOS hosts.
• It uses a plugin‐based architecture to host a variety of task plugins (artifact download, test result parsing, caching, etc.), and communicates over HTTP/REST with Azure DevOps services (JobServer, BlobStore, Telemetry).
File structure analysis
• Top‐level folders:
• .azure-pipelines, .github, release, tools – CI/CD pipelines, signing, release scripts, tooling
• docs – design docs, threat models, troubleshooting, contribution guidelines
• src – main code, broken into subprojects:
– Agent.Listener (CLI entrypoint & job dispatcher)
– Agent.Worker (task execution engine)
– Agent.Service (Windows service wrapper)
– Agent.Plugins (artifact, cache, test publisher, result parser, Git/SVN sources)
– Agent.PluginHost (isolated plugin host process)
– Agent.Sdk (common library for plugins & core)
– Microsoft.VisualStudio.Services.Agent (shared core abstractions)
– Misc (layout scripts, packaging templates)
• Test – unit (L0) and integration (L1) tests structured by component
• Configuration and build: Common.props, NuGet.Config, .NET target files, YAML pipelines, PowerShell/ bash helpers.
Key insights from README
• Emphasizes cross‐platform .NET Core, supported OSes.
• Highlights build & test status, contribution & troubleshooting pointers.
1. Agent.Listener process (CLI):
• Subcomponents: CommandLine handlers (ConfigureAgent, RunAgent, RemoveAgent), Configuration managers (Linux, Windows, macOS), SelfUpdater, TelemetryPublisher.
2. Agent.Service (Windows service host).
3. Agent.Worker process:
• Subcomponents: JobRunner, TaskManager, ExecutionContext, Container manager, CodeCoverage, TestResults, Telemetry, Handlers (PowerShell, Node, ProcessHandler).
4. Agent.PluginHost (optional isolated host for heavy plugins).
5. Agent.Plugins library set:
• Artifact plugins (PipelineArtifact, BuildArtifact, FileShare), PipelineCache, TestFilePublisher, TestResultParser, Git/SVN source plugins, etc.
6. Agent.Sdk & Microsoft.VisualStudio.Services.Agent libraries (shared abstractions, HTTP clients, credential store, logging, knobs).
7. External services:
• Azure DevOps Server / Azure Pipelines REST API (JobServer, TaskServer, BlobStore, LocationServer)
• External systems: Git/TFS repositories, Docker Daemon, Test frameworks, File shares.
• Listener → Azure DevOps Server (poll/push over HTTP)
• Listener → JobDispatcher → Worker(s)
• Worker ↔ PluginHost (loads plugins via Agent.Sdk)
• Worker → Plugins (invoke task code)
• Plugins → External resources (download/upload artifacts, talk to blob store, Git/TFS)
• Worker → Container engine (Docker) for container‐based tasks
• Worker → Test runners → TestResults → Azure DevOps Test APIs
• All components → Telemetry service (CustomerIntelligenceServer)
c. Architectural patterns & design principles
• Plugin architecture: runtime discovery & execution isolation
• Process separation: CLI (Listener) decoupled from execution engine (Worker) and optional PluginHost
• Cross‐platform abstractions: OS‐specific configuration managers under a common interface
• Dependency inversion: core uses interfaces (ITraceWriter, ICredentialStore, IContainerProvider) implemented per platform
• Resilience: retry helpers, cancellation tokens, job timeouts, configuration knobs
d. Technologies & frameworks
• .NET Core / .NET 6/.NET 8 C#
• YAML pipelines for CI (Azure DevOps YAML)
• PowerShell & Bash scripts for installation, signing, release
• REST clients (HttpClient) for Azure DevOps services
• Telemetry (CustomerIntelligenceServer over HTTPS)
• Docker CLI & container APIs