How elastic/elastic-agent works
1. Overall shape
- This repo is a monorepo for Elastic Agent, plus an embedded EDOT/OpenTelemetry collector implementation.
- The codebase is split between the agent runtime, component/runtime libraries, packaging/deployment assets, and a large integration-test harness.
2. Primary entrypoints
- main.go starts the main Elastic Agent binary.
- internal/edot/main.go starts the EDOT collector binary/path.
- internal/pkg/basecmd and internal/pkg/agent/cmd provide the command surface for install, run, enroll, status, diagnostics, upgrade, and related operations.
3. Agent control plane
- The agent’s lifecycle is centered in internal/pkg/agent/application.
- That layer handles enrollment, policy changes, reexec/restart, diagnostics, privilege changes, migration, and upgrade orchestration.
- It is the boundary between external control signals and the local runtime state.
4. Fleet communication
- internal/pkg/fleetapi implements the network protocol for enroll, check-in, ack, and action upload/download.
- The agent uses this to receive policy/actions from Fleet and report status back.
- internal/pkg/agent/application/gateway/fleet adapts those Fleet interactions into the agent’s internal coordinator model.
5. Policy and configuration expansion
- Local and Fleet-managed configuration is normalized in internal/pkg/config, internal/pkg/agent/configuration, and internal/pkg/agent/transpiler.
- internal/pkg/composable is a key boundary: it resolves config from multiple providers such as env, host, path, local, filesource, and kubernetes.
- That config is then transformed into runnable component specs and per-component runtime settings.
6. Component runtime model
- pkg/component and pkg/component/runtime define how the agent launches and supervises individual components.
- This includes subprocess management, command wiring, connection-info exchange, state tracking, log capture, and service-style supervision.
- The runtime layer is the execution boundary between “agent orchestration” and “actual workloads.”
7. OpenTelemetry / EDOT subsystem
- internal/edot is a self-contained OpenTelemetry collector distribution with its own component registry, validation, and command handling.
- internal/pkg/otel contains the agent-side OTel manager, config translation, diagnostics, status serialization, and collector execution management.
- This subsystem bridges Elastic Agent policy/configuration into OTel collector components and output destinations.
8. Monitoring, diagnostics, and health
- internal/pkg/agent/application/monitoring and internal/pkg/otel/manager expose liveness/readiness/status and internal monitoring endpoints.
- Diagnostics are first-class: there are dedicated command handlers, component collectors, and OTel diagnostics support.
- This is important for diagramming because monitoring is not just an add-on; it is a separate runtime concern.
9. State, secrets, security, and upgrades
- Persistent agent state lives under internal/pkg/agent/storage and internal/pkg/agent/vault.
- Secrets and encrypted config are handled with platform-specific implementations, including FIPS-aware code paths.
- Upgrade flow is explicit and multi-step in internal/pkg/agent/application/upgrade, including download, verification, unpack, relink, rollback, and service update logic.
10. Platform and OS-specific boundaries
- There is heavy cross-platform specialization throughout unix, windows, and darwin files.
- The repo includes platform-specific permissions, service management, process handling, sockets, file locking, and keychain/storage paths.
- This matters architecturally because the same logical agent is implemented through different host integration layers per OS.
11. Deployment artifacts
- deploy/helm and deploy/kubernetes provide production deployment manifests for Elastic Agent in Kubernetes, both standalone and Fleet-managed modes.
- docs/manifests and dev-tools/kubernetes contain generated/example manifests and kustomize overlays used for documentation and testing.
- The Kubernetes packaging is a major boundary: it packages the same agent runtime into cluster-native deployment patterns.
12. Packaging and release engineering
- dev-tools/packaging and dev-tools/mage drive build, package, checksum, notice generation, cross-builds, and release artifacts.
- The repo supports multiple package types, including tar/zip, docker images, and platform-specific installers.
- wrapper/windows/archive-proxy is a separate Go module used in Windows packaging workflows.
13. Testing and validation ecosystem
- The repo has a substantial integration-test system under testing/ and test_infra/, covering Elastic Cloud, Kubernetes, serverless, upgrades, and failure modes.
- CI is split across Buildkite and GitHub Actions, with pipelines for unit, integration, packaging, docs, and release automation.
- This testing layer is part of the architecture because many behaviors are validated only through full agent deployment scenarios.
14. External systems and outputs
- The agent ultimately talks to Fleet Server / Fleet, Elasticsearch, Logstash, Kafka, and Kubernetes APIs depending on mode and component.
- EDOT/OTel paths also interact with OTel collector ecosystems and Elastic-specific exporters/extensions.
- The repo is therefore best diagrammed as a control-plane driven agent that orchestrates local component processes, not as a single service.
Main entry points: application Application · integration Integration tests · kubernetes Kubernetes deploy · main.go Main binary
Components 22
Entrypoints
Top-level binaries and command surfaces for agent and EDOT collector
- Main binary · Starts the primary Elastic Agent process
main.go - EDOT binary · Starts the embedded collector distribution
internal/edot/main.go - Base commands · Shared command wiring for install, run, status, diagnostics, and restart
internal/pkg/basecmd
Agent Core
Elastic Agent lifecycle, Fleet control plane, config, storage, upgrade, and runtime orchestration
- Agent CLI · Agent-specific command handlers for install, enroll, run, upgrade, and diagnostics
internal/pkg/agent/cmd - Application · Central agent lifecycle coordinator for policy, enrollment, privileges, reexec, diagnostics, and upgrade
internal/pkg/agent/application - Coordinator · Turns control-plane events into runtime operations and component updates
internal/pkg/agent/application/coordinator - Fleet gateway · Adapts Fleet actions and check-ins into coordinator-facing operations
internal/pkg/agent/application/gateway/fleet - Fleet API · Enroll, check-in, ack, and action upload/download protocol implementation
internal/pkg/fleetapi - Config stack · Normalizes local and Fleet-managed configuration into agent runtime settings
internal/pkg/config - Composable config · Resolves config from env, host, path, local, filesource, docker, and kubernetes providers
internal/pkg/composable - Transpiler · Converts expanded config into runnable component specs and settings
internal/pkg/agent/transpiler - Component runtime · Launches, supervises, and reports state for individual components and subprocesses
pkg/component/runtime - Component model · Defines component structure, specs, and load-time contracts
pkg/component - State & vault · Stores agent state, encrypted config, and secret material
internal/pkg/agent/storage - Upgrade flow · Downloads, verifies, unpacks, relinks, rolls back, and updates services during upgrade
internal/pkg/agent/application/upgrade
EDOT / OTel
Embedded collector distribution and agent-side OTel management
- OTel manager · Executes and supervises the embedded collector, health checks, diagnostics, and status serialization
internal/pkg/otel/manager - OTel translate · Maps agent policy and outputs into collector config and status
internal/pkg/otel/translate - Collector dist · Standalone EDOT/OpenTelemetry collector packaging, registry, validation, and command handling
internal/edot
Platform Boundary
OS-specific process, permissions, service, and IPC integration
- Host integration · Cross-platform process, permissions, IPC, sockets, services, and keychain handling
pkg/core/process
Delivery & Test
Packaging, Kubernetes deployment artifacts, and integration test harnesses
- Kubernetes deploy · Helm and kustomize manifests for standalone and Fleet-managed agent deployment
deploy/kubernetes - Packaging tools · Builds installers, containers, notices, checksums, and release artifacts
dev-tools/packaging - Integration tests · End-to-end validation across Fleet, Kubernetes, upgrades, and failure modes
testing/integration
Connections 28
- Main binary → Base commands: boots · Main binary uses the shared command surface
- Base commands → Agent CLI: dispatches · Command parsing flows into agent-specific handlers
- Agent CLI → Application: starts · CLI commands invoke the lifecycle application layer
- Application → Coordinator: drives · Application delegates runtime orchestration to the coordinator
- Coordinator → Fleet gateway: consumes · Coordinator receives Fleet-driven actions through the gateway adapter
- Fleet gateway → Fleet API: calls · Gateway translates internal operations into Fleet protocol requests
- Fleet API → Application: feeds actions · Enroll/check-in responses and actions update application state
- Application → Config stack: builds config · Control-plane decisions shape the runtime configuration pipeline
- Config stack → Composable config: expands · Configuration is resolved across multiple providers
- Composable config → Transpiler: emits · Expanded config becomes component-oriented runtime input
- Transpiler → Component model: specifies · Transpiler outputs component specs and wiring contracts
- Component model → Component runtime: executes in · Component definitions are launched under the runtime supervisor
- Component runtime → Host integration: uses host APIs · Runtime relies on OS-specific process and service integration
- Application → State & vault: persists · Lifecycle data, secrets, and state are stored durably
- Application → Upgrade flow: orchestrates · Upgrade sequencing is coordinated by the application layer
- Upgrade flow → Host integration: updates host · Upgrade steps interact with OS-specific file, service, and process behavior
- EDOT binary → Collector dist: boots · EDOT entrypoint starts the collector distribution
- Collector dist → OTel translate: validates/translates · Collector distribution translates and validates collector configuration
- OTel translate → OTel manager: configures · Translated config is executed by the OTel manager
- Application → OTel manager: manages OTel · Agent control plane coordinates embedded collector execution
- OTel manager → Component runtime: reuses runtime · Collector execution relies on the same subprocess/runtime boundary
- Kubernetes deploy → Main binary: packages · Kubernetes manifests deploy the agent binary
- Kubernetes deploy → EDOT binary: packages · Kubernetes manifests also package the collector distribution
- Packaging tools → Main binary: builds · Release tooling produces agent binaries and installers
- Packaging tools → EDOT binary: builds · Release tooling produces EDOT collector artifacts
- Integration tests → Application: exercises · Integration tests validate the agent lifecycle and control plane
- Integration tests → Fleet API: simulates · Test harnesses emulate Fleet interactions and actions
- Integration tests → Kubernetes deploy: verifies · Kubernetes integration tests validate deployment artifacts