How teslamotors/ansible_puller works
Project Type and Purpose
• This is a daemon written in Go that extends the functionality of Ansible’s ansible-pull. Instead of using Git for distribution, it relies on HTTP and S3 to download Ansible playbooks, inventories, and related artifacts. Additional features include integration with Prometheus for monitoring and a REST API for controlling daemon operation (e.g., enabling/disabling or triggering a run).
Analyzing the File Structure
• At the top level, you have Go source files (e.g., main.go, ansible.go, http.go, s3_downloader.go) and corresponding test files. This indicates that the core functionalities are divided into dedicated modules.
• There’s a BUILD file, Bazel configuration files (like .bazelrc and .bazelversion), and a WORKSPACE which shows that the project uses Bazel for builds alongside standard Go module management (go.mod, go.sum).
• There are templates in a dedicated directory (templates). These likely form part of the REST API UI (e.g., ansible_controller.html and index.html).
• Utility modules such as idempotent_download.go, unarchive.go, util.go, and venv.go denote distinct functions:
• Downloading artifacts via HTTP or S3.
• Unarchiving the received tarballs.
• Managing Python virtual environments for running the Ansible playbooks.
• A few helper scripts and configuration examples (ansible-puller.json, ansible-puller.service) support deployment and service management.
Architectural Components and their Interactions
• Configuration Manager
• Reads configuration files (paths: /etc/ansible-puller/config.json, $HOME/.ansible-puller.json, or local ansible-puller.json). It also handles runtime configuration such as HTTP listen address, credentials, and S3 parameters.
• Download Manager
• Consists of HTTP downloader modules (http.go, http_downloader.go) and S3 downloader modules (s3_downloader.go). They fetch the Ansible tarball either directly via HTTP or from an S3 endpoint.
• Incorporates MD5 checksum support (via logic in unarchive.go and idempotent_download.go) to validate artifacts before processing.
• Extraction and Preprocessing Module
• Unarchives and prepares the tarball that contains the Ansible resources.
• Reads the playbook, inventory, and requirements files to set up the local Ansible environment.
• Virtual Environment Manager
• Uses venv.go to create and manage the Python virtual environment required for running Ansible, ensuring that dependencies (from requirements.txt) are met.
• Ansible Executor
• Responsible for invoking the Ansible playbook via the configured command (site.yml by default) once the virtualenv is prepared.
• Integrates with the logic for iterating over multiple inventories to determine which one—among production/staging directories or others—matches the current host.
• REST API and Monitoring Interface
• The daemon exposes a REST API that provides control actions (e.g., enabling/disabling and triggering runs).
• Monitoring endpoints are provided, integrating with Prometheus. Metrics such as ansible_puller_last_success, run times, and play summary are exposed.
• The HTTP server (via configurations in http.go and its related modules) handles incoming API requests as well as serves basic UI pages from the templates directory.
• Scheduler/Sleep Mechanism
• Configuration (via the “sleep” option) controls how frequently the daemon triggers new Ansible runs.