# teslamotors/ansible_puller architecture

> Architecture diagram of the GitHub repository teslamotors/ansible_puller, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/teslamotors/ansible_puller
- Repository: https://github.com/teslamotors/ansible_puller

## Overview

##### Project Type and Purpose
- This is a daemon written in Go that extends the functionality of Ansible’s ansible-pull. Instead of using Git for distribution, it relies on HTTP and S3 to download Ansible playbooks, inventories, and related artifacts. Additional features include integration with Prometheus for monitoring and a REST API for controlling daemon operation (e.g., enabling/disabling or triggering a run).

##### Analyzing the File Structure
- At the top level, you have Go source files (e.g., main.go, ansible.go, http.go, s3_downloader.go) and corresponding test files. This indicates that the core functionalities are divided into dedicated modules.
- There’s a BUILD file, Bazel configuration files (like .bazelrc and .bazelversion), and a WORKSPACE which shows that the project uses Bazel for builds alongside standard Go module management (go.mod, go.sum).
- There are templates in a dedicated directory (templates). These likely form part of the REST API UI (e.g., ansible_controller.html and index.html).
- Utility modules such as idempotent_download.go, unarchive.go, util.go, and venv.go denote distinct functions:
- Downloading artifacts via HTTP or S3.
- Unarchiving the received tarballs.
- Managing Python virtual environments for running the Ansible playbooks.
- A few helper scripts and configuration examples (ansible-puller.json, ansible-puller.service) support deployment and service management.

##### Architectural Components and their Interactions
- Configuration Manager
- Reads configuration files (paths: /etc/ansible-puller/config.json, $HOME/.ansible-puller.json, or local ansible-puller.json). It also handles runtime configuration such as HTTP listen address, credentials, and S3 parameters.
- Download Manager
- Consists of HTTP downloader modules (http.go, http_downloader.go) and S3 downloader modules (s3_downloader.go). They fetch the Ansible tarball either directly via HTTP or from an S3 endpoint.
- Incorporates MD5 checksum support (via logic in unarchive.go and idempotent_download.go) to validate artifacts before processing.
- Extraction and Preprocessing Module
- Unarchives and prepares the tarball that contains the Ansible resources.
- Reads the playbook, inventory, and requirements files to set up the local Ansible environment.
- Virtual Environment Manager
- Uses venv.go to create and manage the Python virtual environment required for running Ansible, ensuring that dependencies (from requirements.txt) are met.
- Ansible Executor
- Responsible for invoking the Ansible playbook via the configured command (site.yml by default) once the virtualenv is prepared.
- Integrates with the logic for iterating over multiple inventories to determine which one—among production/staging directories or others—matches the current host.
- REST API and Monitoring Interface
- The daemon exposes a REST API that provides control actions (e.g., enabling/disabling and triggering runs).
- Monitoring endpoints are provided, integrating with Prometheus. Metrics such as ansible_puller_last_success, run times, and play summary are exposed.
- The HTTP server (via configurations in http.go and its related modules) handles incoming API requests as well as serves basic UI pages from the templates directory.
- Scheduler/Sleep Mechanism
- Configuration (via the “sleep” option) controls how frequently the daemon triggers new Ansible runs.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Entities
    HTTP_EXT("HTTP Endpoint (External)"):::external
    S3_EXT("S3 Endpoint (External)"):::external
    PROM("Prometheus (External)"):::external

    %% Core Components
    CM("Configuration Manager"):::config
    SCHED("Scheduler/Sleep Mechanism"):::scheduler

    subgraph "Download Manager"
        HTTP_DM("HTTP Download Manager"):::download
        S3_DM("S3 Download Manager"):::download
    end

    IDEMP("Checksum & Idempotency Validator"):::validator
    UNARCHIVE("Unarchive & Preprocessing Module"):::process
    VENV("Virtual Environment Manager"):::environment
    ANSIBLE("Ansible Executor"):::executor
    REST_API("REST API & Monitoring Interface"):::api

    %% Connections
    SCHED -->|"triggers_run"| HTTP_DM
    SCHED -->|"triggers_run"| S3_DM

    CM -->|"config_params"| HTTP_DM
    CM -->|"config_params"| S3_DM
    CM -->|"config_params"| VENV
    CM -->|"config_params"| REST_API

    HTTP_EXT -->|"provides_artifact"| HTTP_DM
    S3_EXT -->|"provides_artifact"| S3_DM

    HTTP_DM -->|"downloaded_artifact"| IDEMP
    S3_DM -->|"downloaded_artifact"| IDEMP

    IDEMP -->|"validated_artifact"| UNARCHIVE
    UNARCHIVE -->|"prepared_data"| VENV

    VENV -->|"env_ready"| ANSIBLE
    ANSIBLE -->|"execution_output"| REST_API
    ANSIBLE -->|"metrics_data"| PROM

    %% Click Events
    click CM "https://github.com/teslamotors/ansible_puller/blob/master/ansible-puller.json"
    click HTTP_DM "https://github.com/teslamotors/ansible_puller/blob/master/http.go"
    click HTTP_DM "https://github.com/teslamotors/ansible_puller/blob/master/http_downloader.go"
    click S3_DM "https://github.com/teslamotors/ansible_puller/blob/master/s3_downloader.go"
    click IDEMP "https://github.com/teslamotors/ansible_puller/blob/master/idempotent_download.go"
    click UNARCHIVE "https://github.com/teslamotors/ansible_puller/blob/master/unarchive.go"
    click VENV "https://github.com/teslamotors/ansible_puller/blob/master/venv.go"
    click ANSIBLE "https://github.com/teslamotors/ansible_puller/blob/master/ansible.go"
    click REST_API "https://github.com/teslamotors/ansible_puller/blob/master/http.go"
    click REST_API "https://github.com/teslamotors/ansible_puller/tree/master/templates/"
    click SCHED "https://github.com/teslamotors/ansible_puller/blob/master/main.go"

    %% Styles
    classDef config fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef download fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef validator fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef process fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef environment fill:#e2e3e5,stroke:#d6d8db,stroke-width:2px;
    classDef executor fill:#f5c6cb,stroke:#721c24,stroke-width:2px;
    classDef api fill:#cff4fc,stroke:#0c5460,stroke-width:2px;
    classDef scheduler fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef external fill:#e2e3e5,stroke:#999999,stroke-width:2px;
```
