# puppetlabs/puppet architecture

> Architecture diagram of the GitHub repository puppetlabs/puppet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/puppetlabs/puppet
- Repository: https://github.com/puppetlabs/puppet

## Overview

##### Project Type and Purpose
- Puppet is an open-source, Ruby-based configuration management framework.
- It comprises both a CLI (agent, apply, module, face APIs) and a server side (Puppet Master / HTTP services).
- It implements an Indirector pattern, pcore/AST compiler, file-serving, provider subsystems, and HTTP transport.

##### File Structure
- Top-level executables in bin/puppet map to classes in lib/puppet/application/* (agent, apply, catalog, filebucket, lookup, module, node, report, ssl, etc.).
- Core library under lib/puppet handles settings, environments, parsing, compiler, indirector, HTTP client/server, file_serving/file_bucket, providers, and resource types.
- ext/* holds OS-specific integration (init scripts, systemd unit, Windows service, etc.).
- api/* describes JSON/PSON HTTP schemas for catalogs, reports, nodes, facts.
- conf/* holds environment.conf, fileserver.conf, puppet.conf, hiera.yaml.
- tests and acceptance specs live under spec/ and acceptance/.

##### README for Technical Stack
- Ruby gem, uses RSpec for tests, supports HTTP API (REST/JSON or PSON).
- Documentation points to HTTP API Index – you expose services over HTTP.

A. Client & CLI Apps (bin/puppet → lib/puppet/application)
- puppet agent (runs on each node, scheduled run)
- puppet apply (stand-alone manifest compile & apply)
- puppet module, puppet lookup, puppet doc, etc.
B. Core Library (lib/puppet)
- Settings & Configuration loader (puppet/settings, puppet/etc)
- Environment loader & Hiera lookup (puppet/environments, puppet/data_binding)
- Parser & AST / pcore compiler (puppet/parser, puppet/pops)
- Indirector subsystem (puppet/indirector) for data types: facts, catalogs, nodes, reports, file_metadata/content
- HTTP support (puppet/http client & service, puppet/network/http)
- File Serving & File Bucket (puppet/file_serving, puppet/file_bucket)
- Scheduler (puppet/scheduler) & Configurer (puppet/configurer) for plugin sync, file downloads
- Resource application layer (puppet/transaction, puppet/resource) and Providers (puppet/provider/*)
- SSL (puppet/ssl) and X.509 certificate management
C. Master / HTTP Services (could be puppetserver or built-in service definitions)
- CA service (signing, revocation)
- Compiler service (catalog compilation)
- File Server service (module & static file serving)
- Report service (accept JSON reports from agents)
D. Data Stores & File System
- Module path & environment directories (files on disk)
- Hiera data sources (YAML/JSON, configured per environment)
- SSL certificate directory (certs, private keys)
- File bucket (backups of file resources)
E. Extensible OS Integration (ext/*)
- Init scripts (SysV, systemd)
- Windows services & batch scripts
- Platform-specific defaults (solaris SMF, SUSE init, OSX plist)

1. Agent run triggered by cron/system scheduler
2. Agent → HTTP client POST /facts to Master
3. Master HTTP /facts → Indirector → Facts terminus (store/lookup)
4. Master HTTP /catalog?node=NAME → Catalog compiler → environment loader → parser & pops → graph builder → catalog JSON/PSON
5. HTTP client on Agent pulls catalog
6. Agent Configurer downloads plugins via file_serving, file_bucket, plugin_handler
7. Agent applies catalog: resource harness & provider calls (exec, file, package, service, user…)
8. Agent collects report & sends HTTP POST /report to Master
9. Master stores or forwards report (Indirector → report terminus)

##### Architectural Patterns & Design Principles
- Indirector (strategy) pattern: abstract “what” vs. “where” data lives.
- Plugin architecture (“faces” API, resource providers, functions under puppet/functions & parser/functions).
- Separation of concerns: CLI apps, core compiler, HTTP transport, file serving, resource application.
- Extensibility via ext/ and via ruby providers & functions.
- Pluggable back-ends (termini) for each data type (memory, rest, yaml, msgpack).

– Blue: Agent-side components
– Green: Master-side / HTTP services
– Orange: Core libraries / compiler / indirector
– Gray: Ext / OS integration
– Brown: Data store (filesystem, Hiera, SSL, file bucket)
- Technologies:
– Ruby / gem
– HTTP JSON/PSON
– Indirector terminus types
– Parser & pcore compiler
– RSpec & Beaker acceptance tests (optional, shown at edge)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Agent Layer
    subgraph "Agent" 
        direction TB
        CLIApp["CLI Applications"]:::agent
        AgentHTTP["HTTP Client"]:::agent
        AgentConfigurer["Configurer & Scheduler"]:::agent
        AgentResource["Resource Application & Providers"]:::agent
    end

    %% Core Library Subsystems
    subgraph "Core Library Subsystems"
        direction TB
        Settings["Settings & Configuration Loader"]:::core
        EnvHiera["Environment Loader & Hiera Lookup"]:::core
        ParserAST["Parser & pcore Compiler"]:::core
        Indirector["Indirector Subsystem"]:::core
        HTTPSupport["HTTP Support (Client & Service)"]:::core
        FileServe["File Serving & File Bucket"]:::core
        SchedulerCore["Scheduler & Configurer"]:::core
        ResourceCore["Resource Application & Providers"]:::core
        SSLCore["SSL & X.509 Management"]:::core
    end

    %% Master HTTP Services
    subgraph "Master HTTP Services"
        direction TB
        CAService["CA Service"]:::master
        CompilerService["Compiler Service"]:::master
        FileServerService["File Server Service"]:::master
        ReportService["Report Service"]:::master
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        ModulePath["Module Path & Env Directories"]:::data
        HieraData["Hiera Data Sources"]:::data
        SSLCerts["SSL Certificate Directory"]:::data
        FileBucketDS["File Bucket Storage"]:::data
    end

    %% OS Integration
    subgraph "OS Integration (ext/)" 
        direction TB
        ExtOS["ext/"]:::ext
    end

    %% Flows
    CLIApp -->|starts| AgentHTTP
    AgentHTTP -->|"POST /facts"| Indirector
    AgentHTTP -->|"GET /catalog?node=NAME"| CompilerService
    CompilerService -->|"invoke compile"| EnvHiera
    EnvHiera -->|"load manifests & data"| ParserAST
    ParserAST -->|"AST/pcore"| Indirector
    Indirector -->|"catalog JSON/PSON"| CompilerService
    CompilerService -->|"catalog response"| AgentConfigurer
    AgentConfigurer -->|"download plugins"| FileServe
    AgentConfigurer -->|"download files"| FileBucketDS
    AgentConfigurer -->|"apply resources"| AgentResource
    AgentResource -->|"provider calls"| ResourceCore
    AgentResource -->|"SSL handshake"| SSLCore
    AgentResource -->|"OS integration"| ExtOS
    AgentResource -->|"log & events"| ModulePath
    AgentResource -->|"backup files"| FileBucketDS
    AgentHTTP -->|"POST /report"| ReportService
    ReportService -->|"store report"| Indirector

    %% Core Usage of Data Stores
    EnvHiera -->|reads| ModulePath
    EnvHiera -->|reads| HieraData
    SSLCore -->|reads/writes| SSLCerts
    FileBucketDS -->|stores| FileBucketDS

    %% Click Events
    click CLIApp "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/application"
    click Settings "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/settings"
    click Settings "https://github.com/puppetlabs/puppet/blob/main/lib/puppet/etc.rb"
    click EnvHiera "https://github.com/puppetlabs/puppet/blob/main/lib/puppet/environments.rb"
    click EnvHiera "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/data_binding"
    click ParserAST "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/parser"
    click ParserAST "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/pops"
    click Indirector "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/indirector"
    click HTTPSupport "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/http"
    click HTTPSupport "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/http/service"
    click FileServe "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/file_serving"
    click FileServe "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/file_bucket"
    click SchedulerCore "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/scheduler"
    click SchedulerCore "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/configurer"
    click ResourceCore "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/transaction"
    click ResourceCore "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/provider"
    click SSLCore "https://github.com/puppetlabs/puppet/tree/main/lib/puppet/ssl"
    click CAService "https://github.com/puppetlabs/puppet/blob/main/lib/puppet/http/service/ca.rb"
    click CompilerService "https://github.com/puppetlabs/puppet/blob/main/lib/puppet/http/service/compiler.rb"
    click FileServerService "https://github.com/puppetlabs/puppet/blob/main/lib/puppet/http/service/file_server.rb"
    click ReportService "https://github.com/puppetlabs/puppet/blob/main/lib/puppet/http/service/report.rb"
    click ExtOS "https://github.com/puppetlabs/puppet/tree/main/ext/"
    click ModulePath "https://github.com/puppetlabs/puppet/blob/main/conf/environment.conf"
    click ModulePath "https://github.com/puppetlabs/puppet/blob/main/conf/puppet.conf"
    click ModulePath "https://github.com/puppetlabs/puppet/blob/main/conf/fileserver.conf"
    click HieraData "https://github.com/puppetlabs/puppet/blob/main/conf/hiera.yaml"

    %% Styles
    classDef agent fill:#cce5ff,stroke:#004085;
    classDef master fill:#d4edda,stroke:#155724;
    classDef core fill:#ffe5b4,stroke:#e6a800;
    classDef data fill:#f5e1c1,stroke:#663d00;
    classDef ext fill:#e2e3e5,stroke:#6c757d;
```
