How puppetlabs/puppet works
Project Type and Purpose
• Puppet is an open-source, Ruby-based configuration management framework.
• It comprises both a CLI (agent, apply, module, face APIs) and a server side (Puppet Master / HTTP services).
• It implements an Indirector pattern, pcore/AST compiler, file-serving, provider subsystems, and HTTP transport.
File Structure
• Top-level executables in bin/puppet map to classes in lib/puppet/application/* (agent, apply, catalog, filebucket, lookup, module, node, report, ssl, etc.).
• Core library under lib/puppet handles settings, environments, parsing, compiler, indirector, HTTP client/server, file_serving/file_bucket, providers, and resource types.
• ext/* holds OS-specific integration (init scripts, systemd unit, Windows service, etc.).
• api/* describes JSON/PSON HTTP schemas for catalogs, reports, nodes, facts.
• conf/* holds environment.conf, fileserver.conf, puppet.conf, hiera.yaml.
• tests and acceptance specs live under spec/ and acceptance/.
README for Technical Stack
• Ruby gem, uses RSpec for tests, supports HTTP API (REST/JSON or PSON).
• Documentation points to HTTP API Index – you expose services over HTTP.
A. Client & CLI Apps (bin/puppet → lib/puppet/application)
• puppet agent (runs on each node, scheduled run)
• puppet apply (stand-alone manifest compile & apply)
• puppet module, puppet lookup, puppet doc, etc.
B. Core Library (lib/puppet)
• Settings & Configuration loader (puppet/settings, puppet/etc)
• Environment loader & Hiera lookup (puppet/environments, puppet/data_binding)
• Parser & AST / pcore compiler (puppet/parser, puppet/pops)
• Indirector subsystem (puppet/indirector) for data types: facts, catalogs, nodes, reports, file_metadata/content
• HTTP support (puppet/http client & service, puppet/network/http)
• File Serving & File Bucket (puppet/file_serving, puppet/file_bucket)
• Scheduler (puppet/scheduler) & Configurer (puppet/configurer) for plugin sync, file downloads
• Resource application layer (puppet/transaction, puppet/resource) and Providers (puppet/provider/*)
• SSL (puppet/ssl) and X.509 certificate management
C. Master / HTTP Services (could be puppetserver or built-in service definitions)
• CA service (signing, revocation)
• Compiler service (catalog compilation)
• File Server service (module & static file serving)
• Report service (accept JSON reports from agents)
D. Data Stores & File System
• Module path & environment directories (files on disk)
• Hiera data sources (YAML/JSON, configured per environment)
• SSL certificate directory (certs, private keys)
• File bucket (backups of file resources)
E. Extensible OS Integration (ext/*)
• Init scripts (SysV, systemd)
• Windows services & batch scripts
• Platform-specific defaults (solaris SMF, SUSE init, OSX plist)
1. Agent run triggered by cron/system scheduler
2. Agent → HTTP client POST /facts to Master
3. Master HTTP /facts → Indirector → Facts terminus (store/lookup)
4. Master HTTP /catalog?node=NAME → Catalog compiler → environment loader → parser & pops → graph builder → catalog JSON/PSON
5. HTTP client on Agent pulls catalog
6. Agent Configurer downloads plugins via file_serving, file_bucket, plugin_handler
7. Agent applies catalog: resource harness & provider calls (exec, file, package, service, user…)
8. Agent collects report & sends HTTP POST /report to Master
9. Master stores or forwards report (Indirector → report terminus)
Architectural Patterns & Design Principles
• Indirector (strategy) pattern: abstract “what” vs. “where” data lives.
• Plugin architecture (“faces” API, resource providers, functions under puppet/functions & parser/functions).
• Separation of concerns: CLI apps, core compiler, HTTP transport, file serving, resource application.
• Extensibility via ext/ and via ruby providers & functions.
• Pluggable back-ends (termini) for each data type (memory, rest, yaml, msgpack).
– Blue: Agent-side components
– Green: Master-side / HTTP services
– Orange: Core libraries / compiler / indirector
– Gray: Ext / OS integration
– Brown: Data store (filesystem, Hiera, SSL, file bucket)
• Technologies:
– Ruby / gem
– HTTP JSON/PSON
– Indirector terminus types
– Parser & pcore compiler
– RSpec & Beaker acceptance tests (optional, shown at edge)