__ __ ______ _ __
/ / / /___ _ __ / ____/___ _ __(_)________ ____ ____ ___ ___ ____ / /______
/ /_/ / __ \ | /| / / / __/ / __ \ | / / / ___/ __ \/ __ \/ __ `__ \/ _ \/ __ \/ __/ ___/
/ __ / /_/ / |/ |/ / / /___/ / / / |/ / / / / /_/ / / / / / / / / / __/ / / / /_(__ )
/_/ /_/\____/|__/|__/ /_____/_/ /_/|___/_/_/ \____/_/ /_/_/ /_/ /_/\___/_/ /_/\__/____/
__ __ __
/ / / /___ _____ ____ ___ ____ / /
/ /_/ / __ `/ __ \/ __ \/ _ \/ __ \/ /
/ __ / /_/ / /_/ / /_/ / __/ / / /_/
/_/ /_/\__,_/ .___/ .___/\___/_/ /_(_)
═══════════/_/═══/_/═══════════════════════════════════════════════════════════════════════════
┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ │
│ In Puppet 5.3 │
│ │
└─────────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────┐ ┌─────────────┐ ┌───────────────┐
│Puppet Agent │ │Puppet Server│ │Node Classifier│
└─────────────┘ └─────────────┘ └───────────────┘
┃ ┃ ┃
┌────────────────┐
│server_list set?│ ┌───┐ ┃ ┃
│ (HA enabled) ├─┤Yes├──────┐
└───┬───┳────────┘ └───┘ │ ┃ ┃
┌─┴┐ │
│No│ ┃ │ ┃ ┃
└─┬┘ │
┌───┐ ┌───┴───┻───────────┐ │ ┃ ┃
┌ ┤Yes├ ┤node_name_fact set?│ │
└───┘ └───┬───┳───────────┘ │ ┃ ┃
│ ┌─┴┐ │
│No│ ┃ │ ┃ ┃
│ └─┬┘ │
┌ ─ ─ ─ ─ ─ │ ┃ │ ┃ ┃
Pluginsync├ ─ ─ ─◎ │
└ ─ ─ ─ ─ ─ │ ┃ │ ┃ ┃
│ │
┌───┐ ┌───┴───┻────────────────────┐│ ┃ ┃
┌─┤Yes├─┤strict_environment_mode set?││
│ └───┘ └───┬───┳────────────────────┘│ ┃ ┃
│ ┌─┴┐ │
│ │No│ ┃ │ ┃ ┃
│ └─┬┘ │ ┌───┐ ┌────────────────┐
│ └───╋─────────────────────┴───┤GET├─────▶│/puppet/v3/node │◀═══════════▶
│ └───┘ └──┬─────────────┘ ▲
│ ┌───────┻─────────────────────┐ │ ┃ │ ┃
│ │ Switch Env If necessary │◀────────────────┘ │
│ └───┬───┳─────────────────────┘ ┃ │ ┃
│ │ │
│ │ ┃ ┃ │ ┃
│ ┌───┴─────────────────────────┐ ┌──────────────┐ │
└───────┤ │◀════════════▶│ File Server │ │ ┃
│ Pluginsync │◀════════════▶│ API │ │
┌───────┤ │◀════════════▶│ Endpoints │ │ ┃
│ └───┬─────────────────────────┘ └──────────────┘ │
│ │ ┃ ┃ │ ┃
│ │ ┌───────────┐ │
│ │ ┃ ┌─▶│YAML Cache │──────┤ ┃
│ ┌───┴───────┐ │ ├───────────┤
│ │Load Facts │ ├─▶│ PuppetDB │─ ─ ─ ┘ ┃
│ └───┬───────┘ │ └───────────┘
│ │ ┃ │ ┃ ┃
│ │ │
│ ┌───┴───┻─────────────────────┐ ┌────┐ ┌──┴────┻──────────┐ ┃
│ │ Request Catalog │──┤POST├─────▶│/puppet/v3/catalog│◀═════════▶
┌─────┴────┐ └───────┳─────────────────────┘ └────┘ └──┬───────────────┘
│ No │ │
│ (Loop up │ ┌───────┻─────────────────────┐ │
│ to 3x) ├──┤Catalog Environment Matches? │◀────────────────┘
└──────────┘ └───┬───┳─────────────────────┘
│
┌─┴─┐ ┃
│Yes│
└─┬─┘ ┃
│
╔═══╩═══╩═════════════════════╗
║ Apply Catalog ║
╚═════════════════════════════╝
All puppet agent runs start out in the Puppet::Configurer#run method which
is invoked by the puppet daemon and puppet agent -t, or
puppet apply. The workflows used by puppet apply and puppet agents that
are configured to use cached catalogs don't check environments against a Puppet Server, so the
remainder of this document will focus on the workflow of an agent retrieving and enforcing a
freshly compiled catalog.
The first thing an agent run does is to determine the environment to use for pluginsync. This
is done by an GET call to the /puppet/v3/node API endpoint of the Puppet Server.
Notably, this API endpoint does not accept a data payload, which means no facts can be
submitted. However, the API request will include the agent's certificate, which means that
"trusted" node data is sent. the Puppet Server processing the API request will also load facts
from its cache, if available. The Puppet Server cache is pluggable (via routes.yaml) and split
into two levels, commonly configured as follows:
-
YAML cache: On-disk storage of the last factset directly submitted by nodes, or fetched from the level 2 cache. This usually contains factsets submitted by nodes as part of a catalog request and therefore contains "normal facts". Factsets fetched from the level 2 cache might have additional data, as noted below.
-
PuppetDB cache: Contains factsets stored in PuppetDB. This is used if the master doesn't have any data in the level 1 cache, or the data in the level 1 cache is expired. Both of these situations occur commonly in pools of compile masters. Notably, the factsets returned by PuppetDB have "trusted" node data merged together with "normal facts", which creates the potential for interesting edge cases if there is a miss in the level 1 cache.
The Puppet Server forwards the node request along with any cached facts to the configured Node Classifier and returns the response of that service to the agent.
If the puppet agent is part of a HA deployment where server_list is set, the initial request
to /puppet/v3/nodes will be used to select a healthy server from the list of available
options. Otherwise the value of the server setting will be used. The logic branches further
in the non-HA case if the node_name_fact setting is also set, with the agent performing a
pluginsync prior to the initial node request. This behavior is specific to the non-HA case,
and the node_name_fact setting is not commonly used.
If the response to the node request contains a different environment from the one specified
by puppet.conf, the agent will switch to that environment if the strict_environment_mode
setting is not enabled. The following message will be logged at NOTICE level when the agent
switches environments due to a node response:
Notice: Local environment: 'production' doesn't match server specified node environment '<node request env>', switching agent to '<node request env>'.
The agent then performs a pluginsync against the currently active environment and loads facts.
These facts are submitted as the data payload of a POST call to the
[/puppet/v3/catalog API endpoint][catalog-api] of the Puppet Server. The Puppet Server uses
these facts to refresh both the level 1 and level 2 caches and then performs another node
request to the configured Node Classifier service and compiles a catalog for the agent using
the environment returned by the Classifer. The compiled catalog is returned to the agent.
The agent compares the environment of the catalog the Puppet Server returned against its
currently active environment. If they differ and the strict_environment_mode setting is
enabled, then the run is failed. Otherwise, if the environments differ, the agent switches its
active environment and logs the following message at NOTICE level:
Notice: Local environment: 'production' doesn't match server specified environment '<catalog env>', restarting agent run with environment '<catalog env>'
When switching environments at this stage, the agent returns to pluginsync, reloads facts, and
issues a new POST request to the /puppet/v3/catalog API. If the agent receives a catalog that
matches its active environment, then it will continue on to the application phase of its run.
Otherwise, it will retry up to three times before the run is failed.