Skip to content

Instantly share code, notes, and snippets.

@eclecticmiraclecat

eclecticmiraclecat/Puppet 5 – From Beginner to Pro.md

Last active March 20, 2022 12:37
Show Gist options
  • Save eclecticmiraclecat/db838730710aa245557c317d754bbaab to your computer and use it in GitHub Desktop.
Save eclecticmiraclecat/db838730710aa245557c317d754bbaab to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Puppet 5 – From Beginner to Pro.md

notes from https://learning.oreilly.com/videos/puppet-5/9781789131642/

Chapter 1: Managing data with hiera

Why hiera?

  • look up configuration value
  • what changes in the future

Hiera locations

  • /etc/puppetlabs/code/environments/production/hiera.yaml
  • /etc/puppetlabs/puppet/hiera.yaml
$ cat /etc/puppetlabs/code/environments/pbg/hiera.yaml
---
version: 5

defaults:
  datadir: data
  data_hash: yaml_data

hierarchy:
  - name: "Common defaults"
    path: "common.yaml"

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml 
---
  test: 'This is a test'

$ sudo puppet lookup --environment pbg test
--- This is a test

type lookup

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml 
---
  apache_worker_factor: 100
  dns_allow_query: true

$ cat /examples/lookup2.pp 
notice("Apache is set to use ${lookup('apache_worker_factor', Integer)} workers")

notice('dns_allow_query enabled: ', lookup('dns_allow_query', Boolean))

$ sudo puppet apply --environment pbg /examples/lookup2.pp
Notice: Scope(Class[main]): Apache is set to use 100 workers
Notice: Scope(Class[main]): dns_allow_query enabled:  true

Types of hiera data

  • string
  • integer
  • bool
  • array
  • hash/dictionary
$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml 
---
  test: 'This is a test'
  consul_node: true
  apache_worker_factor: 100
  apparmor_enabled: true
  monitor_ips:
    - '10.179.203.46'
    - '212.100.235.160'
  cobbler_config:
    manage_dhcp: true
    pxe_just_once: true
  cms_parameters:
    static:
      sites_root: '/var/www/sites'
      assets_root: 'files'
      web_root: 'public_html'

$ cat /examples/lookup_type.pp
notice(lookup('apparmor_enabled', Boolean))
notice(lookup('test', String))
notice(lookup('apache_worker_factor', Integer))
notice(lookup('monitor_ips', Array))
$heights = lookup('monitor_ips', Array)
notice($heights[0])
notice(lookup('monitor_ips.0', String))
$cobbler_config = lookup('cobbler_config', Hash)
notice($cobbler_config['manage_dhcp'])
notice(lookup('cms_parameters.static.web_root', String))

$ sudo puppet apply --environment pbg /examples/lookup_type.pp 
Notice: Scope(Class[main]): true
Notice: Scope(Class[main]): This is a test
Notice: Scope(Class[main]): 100
Notice: Scope(Class[main]): [10.179.203.46, 212.100.235.160]
Notice: Scope(Class[main]): 10.179.203.46
Notice: Scope(Class[main]): 10.179.203.46
Notice: Scope(Class[main]): true
Notice: Scope(Class[main]): public_html
Notice: Compiled catalog for ubuntu-xenial in environment pbg in 0.07 seconds
Notice: Applied catalog in 0.01 seconds

more example

Interpolation in hiera data

  • lookup()
  • alias()

Can include facter facts

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml
---
  backup_path: "/backup/%{facts.hostname}"

lookup and alias

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml
---
  ips:
    home: '130.190.0.1'
    office1: '74.12.203.14'
    office2: '95.170.0.75'
  firewall_allow_list:
    - "%{lookup('ips.home')}"
    - "%{lookup('ips.office1')}"
    - "%{lookup('ips.office2')}"
  vpn_allow_list: "%{alias('firewall_allow_list')}"

Creating resources with hiera data

Building resources from array

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml
---
  users:
    - 'katy'
    - 'lark'
    - 'bridget'

$ cat /examples/hiera_users.pp 
lookup('users', Array[String]).each | String $username | {
  user { $username:
    ensure => present,
  }
}

$ sudo puppet apply --environment pbg /examples/hiera_users.pp

Building resources from hash

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml
---
  users2:
    'katy':
      ensure: present
      uid: 1900
      shell: '/bin/bash'
    'lark':
      ensure: present
      uid: 1901
      shell: '/bin/sh'
    'bridget':
      ensure: present
      uid: 1902
      shell: '/bin/bash'

$ cat /examples/hiera_users2.pp 
lookup('users2', Hash, 'hash').each | String $username, Hash $attrs | {
  user { $username:
    * => $attrs,
  }
}

$ sudo puppet apply --environment pbg /examples/hiera_users2.pp

Managing Secret Data

# apt-get install gnupg rng-tools

# rngd -f -r /dev/urandom &

# gpg --gen-key

# cat /etc/puppetlabs/code/environments/pbg/hiera.yaml 
---
version: 5

defaults:
  datadir: data
  data_hash: yaml_data

hierarchy:
  - name: "Secret data (encrypted)"
    lookup_key: eyaml_lookup_key
    path: "secret.eyaml"
    options:
      gpg_gnupghome: '/home/ubuntu/.gnupg'
      

# touch /etc/puppetlabs/code/environments/pbg/data/secret.yaml

# gem install gpgme
# gem install hiera-eyaml-gpg

# /opt/puppetlabs/puppet/bin/eyaml edit --gpg-always-trust [email protected] /etc/puppetlabs/code/environments/pbg/data/secret.eyaml
---
 test_secret: DEC::GPG[This is a test secret]!
---

# puppet lookup --environment pbg test_secret
--- This is a test secret

# cat /etc/puppetlabs/code/environments/pbg/data/secret.eyaml 
---
 test_secret: ENC[GPG,hQEMA744bSVIg6Y6AQgAxDcgErU3fY03VGJHNtVGeG/iOc4ArImbKqM1E093GDhxfGwXtP4mcZPDTXavGP8EXuMhjcmebc4GfGvyMJHZyTIURRYI5WIqxLHDsd5qYnQKoJmCOdtj/CORL/fZm4hKxyva2MKTziMjkJVSWeDgxmxuZp/FgNvKX65Vae7c/K1R55QXLZaFuiJr4OHTuxCN3GoEI9eGlL9sU+USpCD2yewkR0n2QiL7iGIRU5iQ5ttgUFWy0icUzzWEyOGC87J7c7rDSvrYEwTR8adWyI3onRzOKuJkj4hOyrkslMJwkj9PaDMq5ckSGviUgHkeW3b3vHlac+tEW6EFPV99NV+0GdJTAQ/H+wlLh3tVhRZ35S95petbak4Pc53vdxvmSzHmlcR/bdfqJzo6d0VmZ85Mum7fGaiFJo9vVHquZGZPyW6houCg3vsFCKknsg0S2bU4rq6HnQA=]
---

Chapter 2: Mastering Modules

Using puppet forge modules

install stdlib module using r10k

$ cat Puppetfile 
forge 'http://forge.puppetlabs.com'
mod 'puppetlabs/stdlib', '4.17.1'

$ sudo r10k puppetfile install

$ sudo puppet apply --environment pbg -e "notice(upcase('hello'))"
Notice: Scope(Class[main]): HELLO

Writing your own modules

  • develop a ntp module
  1. create a module directory structure
$ environments/pbg/modules/
$ mkdir -p pbg_ntp/{files,manifests}
  1. create init.pp in the manifests directory
$ cat pbg_ntp/manifests/init.pp

# Manage NTP
class pbg_ntp {
    ensure_packages(['ntp'])

    file { '/etc/ntp.conf':
    source  => 'puppet:///modules/pbg_ntp/ntp.conf',
    notify  => Service['ntp'],
    require => Package['ntp'],
    }

    service { 'ntp':
    ensure => running,
    enable => true,
    }
}
  1. create ntp.conf file in the files directory
$ cat pbg_ntp/files/ntp.conf

driftfile /var/lib/ntp/ntp.drift

pool 0.ubuntu.pool.ntp.org iburst
pool 1.ubuntu.pool.ntp.org iburst
pool ntp.ubuntu.com

restrict 127.0.0.1
  1. create metadata.json
$ cat pbg_ntp/metadata.json
{
  "name": "pbg_ntp",
  "version": "0.1.0",
  "author": "john doe",
  "summary": "",
  "license": "proprietary",
  "source": "",
  "project_page": null,
  "issues_url": null,
  "dependencies": [

  ]
}
  1. test the module
$ sudo puppet apply --environment=pbg -e 'include pbg_ntp'

Chapter 3: Classes, Roles and Profiles

Classes

class template

class CLASS_NAME {
...
}

include CLASS_NAME

declaring parameters to classes

$ cat /examples/class_params.pp 
# Manage NTP
class pbg_ntp_params (
  String $version = 'installed',
) {
  ensure_packages(['ntp'],
    {
      'ensure' => $version,
    }
  )
}

include pbg_ntp_params

passing parameters to classes via hiera

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml 
---
  pbg_ntp_params::version: 'latest'
  pbg_ntp_params2::start_at_boot: true
  pbg_ntp_params2::version: 'latest'
  pbg_ntp_params2::service_state: 'running'

$ cat /examples/class_params2.pp 
# Manage NTP
class pbg_ntp_params2 (
  Boolean $start_at_boot,
  # String[1] indicates minimum one string character needs to be passed
  String[1] $version                        = 'installed',
  # Enum['running', 'stopped'] indicates allowed values
  Enum['running', 'stopped'] $service_state = 'running',
) {
  ensure_packages(['ntp'],
    {
      'ensure' => $version,
    }
  )

  service { 'ntp':
    ensure => $service_state,
    enable => $start_at_boot,
  }
}

include pbg_ntp_params2

Defined Resource Types and type aliases

 cat /examples/defined_resource_type.pp 
# Manage user and SSH key together
define user_with_key(
  Enum[
    'dsa',
    'ssh-rsa',
    'rsa',
    'ed25519'
  ] $key_type,
  String[1] $key,
) {
  user { $title:
    ensure     => present,
    home       => "/home/${title}",
    managehome => true,
  }

  file { "/home/${title}/.ssh":
    ensure => directory,
    owner  => $title,
    group  => $title,
    mode   => '0700',
  }

  ssh_authorized_key { $title:
    user => $title,
    type => $key_type,
    key  => $key,
  }
}

user_with_key { 'john':
  key_type => 'ssh-rsa',
  key      => 'AAAAB3NzaC1yc2EAAAABIwAAAIEA3ATqENg+GWACa2BzeqTdGnJhNoBer8x6pfWkzNzeM8Zx7/2Tf2pl7kHdbsiTXEUawqzXZQtZzt/j3Oya+PZjcRpWNRzprSmd2UxEEPTqDw9LqY5S2B8og/NyzWaIYPsKoatcgC7VgYHplcTbzEhGu8BsoEVBGYu3IRy5RkAcZik=',
}

aliases

$ cat /examples/type_alias.pp 
type ServiceState = Enum['running', 'stopped'] 

define myservice(ServiceState $state) {
  service { $name:
    ensure => $state,
  }
}

myservice { 'ntp':
  state => 'running',
}

Chapter 4: Managing Files with Templates

What are templates

$ cat /examples/aws_credentials.epp 
<%- | String $aws_access_key | -%>
aws_access_key_id = <%= $aws_access_key %>

$ #aws_access_key_id = 1233aqsd

Using templates in your manifests

$ cat /examples/backup.sh.epp 
<%- | String $data_dir | -%>
#!/bin/bash
mkdir -p /backup
tar cvzf /backup/backup.tar.gz <%= $data_dir %>


$ cat /examples/file_epp.pp 
file { '/usr/local/bin/backup':
  content => epp('/examples/backup.sh.epp',
    {
      'data_dir' => '/examples',
      }
  ), 
  mode    => '0755',
}

$ sudo puppet apply /examples/file_epp.pp

$ cat /usr/local/bin/backup 
#!/bin/bash
mkdir -p /backup
tar cvzf /backup/backup.tar.gz /examples

inline epp

$ cat /examples/file_inline_epp.pp 
$web_root = '/var/www'
$backup_dir = '/backup/www'

file { '/usr/local/bin/backup':
  content => inline_epp('rsync -a <%= $web_root %>/ <%= $backup_dir %>/'),
  mode    => '0755',
}

$ sudo puppet apply /examples/file_inline_epp.pp

$ cat /usr/local/bin/backup 
rsync -a /var/www/ /backup/www/

computation in templates

$ cat /examples/template_compute.epp 
innodb_buffer_pool_size=<%= $facts['memory']['system']['total_bytes'] * 3/4 %>

$ sudo puppet epp render /examples/template_compute.epp
innodb_buffer_pool_size=780066816

conditional statements in templates

$ cat /examples/template_if.epp
<% if $ssl_enabled { -%>
  ## SSL directives
  SSLEngine on
  SSLCertificateFile      "<%= $ssl_cert %>"
  SSLCertificateKeyFile   "<%= $ssl_key %>"
  ...
<% } -%>

$ cat /examples/if_template.pp
$ssl_enabled = true
$ssl_cert = 'hi_cert'
$ssl_key = 1234

file { '/usr/local/bin/backup':
  content => epp('/examples/template_if.epp')
}

$ sudo puppet apply /examples/if_template.pp

$ cat /usr/local/bin/backup 
  ## SSL directives
  SSLEngine on
  SSLCertificateFile      "hi_cert"
  SSLCertificateKeyFile   "1234"
  ...

Iteration in templates

  • iterating over
    1. facter data
    2. structured data
    3. hiera data

hash loop template

HASH.each | KEY, VALUE | {
BLOCK
}

loop over facter data

$ cat /examples/template_iterate.epp 
<% $facts['networking']['interfaces'].each |String $interface, Hash $attrs| { -%>
interface <%= $interface %>;
<% } -%>


$ facter networking.interfaces
{
  enp0s3 => {
  },
  lo => {
  }
}


$ sudo puppet epp render /examples/template_iterate.epp
interface lo;
interface enp0s3;

loop over hiera data

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml 
---
  users:
    - 'katy'
    - 'lark'
    - 'bridget'
    - 'hsing-hui'
    - 'charles'


$ cat /examples/template_hiera.epp 
AllowUsers<% lookup('users').each | $user | { -%>
 <%= $user -%>
<% } %> 


$ sudo puppet epp render --environment pbg /examples/template_hiera.epp
AllowUsers katy lark bridget hsing-hui charles

loop over hiera data with param

$ cat /examples/template_params.epp 
<% | String[1] $aws_access_key,
     String[1] $aws_secret_key,
| -%>
aws_access_key_id = <%= $aws_access_key %>
aws_secret_access_key = <%= $aws_secret_key %>


$ cat /examples/epp_params.pp 
file { '/root/aws_credentials':
  content => epp('/examples/template_params.epp',
    {
      'aws_access_key' => 'AKIAIAF7V6N2PTOIZVA2',
      'aws_secret_key' => '7IBpXjoYRVbJ/rCTVLaAMyud+i4co11lVt1Df1vt',
    }
  ),
}


$ sudo puppet apply /examples/epp_params.pp

$ sudo cat /root/aws_credentials
aws_access_key_id = AKIAIAF7V6N2PTOIZVA2
aws_secret_access_key = 7IBpXjoYRVbJ/rCTVLaAMyud+i4co11lVt1Df1vt

$ cat /etc/puppetlabs/code/environments/pbg/data/common.yaml 
---
  users:
    - 'katy'
    - 'lark'
    - 'bridget'
    - 'hsing-hui'
    - 'charles'


$ cat /examples/template_hiera_params.epp 
<% | Array[String] $users | -%>
AllowUsers<% $users.each | $user | { -%>
 <%= $user -%>
<% } %> 


$ cat /examples/epp_hiera.pp 
file { '/tmp/sshd_config_example':
  content => epp('/examples/template_hiera_params.epp',
    {
      'users' => lookup('users'),
    }
  ),
}


$ sudo puppet  apply --environment pbg /examples/epp_hiera.pp

$ cat /tmp/sshd_config_example 
AllowUsers katy lark bridget hsing-hui charles

troubleshoot templates

$ puppet epp validate /examples/template_params.epp

$ puppet epp render --values "{ 'aws_access_key' => 'foo', 'aws_secret_key' => 'bar' }" /examples/template_params.epp
aws_access_key_id = foo
aws_secret_access_key = bar

$ echo "{ 'aws_access_key' => 'foo', 'aws_secret_key' => 'bar' }" > params.pp

$ puppet epp render --values_file params.pp /examples/template_params.epp
aws_access_key_id = foo
aws_secret_access_key = bar

$ puppet epp render --values "{ 'aws_access_key' => 'foo' }" -e 'hello, <%= $aws_access_key %>'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment