Puppet study notes - Seamus Murray's Notes about Linux and System Engineering

Seamus Murray's Puppet study notes as of January 2015

PuppetLabs own documentation https://puppetlabs.com/learn

Sign up to the PuppetLabs learning portal https://puppetlabs.com/learn

2) go through the puppetlabs videos and questionnaires, it will give you an idea of the language used in the tests https://puppetlabs.com/learn/library

some of the videos seem to be based on lessons that can be found elsewhere of the puppet website (sorry I cant find them now )

3) puppetlabs provides a vm with puppet enterprise installed, inside the zip file there is a pdf with a series of exercises http://puppetlabs.com/download-learning-vm

4) puppet practice exam

https://docs.puppetlabs.com/references/glossary.html

6) glossary of the key puppet terminology

https://puppetlabs.com/services/certification/puppet-professional-practice-exam

Puppet installation

System requirements

Puppet Enterprise 3.7 supports the following systems:

+------------------+---------------------------+-------------+------------------------------------+
|Operating system  |Version(s)                 | Arch        |Component(s)                        |
|------------------|---------------------------|-------------|------------------------------------|
|RHEL              |4, 5, 6, 7                 |x86_64       |all (RHEL 4 supports agent only)    |
|CentOS            |4, 5, 6, 7                 |x86 & x86_64 |all (CentOS 4 supports agent only)  |
|Ubuntu LTS        |10.04, 12.04, 14.04        |i386 & amd64 |all                                 |
|Debian            |Squeeze (6), Wheezy (7)    |i386 & amd64 |all                                 |
|Oracle Linux      |4, 5, 6, 7                 |x86 & x86_64 |all (Oracle Linux 4 agent only)     |
|Scientific Linux  |4, 5, 6                    |x86 & x86_64 |all (Scientific Linux 4 agent only) |
|SUSE Enterprise   |10 (SP4 only), 11(SP1), 12 |x86 & x86_64 |all (SLES 10 supports agent only)   |
|Solaris           |10 (Update 9 or later) 11  |SPARC & i386 |agent                               |
|Microsoft Windows |2008 2008R2 7 Ultimate SP1 |x86 & x64    |agent                               |
|                  |8 Pro, 8.1 Pro 2012 2012R2 |x86 & x64    |agent                               |
|Microsoft Windows |2003, 2003R2               |x86          |agent                               |
|AIX               |5.3, 6.1, 7.1              |Power        |agent                               |
|Mac OS X          |Mavericks (10.9)           |x86_64       |agent                               |
+------------------+---------------------------+-------------+------------------------------------+

https://docs.puppetlabs.com/pe/latest/install_basic.html#about-puppet-enterprise-components the puppet enterprise 3.1 guide refers to puppet master role database support role console role

Monolithic (all-in-one) Installation

Monolithic installs are suitable for deployments up to 500 nodes. We recommend that your hardware meets the following:

The puppet master, PE console, and PuppetDB node: at least 4-8 processor cores, 8 GB RAM
All machines require very accurate timekeeping
Puppet agent nodes: any hardware able to run the supported operating system
For /var/, at least 1 GB of free space for each PE component on a given node
For PE-installed PostgreSQL, at least 100 GB of free space in /opt/ for data gathering
For no PE-installed PostgreSQL, /opt/ needs at least 1 GB of disk space available

Split Installation

For larger deployments (500-1000, or more nodes), we recommend a split install. We recommend that your hardware meets the following:

Puppet master, PE console, and PuppetDB nodes: at least 8 processor cores, 8 GB RAM (per node)
All machines require very accurate timekeeping
Puppet agent nodes: any hardware able to run the supported operating system
For /var/, at least 1 GB of free space for each PE component on a given node
For PE-installed PostgreSQL, at least 100 GB of free space in /opt/ for data gathering
For no PE-installed PostgreSQL, /opt/ needs at least 1 GB of disk space available

Firewall Configuration

Agent nodes contact the puppet master server on TCP ports 8140 (for Puppet) and 61613 (for orchestration). Any hosts that need access to the html GUI will contact the console server on port 443(can be changed to a different port).

+-----------------+---------+---------+---------+----------+
|                 |  AGENT  | MASTER  | CONSOLE | ADMIN PC |
|-----------------|---------|---------|---------|----------|
|puppet           |   ----> |   8140  |         |          |
|Orchestration    | 61613-->|<--61613 |         |          |
|HTML GUI         |         |         |   443   | <----    |
|Master Installer |         |   3000  |         | <----    | #html installer
+-----------------+---------+---------+---------+----------+

From 3.1 installation guide... https://docs.puppetlabs.com/pe/3.1/install_system_requirements.html

Hardware requirements for the various puppet roles/components

Puppet Enterprise’s hardware requirements depend on the roles a machine performs.

The puppet master role should be installed on a robust, dedicated server. Minimum requirements: 2 processor cores, 1 GB RAM, and very accurate timekeeping. Recommended requirements: 2-4 processor cores, at least 4 GB RAM, and very accurate timekeeping. Performance will vary, but this configuration can generally manage approximately 1,000 agent nodes.
The database support role can be installed on the same server as the console or, optionally, on a separate, dedicated server. Minimum requirements: These will vary considerably depending on the size of your deployment. However, you’ll need a machine able to handle moderate network traffic, perform processor-intensive background tasks, and run a disk-intensive PostgreSQL database server. The machine should have two to four processor cores. As a rough rule of thumb for RAM needed, start here: 1-500 nodes: 192-1024MB, 500-1000 nodes: 1-2GB, 1000-2000 nodes: 2-4 GB, 2000+ nodes, 4GB or greater. So as your deployment scales, make sure to scale RAM allocations accordingly. More information about scaling PuppetDB is available in the PuppetDB manual’s scaling guidelines.
The console role should usually be installed on a separate server from the puppet master, but can optionally be installed on the same server in smaller deployments. Minimum requirements: A machine able to handle moderate network traffic and perform processor-intensive background tasks. It should have a very fast network connection to the database support server, which it uses for all of the console’s database requirements. Requirements will vary significantly depending on the size and complexity of your site.
The optional cloud provisioner role has very modest requirements. Minimum requirements: A system which provides interactive shell access for trusted users. This system should be kept very secure, as the cloud provisioning tools must be given cloud service account credentials in order to function.
The puppet agent role has very modest requirements. Minimum requirements: Any hardware able to comfortably run a supported operating system.

Install script

[root@master puppet-enterprise-3.7.1-el-6-x86_64]# ./puppet-enterprise-installer -h

Puppet Enterprise v3.7.1 installer

Puppet Enterprise documentation can be found at http://docs.puppetlabs.com/pe/3.7/

USAGE: puppet-enterprise-installer [-a ANSWER_FILE] [-A ANSWER_FILE] [-D] [-h] [-l LOG_FILE] [-n] [-q] [-s ANSWER_FILE] [-V]

OPTIONS:

-a ANSWER_FILE - Read answers from file and quit with error if an answer is missing.
-A ANSWER_FILE - Read answers from file and prompt for input if an answer is missing.
-D - Display debugging information.
-h - Display this help.
-l LOG_FILE - Log commands and results to file.
-n - Run in 'noop' mode; show commands that would have been run during installation without running them
-q - Run in quiet mode; the installation process is not displayed. Requires answer file.
-s ANSWER_FILE - Save answers to file and quit without installing.
-V - Display very verbose debugging information.

Install the pe-agent

curl -k https://<puppet master server>:8140/packages/current/install.bash | sudo bash

puppet.conf

/etc/puppetlabs/puppet/puppet.conf {puppet enterprise} or /etc/puppet/puppet.conf {puppet opensource}

[main]
    certname = learning.puppetlabs.vm
    vardir = /var/opt/lib/pe-puppet
    logdir = /var/log/pe-puppet
    rundir = /var/run/pe-puppet
    basemodulepath = /etc/puppetlabs/puppet/modules:/opt/puppet/share/puppet/modules
    environmentpath = /etc/puppetlabs/puppet/environments
    server = learning.puppetlabs.vm
    user  = pe-puppet
    group = pe-puppet
    archive_files = true
    archive_file_server = learning.puppetlabs.vm
    module_groups = base+pe_only

[agent]
    report = true
    classfile = $vardir/classes.txt
    localconfig = $vardir/localconfig
    graph = true
    pluginsync = true
    environment = production

SSL certificates and accurate time

If there is a clock sync issue between an agent and the master you will receive certificate verification errors..

root@node3:~# puppet agent -t Warning: Unable to fetch my node definition, but the agent run will continue: Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=master.puppetlabs] Info: Retrieving plugin Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=master.puppetlabs] Error: /File[/var/lib/puppet/lib]: Could not evaluate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=master.puppetlabs] Could not retrieve file metadata for puppet://puppet/plugins: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=master.puppetlabs] Error: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=master.puppetlabs] Warning: Not using cache on failed catalog Error: Could not retrieve catalog; skipping run Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=master.puppetlabs]

resolution sync the clock's and restart puppet agent service or manually run puppet agent -t

Resetting a nodes certificate

+--------------------------------+-----------------------------+
|       PUPPET AGENT             |    PUPPET MASTER            |
|--------------------------------|-----------------------------|
|  service puppet stop           |                             |
|  puppet config print ssldir    |                             |
|  rm -rf  /var/lib/puppet/ssl   |                             |
|                                | puppet cert list --all      |
|                                | puppet cert clean <certname>|
|  puppet agent -t               |                             |
|                                | puppet cert list            |
|                                | puppet sign <certname>      |
|  puppet agent -t               |                             |
|  service puppet start          |                             |
+--------------------------------+-----------------------------+

Until the puppet master signs the nodes cert the agent will display the following message

root@node3:~# puppet agent -t
Exiting; no certificate found and waitforcert is disabled

Puppet Cert command line options

puppet cert  <action>  <host>

clean: Revoke a host's certificate (if applicable) and remove all files related to that host from puppet cert's storage. This is useful when rebuilding hosts, since new certificate signing requests will only be honored if puppet cert does not have a copy of a signed certificate for that host. If '--all' is specified then all host certificates, both signed and unsigned, will be removed.
fingerprint: Print the DIGEST (defaults to the signing algorithm) fingerprint of a host's certificate.
generate: Generate a certificate for a named client. A certificate/keypair will be generated for each client named on the command line.
list: List outstanding certificate requests. If '--all' is specified, signed certificates are also listed, prefixed by '+', and revoked or invalid certificates are prefixed by '-' (the verification outcome is printed in parenthesis).
print: Print the full-text version of a host's certificate.
revoke: Revoke the certificate of a client. The certificate can be specified either by its serial number (given as a hexadecimal number prefixed by '0x') or by its hostname. The certificate is revoked by adding it to the Certificate Revocation List given by the 'cacrl' configuration option. Note that the puppet master needs to be restarted after revoking certificates.
sign: Sign an outstanding certificate request.
verify: Verify the named certificate against the local CA certificate.
reinventory: Build an inventory of the issued certificates. This will destroy the current inventory file specified by 'cert_inventory' and recreate it from the certificates found in the 'certdir'. Ensure the puppet master is stopped before running this action.

Node Classification site.pp ENC

Certname

By default each nodes certification name is its FQDN. When the puppet agent on the nodes request at catalogue from the puppet master it does so using its current FQDN {even if they had already generated a certificate and had it signed by the master}. Therefore if the nodes host or domain name changes it will make a request based on the new name and you will have to sign a new cert on the master.

Classification of nodes

In puppet the term classification refers to the linking/assigning of classes to nodes. This can be achieved in many ways

** Using site.pp **

node '<group name>' {
  include <class>
}

node '<node name>' {
  include <class>
}

** Using the puppet enterprise console**

Click on Classification in the console navigation bar. ( Optional steps to create a new group )
1. Enter a new node group name
2. click Add group
Click on the desired group and set the rules for this group based on facter values name/osfamily etc... click add rule
Click commit change

If your changes to the site.pp manifest are'nt reflected in a puppet run triggered by the puppet agent -t command, try running the command again.

** Using an ENC External node Classifier ** An executable that can be called by puppet master; it doesn’t have to be written in Ruby. Its only argument is the name of the node to be classified, and it returns a YAML document describing the node.

Node statements are an optional feature of Puppet. They can be replaced by or combined with an external node classifier. You can also use conditional statements upon facts to classify nodes.

site.pp syntax

The Default Node

The site.pp file has an optional default node declaration . It follows the same format as a normal node declaration except the word default (without quotes) is used in place of a cert/host name. If a node requests a catalogue and there is no matching cert/host name in the site.pp then the node will be assigned what ever settings have be set for the default node.

You can simply use the node name:

node 'www.example.com' {
  include common
  include apache, squid
}

You can use a comma-separated list of names to create a group of nodes with a single node statement:

node 'www1.example.com', 'www2.example.com', 'www3.example.com' {
  include common
  include apache, squid
}

You can use Regular expressions (regexes) can be used as node names. In puppet regexes must be surrounded by forward slashes /

match any node with the name www followed by 1 or more digits (www1 , www12345)

node /^www\d+$/ {
  include common
}

match foo.example.com and bar.example.com

node /^(foo|bar)\.example\.com$/ {
  include common
}

If site.pp contains at least one node definition, it must have one for every node; compilation for a node will fail if one cannot be found. (Hence the usefulness of the default node.) If site.pp contains no node definitions, this requirement is dropped.

Matching

If a nodes certname/hostname is matched by more than one node statement it will only get the contents of one node definition. Puppet will do the following checks in order when deciding which definition to use:

If there is a node definition with the node’s exact name, Puppet will use it.
If there is a regular expression node statement that matches the node’s name, Puppet will use it. (If more than one regex node matches, Puppet will use one of them, with no guarantee as to which.)
If the node’s name looks like a fully qualified domain name (i.e. multiple period-separated groups of letters, numbers, underscores and dashes), Puppet will chop off the final group and start again at step 1. (That is, if a definition for www01.example.com isn’t found, Puppet will look for a definition matching www01.example.)
Puppet will use the default node.

Thus, for the node www1.example.com, Puppet would try the following, in order:

www1.example.com
A regex that matches www1.example.com
www1.example
A regex that matches www1.example
www1
A regex that matches www1
default

In my testing with a site.pp

node 'node2.puppetlabs' {
  include hosts
}

node /^node(2|4)/ {
  include regexmod
  include definedmod
  include ntp
}

with the node/cert name of node2.puppetlabs it is matched my the most accurate name ie "node2.puppetlabs"

if the site.pp only has the shortname and a shortname+regex then the regex its matched

eg..

node 'node2' {
  include hosts
}

node /^node(2|4)/ {
  include regexmod
  include definedmod
  include ntp
}

Merging site.pp With ENC Data

Node definitions and external node classifiers can co-exist. Puppet merges their data as follows:

Variables from an ENC are set at top scope and can thus be overridden by variables in a node definition. Classes from an ENC are declared at node scope, which means they will be affected by any variables set in the node definition.

Although both will work together it is recommend to use either node definitions or an ENC.

External nodes override node configuration in the manifest files. If you enable an external node classifier, any duplicate node definitions in your manifest files will not be processed; they will be ignored by Puppet.

The use of LDAP nodes overrides node definitions in your manifest files and your ENC. If you use LDAP node definitions, you cannot define nodes in your manifest files or in an ENC.

Inheritance

In earlier versions of Puppet, nodes could inherit from other nodes using the inherits keyword. This feature is deprecated in Puppet 3.7,

node classification in puppet enterprise console

click the classification tab select the group or create a new group set the rules to match the node

static matching, add the nodes FQDN in the certname field and pin the node to the group
dynamic matching, create a fact + operator + value based rule to match the node, e.g.

Table detailing possible matching rules

+-----------+----------------+------------------------+
|  FACT     | OPERATOR       |  VALUE or REGEX        |
+-----------+----------------+------------------------+
|  hostname |  is            |  pe-node2              |
|  name     |  is            |  pe-node3.puppetlabs   |
|  name     |  matches regex |  pe-node\d.puppetlabs  |
|  name     |  matches regex |  node(4|5).puppetlabs  |
|  osfamily |  is            |  RedHat                |
+-----------+----------------+------------------------+

Note Don't forget to click add Rule and then click commit x change

default node group

PE comes preconfigured with the default node group. This is the root of the node group hierarchy and is a parent to all other node groups. The classes that are assigned to the default node group are applied to all of the nodes in your deployment.

Deleting Nodes from puppet enterprise console

There are three options for deleting a node:

Hide Node
Removes the node from the node list view. To hide a node:
```
On the Nodes page, click the node.
Click Hide.
```
Delete Node

Removes all reports and information for the node from the console. The node no longer appears in the list of nodes on the Nodes page, but it continues to appear in Matching nodes until it is purged from PuppetDB. The node will reappear as a new node on the Nodes page if it submits a new Puppet run report. To delete a node:
```
On the Nodes page, click the node.
Click Delete.
```
Deactivate Node

Completely deactivates the node and frees up the license assigned to the node. Any deactivated node will be reactivated if PuppetDB receives new catalogues or facts for it. For details, see Deactivating a PE agent node.
1. Stop the agent service on the node you want to deactivate.
2. On the Puppet master, deactivate the agent; run puppet node deactivate {NODE NAME}. This deactivates the agent in PuppetDB. In some cases, the PE license count in the console will not decrease for up to 24 hours, but you can restart the pe-memcached service to update the license count sooner.
3. On the Puppet master, revoke the agent certificate; run puppet cert revoke {AGENT CERTNAME}.
4. Still on the Puppet master, run puppet agent -t to kick off a Puppet run. This Puppet run will copy the certificate revocation list (CRL) to the correct SSL directory for delivery to the agent.
5. Restart the Puppet master with service pe-puppetserver restart. The certificate is only revoked after running service pe-puppetserver restart. In addition, the Puppet server process won’t re-read the certificate revocation list until the service is restarted. If you don’t run service pe-puppetserver restart, the node will check in again on the next Puppet run and re-register with PuppetDB, which will increment the license count again.
6. Delete the node from the console. In the console, click Nodes. Click the node that you want to delete and click the Delete button. This action does NOT disable MCollective/live management on the node.
7. To disable MCollective/live management on the node, uninstall the Puppet agent, stop the pe-mcollective service (on the agent, run service pe-mcollective stop), or destroy the agent altogether. On the agent, remove the node certificates in /etc/puppetlabs/mcollective/ssl/clients.
8. clean the cert from the pauppet master ...puppet cert clean {AGENT CERTNAME}
Note: If you delete a node from the node view without first deactivating the node, the node will be absent from the node list in the console, but the license count will not decrease, and on the next Puppet run, the node will be listed in the console.

At this point, the node should be fully deactivated.

Regex in puppet

Regexes in Puppet cannot have options or encodings appended after the final slash. However, you may turn options on or off for portions of the expression using the (?:) and (?-:) notation. The following example enables the i option while disabling the m and x options:

 $packages = $operatingsystem ? {
   /(?i-mx:ubuntu|debian)/        => 'apache2',
   /(?i-mx:centos|fedora|redhat)/ => 'httpd',
 }

The following options are allowed:

i — Ignore case
m — Treat a newline as a character matched by .
x — Ignore white space and comments in the pattern

Puppet run overview

The SSL checks are made first
The agent checks for a certificate matching its FQDN
- If one is not found the agent generates one.
The agent checks for the CA cert.
- if not it will send its cert to the CA servers and request for its cert to be signed.
If plugin sync is enabled, the agent checks the master for new plugins and downloads them if necessary.
The agent ask's Facter for a set of facts about itself. { facter -p }
The agent sends the facts the master whilst requesting a catalog.
The master injects those facts as variables in the root scope and processes the manifests.
The master then sends the catalog to the agent.
The agent apply's the catalog.
If reporting is enabled the agent sends a report back to the master.

Diagram of puppet run

+-----------------------+     +-----------------------+
|        AGENT          |     |        MASTER         |
|  +-----------------+  |     |                       |
|  | Check local Cert|  |     |     +--------------+  |
|  | create if needed|  |     | +-->| Send CA Cert |  |
|  +-----------------+  |     | |   +--------------+  |
| +------------------+  |     | |                     |
| | Check 4 copy of  |  |     | |                     |
| | CA Cert,retrieve |  |     | |                     |
| | if needed        |<---------+                     |
| +------------------+  |     |                       |
| +-------------------+ |     |  +------------------+ |
| | Is my Cert signed |<-------->| Sign Client Cert | |
| +-------------------+ |     |  +------------------+ |
|                       |     |                       |
|  +-----------------+  |     |  +-----------------+  |
|  | Request plugins |---------> |  Send Plugins   |  |
|  +-----------------+  |     |  +-----------------+  |
|                       |     |           |           |
|  +---------------- +  |     |           |           |
|  | Import plugins  |<-------------------+           |
|  +-----------------+  |     |                       |
|                       |     |                       |
|  +-----------------+  |     |     +--------------+  |
|  |                 |  |     |     |   CLASSIFY   |  |
|  | REQUEST CATALOG +------------> |The Node based|  |
|  |  Send node name |  |     |     |on certname + |  |
|  |  and facts to   |  |     |     | node groups  |  |
|  |  the master     |  |     |     +--------------+  |
|  |                 |  |     |     ______|______     |
|  ------------------+  |     |    |      |      |    |
|                       |     | +--V--++--V--++--V--+ |
|                       |     | |CLASS||CLASS||CLASS| |
|  +---------------+    |     | +-----++-----++-----+ |
|  |               |    |     |      \    |     /     |
|  |    CATALOG    | <-----------+  +-V---V----V-+    |
|  |               |    |     |  |  |            |    |
|  +--------+------+    |     |  |  |  COMPILE   |    |
|           |           |     |  |  |            |    |
|  +--------V---------+ |     |  |  +------------+    |
|  |APPLY +--------+  | |     |  |        V           |
|  |      | QUERY  |  | |     |  |        |           |
|  |      | STATUS |  | |     |  +----<---+           |
|  |      +----|---+  | |     |                       |
|  |    +------V----+ | |     |                       |
|  |    |  ENFORCE  | | |     |                       |
|  |    |  DEFINED  | | |     |                       |
|  |    |  STATE    | | |     |                       |
|  |    +-----------+ | |     |    +-------------+    |
|  +----+-------------+ |     |    |             |    |
|           |           |     |    |   REPORT    |    |
|           |--------------------> |             |    |
|           |           |     |    +-------------+    |
|  +--------V---------+ |     |                       |
|  |  DEFINED SYSTEM  | |     |                       |
|  |      STATE       | |     |                       |
|  +------------------+ |     |                       |
+-----------------------+     +-----------------------+

Resources ---} Classes ---} Manifest

Manifests

A puppet manifest is text file that contains puppet code and is appended by the .pp file extension.

Catalog

In a standard agent + master puppet configuration, the agent never receives a copy of the modules, manifests, functions or variables. The agent only receives the compiled catalogue of resources and relationships.

It is not recommended by puppetlabs, however you can send each node an entire copy of the manifests for the nodes the compile the catalog and apply it locally, This may be desirable in situations where there is no direct network connectivity between the agent and master. Warning: each node will be able to see the entire manifest which may contain sensitive information relating to other nodes in your environment.

Modules

Modules are just directories and files with a predictable structure located in the module path. The following commands will display the module path.

# puppet agent --configprint modulepath
# puppet config print modulepath
/etc/puppet/modules:/usr/share/puppet/modules

Modules will often have a main class that shares the name of the module.

module-name
|-manifests/
|-files/
|-templates/
|-lib/
|-tests/
|-spec/

If a class is defined in a module you can then declare that class in any manifest by name.

Autoloading in Puppet means that your modules will be loaded by Puppet at compile time, as long as they follow a predictable structure.

puppet _\   compile  _\   load
 run    /   catalog   /  modules

Example module's

tree /etc/puppetlabs/puppet/environments/production/modules/sshd

/etc/puppetlabs/puppet/environments/production/modules/sshd
├── files
│   └── sshd_config
├── manifests
│   └── init.pp
└── tests
    └── init.pp

example module with class includes

cowsayings
├── manifests
│   ├── init.pp     ----->   class cowsayings {
│   │                          include cowsayings::cowsay
│   │                          include cowsayings::fortune
│   │                        }
│   │
│   │
│   ├── cowsay.pp   ----->   class cowsayings::cowsay {
│   │                          package { 'cowsay':
│   │                          ensure => 'present',
│   │                        }
│   │
│   │
│   └── fortune.pp  ----->   class cowsayings::fortune {
│                                package { 'fortune-mod':
│                                  ensure => 'present',
│                                }
│                            }
│
└── tests
    │
    ├── init.pp    -->   include cowsayings
    │
    │
    ├── cowsay.pp  -->   include cowsayings::cowsay
    │
    │
    └── fortune.pp -->   include cowsayings::fortune

puppet module command line

#puppet module <action>

ACTIONS:

build - Build a module release package.
changes - Show modified files of an installed module.
generate - Generate boilerplate for a new module.
install - Install a module from the Puppet Forge or a release archive.
list - List installed modules
search - Search the Puppet Forge for a module.
uninstall - Uninstall a puppet module.
upgrade - Upgrade a puppet module.

When uploading a module to the forge your must include a ** modulefile ** containing the modules metadata (name, version, source, author, description, etc.)

Resources

Each resource describes some aspect of a system and its state e.g....

a service that must be running
a package that must be installed
a user that must be configured

A resource declaration is the puppet block of code that describes a resource. Resource declarations are written the in the puppet DML Declarative Modelling Language.

Puppet's DML is a declarative language rather than an imperative one. This means that instead of defining a process or set of commands, Puppet code describes (or declares) only the desired end state, and relies on built-in providers to deal with implementation.

Resource declaration syntax example

type { 'title':
  attribute_1  =>  value_1,
  attribute_2  =>  value_2,
}

type and title must be unique for a node
values must be alphanumeric (quote strings)
each attribute + value pair must be followed by a comma

Individual resources are combined together to represent the desired system configuration.

Similar resources can be grouped into types. such as the user type

Resource Abstraction Layer

    +---------------------------------------+
    | file | package | service  |  user     |  Resources
    |---------------------------------------|
    | Ruby | Apt     | Redhat   | Useradd   |  Providers
    |      | Yum     | Launchd  | LDAP      |
    |      | Gems    | SMF      | Netinfo   |
    |      | Deb     | Debian   |           |
    |      | RPM     |          |           |
    +---------------------------------------+

Query a resource using the puppet command line tool

puppet resource <type> {display all the instances of the specified type}
puppet resource package
puppet resource host
puppet resource user 
puppet resource <type> <title> {display the details of a particular resources instance }
puppet resource package apache
puppet resource host example.com
puppet resource user seamus
puppet resource --type  {list all types}
puppet describe <type> {display a description of the type and its options}

Puppet Apply

You can use the Puppet resource declaration syntax with the puppet apply tool to make quick changes to resources on the system.

You can either change the values directly from the command line using the syntax..

puppet apply -e "user { 'seamus': ensure => 'present', }"

Or you can open the resource declaration in vim with the syntax...

puppet apply -e user seamus

The resource will be opened in vim. Make any desired changed and when you quit vim the new resource values will be applied.

Defined Resource Type (aka defined types or defines)

are types that can be evaluated multiple times using different parameters during declaration. Upon each new declaration with new parameters they act like a new resource type.

Seamus sample module using defined resources [root@master modules]# tree definedmod

definedmod
├── manifests
│   ├── definedresource.pp
│   └── init.pp
└── tests
    └── init.pp

[root@master modules]# cat definedmod/manifests/init.pp

class definedmod {
  definedresource {'mydefinedresource':
         var2 => 'twovar',
         var1 => 'variable1',
         var3 => 'thirdvariable',
  }
}

[root@master modules]# cat definedmod/manifests/definedresource.pp

define definedresource ($var1, $var2 = default2 , $var3 = default3) {

  file { "/tmp/${var1}" :
    ensure  => present,
    content => "this file name is not $var2 or $var3",
  }

  file { "/tmp/${var2}" :
    ensure  => present,
    content => "this file name is not $var1 or $var3",
  }

}

[root@master modules]# cat definedmod/tests/init.pp

include definedmod

[root@node4 ~]# cat /tmp/variable1 
this file name is not twovar or thirdvariable

[root@node4 ~]# cat /tmp/twovar 
this file name is not variable1 or thirdvariable

[root@node4 ~]# cat /tmp/thirdvariable 
this file name is not variable1 or twovar[root@node4 ~]#

Resource duplicates

Instances of type + title must be singleton/unique for each node, otherwise the nodes catalog will fail to compile.

sample duplicate resource declarations:

class resourceduplicates {
  user { 'user1': # duplicate title of the user resource below; 
                  # the nodes catalog will fail to compile.
    ensure => 'present',
    uid    => '1007',
  }

  user { 'user1':
    ensure => 'present',
    uid    => '1007', # duplicate UID's, puppet catalog will compile; 
  }                   # But the useradd provider will fail on the client.
}

sample error of duplicate resource:

Error: Could not retrieve catalogue from remote server: Error 400 on SERVER: Duplicate declaration: User[user2] is already declared in file

Resource refresh

Puppet will refresh a service for each instance of a refresh, e.g if 3 files are being updated and each update triggers a refresh of the same service 3 separate refreshes will be triggered doe each of the file updates.

# This is a test to see if puppet will restart a service multiple time or-
# just once when multiple refresh events are triggered
class loop {

   file { '/tmp/loop1' :
     ensure  => present,
     content => "$::uptime_seconds \n", # the "uptime_seconds" is used here to ensure that
   }                                    # the content of the file will change upon each run

   file { '/tmp/loop2' :
     ensure  => present,
     content => "$::uptime_seconds \n",
   }

   file { '/tmp/loop3' :
     ensure  => present,
     content => "$::uptime_seconds \n",
   }

   service { 'crond' :
     ensure => running,
   }

   File['/tmp/loop1'] ~> Service['crond']
   File['/tmp/loop2'] ~> Service['crond']
   File['/tmp/loop3'] ~> Service['crond']

}

Resource ordering and relationships with metaparameters

Via a mechanism called autorequires; For some of the built in resource types such as users and groups, Puppet is able to automatically determine the dependency relationships among some of the built-in resources such as user and groups without a user/admin having to declare the resource interdependencies.

For the majority of resources there is no autorequire and therefore the order of resources in a puppet manifest is generally ignored. If a group of resources have interdependencies you should use one of the metaparameters to specify the relationships between them

before - Causes a resource to be applied before the target resource
require - Causes a resource to be applied after the target resource
notify - Causes a resource to be applied before the target resource. The target resource will refresh if the notifying resource changes.
subscribe - Causes a resource to be applied after the target resource. The subscribing resource will refresh if the target resource changes.

Note: when creating a resource reference use the syntax

Type['title']

Note: Use lower-case for the type name when declaring it but use a capitalised type name when calling it.

metaparameter declaration examples...

before

  package { 'openssh-server':
    ensure => present,
    before => File['/etc/ssh/sshd_config'],
  }

require

  file { '/etc/ssh/sshd_config':
    ensure  => file,
    mode    => 600,
    source  => 'puppet:///modules/sshd/sshd_config',
    require => Package['openssh-server'],
  }

Ordering Arrows

 Package['openssh-server'] -> File['/etc/ssh/sshd_config']

Causes the resource on the left to be applied before the resource on the right.

notify

 file { '/etc/ssh/sshd_config':
   ensure  => file,
   mode    => 600,
   source  => 'puppet:///modules/sshd/sshd_config',
   notify  => Service['sshd'],
 }

 service { 'sshd':
   ensure    => running,
   enable    => true,
   subscribe => File['/etc/ssh/sshd_config'],
 }

Notification Arrow

 File['/etc/ntp.conf'] ~> Service['ntpd']

Causes the resource on the left to be applied first and sends a refresh to the resource on the right if the resource on the left changes.

Complete example

class sshd {

  package { 'openssh-server':
    ensure => present,
    before => File['/etc/ssh/sshd_config'],
  }

  file { '/etc/ssh/sshd_config':
    ensure => file,
    mode   => '0600',
    source => 'puppet:///modules/sshd/sshd_config',
  }
}

  service { 'ssh':
    ensure    => running,
    enable    => true,
    subscribe => File['/etc/ssh/sshd_config'],
}

extra metaparameters..

Manage dependencies (before, require, subscribe, notify, stage)

Manage resources' application policies (audit, noop, schedule, loglevel)

Add information to a resource (alias, tag)

Virtual resources

are a hack to get around puppets limitation of sigleton resource definitions. They enable an a particular instance of a resource type+title to be used in multiple classes.

They are defined as virtual by pre-pending them with the "@" symbol

@user { seamus: ensure => present }

They can then be declared in multiple classes by using one of the following 3 syntaxes using either the realise function or the spaceship collector.

User <| title == seamus |>
realize User[seamus]
realize(User[seamus])

Exported resources

Enable a resource that is defined on a host to be exported to the puppetdb and then be used by 1 or more other hosts. Puppet collects and stores the exported resources during configuration runs.

/etc/hosts example

[root@master modules]# cat exportedresource/manifests/init.pp

class exportedresource {
  @@host { $hostname:  ## sends the local nodes host resource to the puppetdb and 
                       ## tags it as exported
    ip           => $::ipaddress,
    name         => $::fqdn,
    host_aliases => $::hostname,
    comment      => 'addded by the exported resource class',
   }
  Host <<| |>>  ## retrieves all the host resources from the puppetdb
}

SSH example

class ssh::hostkeys {

  @@sshkey { "${::fqdn}_dsa":
    host_aliases => [ $::fqdn, $::hostname, $::ipaddress ],
    type         => dsa,
    key          => $::sshdsakey,
  }

  @@sshkey { "${::fqdn}_rsa":
    host_aliases => [ $::fqdn, $::hostname, $::ipaddress ],
    type         => rsa,
    key          => $::sshrsakey,
  }

}

SSH example of retrieving the exported resources and applying them

class ssh::knownhosts {
  Sshkey <<| |>> { ensure => present }
}

The ssh::knownhosts class should be included in the catalog for all nodes where Puppet should manage the SSH known_hosts file. Notice that we’ve used double angle braces to collect resources from PuppetDB.

define balancermember($url) {
  file { '/etc/httpd/conf.d.members/worker_${name}.conf':
    ensure  => file,
    owner   => 'root',
    group   => 'root',
    mode    => '0644',
    content => "BalancerMember $url \n",
  }
}

This configuration file fragment contains a single line, the URL to a member of the load balancer pool. Puppet recommends using a defined resource type because all resources declared within the type will be exported when the defined type itself is exported.

class loadbalancer_members {
   Balancermember <<| |>> { notify => Service['apache'] }
}

This code uses the double angle brace syntax to collect all balancermember resources from the stored configuration database. In addition, it uses a parameter block to notify the Apache service of any changes Puppet makes to the balancermember resources. Just as with virtual resources, a parameter block may be specified to add further parameters to collected resources.

Removing retired nodes from PuppetDB

# puppet node deactivate mail.example.com
Submitted 'deactivate node' for <node_name> with UUID aaaaa-bbbbb-ccccc-ddddd-eeee

After you’ve run this on the puppet master, any resources exported by this node will no longer be collected on your Puppet clients. Note: Deactivated node resources will not be removed from the systems that collected them. You will need to clean up those configurations manually; or some resources can be purged using the resource metatype.

Exec Resource

Exec resources execute external commands, it is important that any commands executed are Idempotent

Any command in an exec resource must be able to be run multiple times without casing harm. There are 3 main ways for commands in exec resources to be idempotent 1. The commands them selves can be idempotent 2. You use an (onlyif, unless, or creates) attribute, which will prevent puppet from running a specific command unless a specific condition is met. 3. The exec resource has the ( refreshonly=>true ) value, which only allows puppet to run the command when some other resource is changed

syntax

 exec { 'resource-title':
   attribute_1 => value_1,
   attribute_2 => value_2,
 }

example

exec { "updatedb':
  path    => '/usr/sbin',
  command => 'updatedb',
}

Note: if you do not set the command value it will default to the exec resource title. You must specifiy the path because the exec resource does not inherit paths.

Classes

Classes define a collection of resources that are managed together as a single unit. You can also think of them as named blocks of Puppet code, which are created in one place and invoked elsewhere.

Using a Puppet class requires two steps.

First, you'l need to define it by writing a class definition and saving it in a manifest file. When Puppet runs, it will parse this manifest and store your class definition.
Secondly, The class can then be declared to apply it to nodes in your infrastructure.

example class for ssh that contains 3 resources { package, file, service}..

resource     class

package  \
file      >-  ssh
service  /

Example class syntax....{with no relationships defined}

class ssh {
  package  { 'openssh-clients':
    ensure  => present,
  }

  file { '/etc/ssh/ssh_config':
    ensure  => file,
    owner   => 'root',
    group   => 'root',
    source  => 'puppet:///modules/ssh/ssh_config',
  }

  service { 'sshd':
    ensure  => running,
    enable  => true,
  }
}

Defining a class specifies the contents and behaviour but does not automatically include (apply) it in a configuration

Declaring a class, includes the class in the catalogue which will then be applied upon next agent run.

To declare a class use either of the following syntaxes in site.pp.. or in /tests/init.pp

** include my_calss ** or ** class { 'my_class': } **

classes are reusable
classes are singleton
classes can only be used once per node

When applying a class make sure you run

puppet apply module/tests/init.pp

and NOT

puppet apply module/manifest/init.pp

otherwise no actions will take place and no errors will be logged, this is because you would be defining the class but not declared it anywhere (on a node).

Parameterised Classes

Class parameters provide a method to set variables in a class as it's declared. The syntax for parameterised classes is similar to resource declarations.

class { 'ntp':
  servers =>
  ['node1.example.com','node2.example.com','node3.example.com']
}

The servers parameter can be populated with a single server or an array of servers.

Parameterised class definitions can be set in the site.pp file.

node default {
  # This is where you can declare classes for all nodes.
  # Example:
  #   class { 'my_class': }
  class { 'ntp':
    servers =>
    ['node1.example.com','node2.example.com','node3.example.com']
  }
}

You can also override values with

class { '::mysql::server':
  override_options => { 'mysqld' => { 'max_connections' => '1024' } },
}

Class Inheritance / Derived Classes

By using the inherits keyword classes can be derived from other classes.

When a derived class is declared, its base class is automatically declared first and the variable are set as the parent scope. The new class receives a copy of all the base class's variables and resource defaults. Code in the derived class is able to override any resource attributes that were originally set in the base class.

Class Inheritance is only useful for overriding resource attributes. For any other use case it is better archive your objective by using some other method.

example...

We create a new class zsh::developer that will enable us to deploy the zshrc.dev file instead of the zshrc file on certain nodes..

[root@learn modules]# tree zsh/
zsh/
├── files
│   ├── zshrc
│   └── zshrc.dev
├── manifests
│   ├── developer.pp
│   └── init.pp
├── Modulefile
├── README
├── spec
│   └── spec_helper.rb
└── tests
    ├── developer.pp
    └── init.pp

cat zsh/manifests/init.pp

class zsh {
  package { 'zsh':
    ensure => present,
    before => File['/etc/zshrc'],
  }
  file { '/etc/zshrc':
    ensure => file,
    owner  => 'root',
    group  => 'root',
    source => 'puppet:///modules/zsh/zshrc',
  }
}

[root@learn modules]# cat zsh/manifests/developer.pp

#Note there is no need for a "package { 'zsh':" resource definition here as it will be inherited from the zsh class
class zsh::developer inherits zsh {
  File ['/etc/zshrc'] {
    source => 'puppet:///modules/zsh/zshrc.dev',
    }
}

[root@learn modules]# cat zsh/tests/init.pp

include zsh

[root@learn modules]# cat zsh/tests/developer.pp

include zsh::developer

Variables and Class Parameters

Variables are prefixed with the $ sign and assigned with the = operator

$mystring = 'this is my string'
$mypath   = '/tmp/puppet/'

Once a variable it is defined it can be used anywhere in the manifest in place of an regular assigned value.

Unlike resource declarations, variable assignments are parse-order dependent. This means that you must assign a variable in your manifest before you can use it.
If you try to use a variable that has not been defined, the Puppet parser won't complain. Instead, Puppet will treat the variable as having the special undef value.
You can only assign a variable once within a single scope. Once it's assigned, the value cannot be changed. (therefore it should be called a constant but hey its not like the puppet people actually follow best practices or common conventions anywhere else so why would this be any different)

Variable interpolation enables a string that is stored as a variable to be inserted into another string. eg..

file { "${mypath}file1.txt":
  ...
}
file { "${mypath}file2.txt":
  ...
}

Note: A string that includes an interpolated variable must be wrapped in double quotation marks ("..."), rather than the single quotation marks that surround an ordinary string

Class Parameters provide a method of setting the variables within a class when the class is declared rather than when it is defined.

Once defined a Parameterised Class can be declared with a similar syntax to a resource declaration

When defining a class, include a list of parameters and optional default values between the class name and the opening curly brace. So a parameterised class is defined as below:

class classname ( $parameter = 'default' ) {
  ...
 }

Once defined, a parameterised class can be declared with a syntax similar to that of resource declarations, including key value pairs for each parameter you want to set.

class {'classname':
  parameter => 'value',
}

Seamus sample module using defined resource type class with paramaters (as described in the "defined resource section above"

[root@master modules]# tree definedmod

definedmod
├── manifests
│   ├── definedresource.pp
│   └── init.pp
└── tests
    └── init.pp

[root@master modules]# cat definedmod/manifests/init.pp

class definedmod {
  definedresource {'mydefinedresource':
         var2 => 'twovar',
         var1 => 'variable1',
         var3 => 'thirdvariable',
  }
}

[root@master modules]# cat definedmod/manifests/definedresource.pp

define definedresource ($var1, $var2 = default2 , $var3 = default3) {

  file { "/tmp/${var1}" :
    ensure  => present,
    content => "this file name is not $var2 or $var3",
  }

  file { "/tmp/${var2}" :
    ensure  => present,
    content => "this file name is not $var1 or $var3",
  }

}

[root@master modules]# cat definedmod/tests/init.pp

include definedmod

[root@node4 ~]# cat /tmp/variable1 
this file name is not twovar or thirdvariable

[root@node4 ~]# cat /tmp/twovar 
this file name is not variable1 or thirdvariable

[root@node4 ~]# cat /tmp/thirdvariable 
this file name is not variable1 or twovar[root@node4 ~]#

puppet-lint

A 3rd party tool used to display common syntax and style errors. Packaged as a ruby gem.

gem install puppet-lint

puppet-lint /etc/puppetlabs/puppet/modules/ntplint/manifests/init.pp

you can ignore certain syntax checks

on command line

puppet-lint  <path>/<file>.pp --no-80chars-check
puppet-lint  <path>/<file>.pp --no-ensure_first_param-check

~/.puppet-lint.rc

--no-80chars-check
--no-ensure_first_param-check

using a rake file to test multiple manifests in one go by including the following values in your rake file

require 'puppet-lint/tasks/puppet-lint'
PuppetLint.configuration.send("disable_<check>")

Idempotency

By default puppet modules describe the desired final state rather than detail a series of step to follow, this mean that no matter how many times they are run the same end state will occur.

Puppet Console

The puppet enterprise console is a web-based gui. It can help..

manage node requests to join the puppet deployment
assign puppet classes to nodes and groups
view reports and activity graphs
browse and compare resources on your nodes
view inventory data
manage console users and their access privileges

Event Inspector

The Event Inspector is part of the Puppet Enterprise (PE) Console. It is a reporting tool that provides data for investigating the current state of your infrastructure. Its focus is on correlating information and presenting it from multiple perspectives, in order to reveal common causes behind related events. Event inspector lets you accomplish two important tasks: monitoring a summary of your infrastructure’s activity and analyzing the details of important changes and failures. It displays events from three perspectives Classes Nodes and Resources

Resetting the admin user password

This must be performed from a shell session on the console server and executing a ruby script that was part of the Puppet Enterprise installer.

q_puppet_enterpriseconsole_auth_password=newpassword q_puppetagent_certname=$(puppet config print certname) /opt/puppet/bin/ruby update-superuser-password.rb

Environments

By default, all nodes are assigned to a default environment named production.

There are three ways to assign nodes to a different environment:

Via your ENC or node terminus
Via each agent node’s puppet.conf
Via the PE console to set the environment for each node group.

Assigning Environments Via an ENC

The interface to set the environment for a node will be different for each ENC. Some ENCs cannot manage environments.

When writing an ENC, simply ensure that the environment: key is set in the YAML output that the ENC returns. See the documentation on writing ENCs for details.

If the environment key isn’t set in the ENC’s YAML output, the Puppet master will just use the environment requested by the agent. Assigning Environments Via the Agent’s Config File

Assigning Environment via agent config

In puppet.conf on each agent node, you can set the environment setting in either the agent or main config section. When that node requests a catalog from the Puppet master, it will request that environment. Note: If you are using an ENC and it specifies an environment for that node, it will override whatever is in the config file.

Assigning Environments via PE console Click Classification - select node group - click "edit node group metadata" - select enviroment

Scope

Scope helps to organise classes, telling Puppet where to look within the module directory structure to find each class. It also separates namespaces within the module and your Puppet manifests, preventing conflicts between variables or classes with the same name.

The init.pp in the /manifest/ directory must contain a class with the same name as the module name

Class name = mysql File location = /modules/mysql/manifests/init.pp )

Class name = mysql::server File location = /modules/mysql/manifests/server.pp

Class name = mysql::server::account_security File location = /modules/mysql/manifests/server/account_security.pp

Types and Providers

A type defines the interface for a resource: the set of properties you can use to define a desired state for the resource, and the parameters that don't directly map to things on the system, but tell Puppet how to manage the resource. Both properties and parameters appear in the resource declaration syntax as attribute value pairs.

command to show the types available

[root@master ~]# puppet resource --type

+-----------------------------------------------------------------------------+
| anchor            |  mcx                        |   puppetdb_conn_validator |
| apt_key           |  mount                      |   resources               |
| augeas            |  nagios_command             |   router                  |
| computer          |  nagios_contact             |   schedule                |
| cron              |  nagios_contactgroup        |   scheduled_task          |
| exec              |  nagios_host                |   selboolean              |
| file              |  nagios_hostdependency      |   selmodule               |
| file_line         |  nagios_hostescalation      |   service                 |
| filebucket        |  nagios_hostextinfo         |   ssh_authorized_key      |
| firewall          |  nagios_hostgroup           |   sshkey                  |
| firewallchain     |  nagios_ser^ice             |   stage                   |
| group             |  nagios_ser|icedependency   |   tidy                    |
| host              |  nagios_ser|iceescalation   |   user                    |
| ini_setting       |  nagios_ser|iceextinfo      |   vlan                    |
| ini_subsetting    |  nagios_servicegroup        |   whit                    |
| interface         |  nagios_timeperiod          |   yumrepo                 |
| k5login           |  notify                     |   zfs                     |
| macauthorization  |  package                    |   zone                    |
| mailalias         |  postgresql_conf            |   zpool                   |
| maillist          |  postgresql_psql            |                           |
+-----------------------------------------------------------------------------+

Core Types

File

manages local files and ensures if a file should exist sets its parameters present / absent file / directory / link source puppet:///modules/

Package

Manages software packages ensures that a package is present / absent / latest /purged / version

Service

Manages services ensures running / stopped enable true / false

notify

echo's message to the agent run-time log message => " hello world ! ",

exec

executes an arbitrary command on the agent node.

cron

manages cron jobs

user

manages user accounts ensure present / absent / role name, uid, gid, groups, home, shell

group

ensure present / absent name, gid A provider is what does the heavy lifting to bring the system into line with the state defined by a resource declaration. Providers are implemented for a wide variety of supported operating systems. They are a key component of the Resource Abstraction Layer (RAL), translating the universal interface defined by the type into system-specific implementations.

Resource Abstraction Layer

   +---------------------------------------+
   | file | package | service  |  user     |  Resources
   |---------------------------------------|
   | Ruby | Apt     | Redhat   | Useradd   |  Providers
   |      | Yum     | Launchd  | LDAP      |
   |      | Gems    | SMF      | Netinfo   |
   |      | Deb     | Debian   |           |
   |      | RPM     |          |           |
   +---------------------------------------+

Conditional Statements

**Note: ** string matches are not case sensitive

if (optional elseif and else)

class accounts ($name) {

  if $::operatingsystem == 'centos' {
    $groups = 'wheel'
  }
  elsif $::operatingsystem == 'debian' {
    $groups = 'admin'
  }
  else {
    fail( "This module doesn't support ${::operatingsystem}." )
  }

  notice ( "Groups for user ${name} set to ${groups}" )

  ...

}

unless

unless $memorysize > 1024 {
  $maxclient = 500
 }

case

case $::operatingsystem {
  'CentOS': { $apache_pkg = 'httpd' }
  'Redhat': { $apache_pkg = 'httpd' }
  'Debian': { $apache_pkg = 'apache2' }
  'Ubuntu': { $apache_pkg = 'apache2' }
  default: { fail("Unrecognised operating system for webserver.") }
}

package { $apache_pkg :
  ensure => present,
}

selector

Selector statements are similar to case statements, but return a value instead of executing a code block. Selectors can only be used at places in the code where a plain value is expected an not inside another selector or case statement.

$rootgroup = $::osfamily ? {
  'Solaris'  => 'wheel',
  'Darwin'   => 'wheel',
  'FreeBSD'  => 'wheel',
  'default'  => 'root',
}

my sample class with all the selectors

[root@pe-puppet modules]# cat noticemod/manifests/init.pp

class noticemod {
  notify { "Hello World!": }

## SELECTOR statement
  notify { " SELECTOR warning !":
    message => $seamus ? {
      'mytrue' => " the value is true.",
      'myfalse' => " the value is false.",
      default => " the value is not set.",
      }, # selector statements warn you if no match is found and there is no default
  }

## CASE statement
  case $seamus {
    'mytrue':   { notify { " CASE warning ! the value is true.": } }
    'myfalse':  { notify { " CASE warning ! the value is false.": } }
    default: { fail(" CASE warning ! failing now... if the variable is undef the whole catalog compile will fail") }
  } # case statements silently fall through the bottom if no match is found


## IF statement 
  if 'rue' in $seamus {
      notify { " IF warning ! the value is true.": } 
  }
  elsif 'alse' in $seamus {
      notify { " IF  warning ! the value is false.": } 
  }
  else {
      notify { " IF  warning ! the value does not match.": }
  }


## UNLESS statement

  unless $seamus == 'myfalse' {
    notify { " UNLESS statement - \$seamus does not == myfalse, it is set to $seamus ": } 
  }

  unless $seamus == 'mytrue' {
    notify { " UNLESS statement - \$seamus does not == mytrue, it is set to $seamus ": } 
  }


}

PuppetDB

stores the most recent facts from every node
the most recent catalog from every node
(optionally) 7 days of event reports from every node

it is searchable using either...

PuppetDB's query API
Puppet's inventory service
exported resources

example

[root@learning ~]# puppet node status learning.puppetlabs.vm learning.puppetlabs.vm Currently active Last catalog: 2015-01-24T13:31:16.291Z Last facts: 2015-01-24T13:31:03.782Z

Declaring an exported resource causes that resource to be added to the catalog and marked with an “exported” flag, which prevents the puppet agent from managing the resource. When PuppetDB receives the catalog, it stores a record for each resource with the flag set.

Delete reports older then 1 month :

sudo -u puppet-dashboard rake RAILS_ENV=production reports:prune upto=1 unit=mon

Hiera

https://ask.puppetlabs.com/question/13592/when-to-use-hiera-hiera_array-and-hiera_hash/

Hiera is a key/value lookup tool for configuration data, built to make puppet better and let you set node-specific data without repeating yourself. Hiera support is built into puppet 3, and is available as an add-on for puppet 2.7.

It keeps site specific data out of the manifest. Puppet classes can query any data they need and hiera will act as a site wide config file.

Hiera makes it easier to: 1. separate configuration data from the modules code. 2. configure your own nodes: default data with multiple levels of overrides. 3. re-use public puppet modules: you don't have to edit the code you just have to put the necessary data in hiera. 4. makes it easier to publish your own developed modules to the forge without clashing variable names or having to redact your config values. 5. create common data for most nodes and.. - Override some values for machines located at a particular facility - Override some values for specific machines 6. enables you to only write down the differences that are needed (doesn't make sense)

To get started with hiera there are 5 steps.. 1) download and install hiera 2) create a hiera.yaml config file 3) arrange a hierachy that suite your site and data 4) write your data sources 5) configure puppet to use hiera

https://ask.puppetlabs.com/question/13592/when-to-use-hiera-hiera_array-and-hiera_hash/ hiera - Performs a standard priority lookup and returns the most specific value for a given key. The returned value can be data of any type (strings, arrays, or hashes).

hiera_array - Returns all matches throughout the hierarchy — not just the first match — as a flattened array of unique values. If any of the matched values are arrays, they’re flattened and included in the results.

hiera_hash - Returns a merged hash of matches from throughout the hierarchy. In cases where two or more hashes share keys, the hierarchy order determines which key/value pair will be used in the returned hash, with the pair in the highest priority data source winning.

example of using hiera....

[root@learn ~]# vim /etc/puppetlabs/puppet/hieradata/common.yaml

---
message: This string is the value that is returned when hiera('message') is called.
motd:  Hello there Seamus

test that the key is set correctly using the puppet apply -e

[root@learn hieradata]# puppet apply -e 'notice(hiera("motd"))'
Notice: Scope(Class[main]): Hello there Seamus
Notice: Compiled catalog for learn.puppetlabs.com in environment production in 0.04 seconds
Notice: Finished catalog run in 0.02 seconds

Edit the motd module to use the key from hiera instead of the value in the module files

[root@learn hieradata]# cd /etc/puppetlabs/puppet/modules/
[root@learn modules]# tree motd/
motd/
├── manifests
│   └── init.pp
├── Modulefile
├── README
├── spec
│   └── spec_helper.rb
└── tests
    └── init.pp

[root@learn modules]# vim motd/manifests/init.pp

  Class: motd

class motd {

  file { '/etc/motd':
    ensure  => file,
    owner   => 'root',
    group   => 'root',
    content =>  (hiera("motd")),
  }

}

[root@learn modules]# puppet parser validate motd/manifests/init.pp [root@learn modules]# puppet apply motd/tests/init.pp Notice: Compiled catalog for learn.puppetlabs.com in environment production in 0.09 seconds Notice: /Stage[main]/Motd/File[/etc/motd]/ensure: defined content as '{md5}13c0016553b5315a8cd66f13b23f14a3' Notice: Finished catalog run in 0.03 seconds

[root@learn modules]# cat /etc/motd

Hello there Seamus

Facter

Facter stores facts as pairs of keys and values.

Facter command line options

-y, --yaml                       Emit facts in YAML format.
-j, --json                       Emit facts in JSON format.
    --plaintext                  Emit facts in plaintext format.
    --trace                      Enable backtraces.
    --external-dir DIR           The directory to use for external facts.
    --no-external-dir            Turn off external facts.
-d, --debug                      Enable debugging.
-t, --timing                     Enable timing.
-p, --puppet                     Load the Puppet libraries, thus allowing Facter to load Puppet-specific facts.
-v, --version                    Print the version and exit.
-h, --help                       Print this help message.

A shell enviroment variable prepened with the name FACTER_ will be readable by facter as a fact. eg...

[root@node1 ~]# export FACTER_seamus=murray
[root@node1 ~]# facter seamus
murray
[root@node1 ~]# facter |grep seamus
seamus => murray

Facter values can be temporarily overridden by setting a shell variable prepended with FACTER_ and the name of the fact. eg...

FACTER_operatingsystem=Debian

You can even test the effect that the fake fact will have using puppet apply..

FACTER_operatingsystem=Debian puppet apply --noop accounts/tests/init.p

The best way to distribute facts is to include them in your modules using the puppet plug-ins. Puppet will distribute the custom facts, custom types, providers and functions to any host that includes the module.

you can either distribute the facts with the modules that need them or you can create a module combining all your custom facts needed by all your other modules.

Facter is called by the puppet agent. Facts apear as normal and top scope variables they can be called by

$ipaddress

or

${::ipaddress}  (best practice)

Pluginsync

Custom Facts and types can be exported to the the agent nodes from the puppet master using the plugin sync mechanism. Before the client requests a catlog it checks to see if the are any new plugins or if its plugins have been updated. If there has been a change it pulls down the plugins and executes the code in the custom types and facts.

This and be disabled from the agent nodes puppet.conf config file

[main]
pluginsync = false

Custom facts

you can create a custom fact with a bit of ruby code on the puppet master and use plugins and modules to distribute it to the clients/nodes

Create a ruby file in

<modulename>/lib/facter/<modulename>.rb

For simple shell commands, just insert the shell command in between the setcode do and the end. For more complex comands use Facter::Core::Execution.exec('') between the setcode do and the end.

Facter.add('mycustomfact') do
  setcode do
    Facter::Core::Execution.exec('/bin/date')
  end
end

on the agents use plugin sync to automatically pull down the facts to /var/lib/puppet/lib/facter/mycustomfact.rb

puppet agent -t

use the -p facter flag to test the puppet fact

facter -p  mycustomfact

A fact can be confined to run only if another fact is of a certain value

e.g.

Facter.add(:powerstates) do
  confine :kernel => 'Linux'
  setcode do
    Facter::Core::Execution.exec('cat /sys/power/states')
  end
end

External facts

Can be written in any language as long as they return key/value pairs to stdout.

They are stored in ..

<MODULEPATH>/<MODULE>/facts.d/

and are distributed to the clients via the pluginsync mechanism.

Example fact written in bash...

$ cat/etc/puppet/modules/myexternalfact/facts.d/myexternalfacts.sh
#!/bin/bash
echo "myexternalfact1=`/bin/df -P / |/usr/bin/tail -1 | /bin/awk '{print $5}'`"
echo "extfact1=one"
echo "extfact2=two"
echo "extfact3=three"

Calling the external fact from the command line on a node that has recieved the external fact by pluginsync...

$ facter --external-dir  /var/lib/puppet/facts.d/ extfact1 extfact2 extfact3 myexternalfact1
myexternalfact1 => 33%
extfact1 => one
extfact2 => two
extfact3 => three

Example fact written in c...

#include <stdio.h>

int main() {
   printf("my_external_fact_in_c=\"Hello World\"\n");
 return 0;
}

Calling the external fact written in c...

# facter --external-dir ~/puppet-study/external-facts/ my_external_fact_in_c
"Hello World"

Distributing the external facts to the puppet client nodes

It is best to put the external facts into a module so that they can be distributed to all the agent nodes via plugin sync..

[root@master myexternalfact]# tree  /etc/puppet/modules/myexternalfact
/etc/puppet/modules/myexternalfact
└── facts.d
    ├── myexternalfactinc
    └── myexternalfact.sh


[root@master ~]# ls -l /etc/puppet/modules/myexternalfact/facts.d/
total 16
-rwxr-xr-x. 1 root root 8523 Feb 10 00:27 myexternalfactinc
-rwxr-xr-x. 1 root root  235 Feb 10 00:36 myexternalfact.sh

Trigger the plugin sync to trasfer the new external facts..

[root@node1 ~]# puppet agent -t

Use the puppet apply -e tool on the agent nodes to test the facter values..

[root@node1 ~]# puppet apply -e ' notice( "${::my_external_fact_in_c}" ) '
Notice: Scope(Class[main]): "Hello World"
Notice: Compiled catalog for node1.puppetlabs in environment production in 0.02 seconds
Notice: Finished catalog run in 0.03 seconds

[root@node1 ~]# puppet apply -e ' notice( "${::my_external_fact_in_bash_1}" ) '
Notice: Scope(Class[main]): 32%
Notice: Compiled catalog for node1.puppetlabs in environment production in 0.02 seconds
Notice: Finished catalog run in 0.03 seconds

[root@node1 ~]# puppet apply -e ' notice( "${::my_external_fact_in_bash_2}" ) '
Notice: Scope(Class[main]): two
Notice: Compiled catalog for node1.puppetlabs in environment production in 0.02 seconds
Notice: Finished catalog run in 0.03 seconds

Structured facts

Can come straight from YAML,JSON or txt files in the directories..

/etc/facter/facts.d/ or /etc/puppetlabs/facter/facts.d/

$ cat /etc/facter/facts.d/mystructuredfact.yaml
---
mystructuredfact1: factone
mystructuredfact2: facttwo
mystructuredfact3: factthree

$ facter mystructuredfact1 mystructuredfact2 mystructuredfact3
mystructuredfact1 => factone
mystructuredfact2 => facttwo
mystructuredfact3 => factthree

Roles and Profiles

Roles and profiles are just modules written in a specific way.

roles are assigned to nodes
profiles are assigned to roles

example role

class role::weibserver inherits role {
  include profile:: web
}

class role::dbserver inherits role {
  include profile::db
}

class role::webdbserver inherits role {
  include profile:web
  include profile::db
}

classification = assignment of classes to nodes classification = assignment of roles to nodes

example profile

class profile::web {
  include apache
  include php
  include tomcat
  include jdk
  include memcache
}

__

Tip: classes are singletons, they can be included multiple times on a node, but they will only be evaluated once. A defined class can be declared multiple times, because it takes parameters and each new declaration will be evaluated.

You can see we’ve included our two classes but not the definition, apache::vhost. This is because of some module magic called autoloading. Puppet scans your module and loads any .pp file in the manifests directory that is named after the class it contains; for example, the install.pp file contains the apache::install class and so is autoloaded.

The same thing happens with definitions: The vhost.pp file contains the definition apache::vhost, and Puppet autoloads it. However, as we declare definitions, for example calling apache::vhost where we need it, we don’t need to do an include apache::vhost because calling it implies inclusion.

...............................................

class puppet::params {
  $puppetserver = hiera('puppetserver')
}

In Puppet 3, but not Puppet 2.7, there is an automatic lookup of parameters in a parameterized class. We can use this to rewrite the puppet::params class further:

class puppet::params (
  $puppetserver,
){
}

When this is called with no arguments, Puppet 3 will attempt to look up the puppet::params::puppetserver key in Hiera and populate it accordingly. It will not fail unless the Hiera lookup fails.

Puppet Templates

Puppet supports templates written in the ERB templating language, which is part of the Ruby standard library. Templates are always evaluated by the parser, not by the client. This means that if you are using a puppet master server, then the templates only need to be on the server, and you never need to download them to the client. The client sees no difference between using a template and specifying all of the text of the file as a string. Note that the template function simply returns a string, which can be used as a value anywhere — the most common use is to fill file contents, but templates can also provide values for variables:

ags

The tags available in an ERB file depend on the way the ERB processor is configured. Puppet always uses the same configuration for its templates (see “trim mode” below), which makes the following tags available:

** <%= Ruby expression %> ** — This tag will be replaced with the value of the expression it contains.
** <% Ruby code %> ** — This tag will execute the code it contains, but will not be replaced by a value. Useful for conditional or looping logic, setting variables, and manipulating data before printing it.
** <%# comment %> ** — Anything in this tag will be suppressed in the final output.
** <%% or %%> ** — A literal <% or %>, respectively.
** <%- — Same as <% ** , but suppresses any leading whitespace in the final output. Useful when indenting blocks of code for readability.
** -%> ** — Same as ** %> ** , but suppresses the subsequent line break in the final output. Useful with many lines of non-printing code in a row, which would otherwise appear as a long stretch of blank lines.

Puppet parser

Is the part of puppet that pareses the puppet DSL code in manifests files. It can be called from the command line to validate your puppet code

[root@pe-puppet]# puppet parser validate /etc/puppetlabs/puppet/modules/myexternalfact/manifests/init.pp 
  Error: Could not parse for environment production: Syntax error at '{'; expected '}' at /etc/puppetlabs/puppet/modules/myexternalfact/manifests/init.pp:3

Functions

Functions are extensions of the Puppet Parser. They are located and are executed on the master and only have access to resources and data that is available on the master. During the parsing of the manifest's any function code is executed and the subsequent return value/s are inserted into the resulting compilation.

There are two types of functions: statements and rvalues.

Statements, such as the fail function, which stops the Puppet run with a parser error, perform some action.
Rvalues return values and can be used anywhere a normal value is expected. (This includes resource attributes, variable assignments, conditions, selector values, the arguments of other functions, etc.) These values can come from a variety of places;
- template function reads and evaluates a template to return a string.
- stdlib’s str2bool and num2bool functions convert values from one data type to another.
- split function parses a string and returns array elements.

RBAC Role Based Access Control

Puppet enterprise ships with 3 default User Roles

Administrators
Operators
Viewers

Todo.................................

Seamus learn the +> operator

Does site.pp override the values in an enc?

Does the values in an enc override the values in a class?

Yumrepo['custom_packages'] -> Package <| tag == 'custom' |>

Will create an order relationship with several package resources?

Using puppet node to set password when the nodes cant contact the password server, do you use an exec resource ?

Do you have to configure the agents to use heira or just the master?

What is a node_terminus where is it set and what overrides it

https://docs.puppetlabs.com/guides/external_nodes.html

Learn the spaceship operator

The puppet db has multiple terminus's where data is stored

Facts
Catalogs
Resources

Watch the puppetdb talk by james sweanie puppetny https://www.youtube.com/watch?v=HTr4b02aU7A at 14 mins

Do the values set in the console override enc or site.pp (where are the values stored) What is the effect of node default {} in site.pp can this be overridden by ENC or by a specific node hostname {} in site.pp

https://docs.puppetlabs.com/guides/external_nodes.html

Workout resource interdependency loops

............

sources of information I used to study

1) {book} pro puppet second edition

Last update: (made some formatting changes on 13 Feb 2024)

PuppetLabs own documentation https://puppetlabs.com/learn

Puppet installation

System requirements

Firewall Configuration

Hardware requirements for the various puppet roles/components

Install script

Install the pe-agent

puppet.conf

SSL certificates and accurate time

Resetting a nodes certificate

Puppet Cert command line options

Node Classification site.pp ENC

Certname

Classification of nodes

site.pp syntax

The Default Node

Merging site.pp With ENC Data

node classification in puppet enterprise console

default node group

Deleting Nodes from puppet enterprise console

Hide Node

Delete Node

Deactivate Node

Regex in puppet

Puppet run overview

Resources ---} Classes ---} Manifest

Manifests

Catalog

Modules

Example module's

puppet module command line

Resources

Resource Abstraction Layer

Query a resource using the puppet command line tool

Puppet Apply

Defined Resource Type (aka defined types or defines)

Resource duplicates

Resource refresh

Resource ordering and relationships with metaparameters

before

require

Ordering Arrows

notify

subscribe

Notification Arrow

extra metaparameters..

Virtual resources

Exported resources

Exec Resource

Classes

Parameterised Classes

Class Inheritance / Derived Classes

Variables and Class Parameters

puppet-lint

Idempotency

Puppet Console

Event Inspector

Resetting the admin user password

Environments

Scope

Types and Providers

Core Types

File

Package

Service

notify

exec

cron

user

group

Resource Abstraction Layer

Conditional Statements

if (optional elseif and else)

unless

case

selector

my sample class with all the selectors

PuppetDB

Hiera

Facter