Puppet – How Puppet Modules makes use of Hiera

Modules making use of simple string data that has been retrieved from Hiera

There are 2 syntax styles that you can use to declare a class, include-like, and resource-like. Hence we have:

include user_account

# or 

class {'user_account':}

During a puppet run, whenever a puppet master encounters a resource declaration for a class, then the puppetmaster will do the following:

  1. checks to see if the class has any class parameters.
  2. If there are class parameters, If resource-like declaration is used, then check if any class parameters have been defined.
  3. if no luck, then it requests hiera to find the information
  4. If hiera returns nil, then it will resort to using the class parameter’s default values, if any has been defined in the class definition
  5. If no defaults have been specified then the puppet master fails and returns an error

Let’s assume for now that we haven’t setup/configured hiera yet. Also let’s say that in our site.pp we have:

node 'puppetmaster' {
  include user_account
}

It’s impossible to tell from this whether we need to specify any class parameters for the user_account class, and if so, whether there are any default values pre-defined as part of the class definition.

Hence we need to take a look at the class definition, which is in the module’s init.pp file:

[root@puppetmaster /]# tree /etc/puppet/modules/user_account/
/etc/puppet/modules/user_account/
├── files
├── lib
├── manifests
│     └── init.pp
├── spec
└── templates

5 directories, 1 file

[root@puppetmaster /]# cat /etc/puppet/modules/user_account/manifests/init.pp
class user_account ($username) {

  user { $username:
    ensure => present,
    shell  => '/bin/bash',
    home   => "/home/$username",
  }

  file {"/tmp/$username.txt":
    ensure => file,
    content => "This machine is called $::hostname \n",     #notice, that we used a facter value here.
  }

}

Here we can see that, we do have a class parameter, “$username”, which needs to be defined. However the class doesn’t specify a default value for this class parameter. That means that if we do a puppet run as is, then the puppet master won’t be able to resolve the class variable and will then fail, like this:

[root@puppetmaster /]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Error: Could not retrieve catalog from remote server: 
Error 400 on SERVER: Must pass username to Class[User_account] at /etc/puppet/manifests/site.pp:2 on node puppetmaster.codingbee.dyndns.org
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
[root@puppetmaster /]#

To fix this, we can define the class parameter in the site.pp file, by using the resource-like class declaration syntax. However that will end up adding a lot more lines of code in our site.pp file. Hence we want to continue to use the “include” syntax instead of the resource-like class syntax.

Otherwise the puppetmaster by default will automatically ask Hiera for this information, aka automatic parameter lookup. If this also fails, then the puppetmaster resorts to using the default parameter values, if any have been defined in the class definition. If not, then the Puppetmaster throws an error.

When puppetmaster submits a request to hiera, it provides the parameter’s key in the form a of an fqdn. This key hence takes the form of:

{class-name}::{parameter-name}

Note, if the class parameter is in a sub-class, then we do:

{module-name}::{class-name}::{parameter-name}

The puppetmaster also always tell’s hiera the path to the hiera.yaml file it should use, in the form of hiera_config puppet.conf variable, which by default is set to /etc/puppet/hiera.yaml. It does this via hiera’s “–config” option:

Usage: hiera [options] key [default value] [variable='text'...]

The default value will be used if no value is found for the key. Scope variables
will be interpolated into %{variable} placeholders in the hierarchy and in
returned values.

    -V, --version                    Version information
    -d, --debug                      Show debugging information
    -a, --array                      Return all values as an array
    -h, --hash                       Return all values as a hash
    -c, --config CONFIG              Configuration file
    -j, --json SCOPE                 JSON format file to load scope from
    -y, --yaml SCOPE                 YAML format file to load scope from
    -m, --mcollective IDENTITY       Use facts from a node (via mcollective) as scope
    -i, --inventory_service IDENTITY Use facts from a node (via Puppet's inventory service) as scope

Therefore, when the puppetmaster automatically submit’s a lookup request to hiera (as in the case above), then behind the scenes puppet runs the following hiera command:


hiera user_account::username --config /etc/puppet/hiera.yaml

This means our yaml file needs to contain a key called “user_account::username”. For Hiera, the double colons are treated like any other literal characters, and hence doesn’t have any special meaning to hiera. So let’s say, our hiera.yaml file points to just one yaml file, which is /etc/hieradata/yaml/global.yaml, and this file contains:

---
user_account::username: homer 

In that case we would end up with:

[root@puppetmaster /]# hiera user_account::username --config /etc/puppet/hiera.yaml
homer

So if our site.pp file contains:

[root@puppetmaster puppet]# cat manifests/site.pp
node 'puppetmaster' {
 class {user_account:}
}

Note here that we didn’t specify any parameter values, so this will prompt puppetmaster to try a hiera lookup. The same would have happened if we used the include-syntax.

Then we would get the following during an agent run:

[root@puppetmaster puppet]# cat /etc/passwd | grep "homer"
[root@puppetmaster puppet]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for puppetmaster.codingbee.dyndns.org
Info: Applying configuration version '1421705732'
Notice: /Stage[main]/User_account/User[homer]/ensure: created
Notice: Finished catalog run in 0.34 seconds
[root@puppetmaster puppet]# cat /etc/passwd | grep "homer"
homer:x:501:504::/home/homer:/bin/bash
[root@puppetmaster puppet]#

Success!

Modules making use of arrays that has been retrieved from Hiera

Earlier we saw that yaml files can be used to store mapped sequences (aka arrays). In most programming languages, arrays are iterated through loops. However in puppet there is no such thing as “loops” in manifest. However you can pass an array straight into a resource’s title.

For example, lets say our class shows:

[root@puppetmaster modules]# tree user_account/
user_account/
├── files
├── lib
├── manifests
│     └── init.pp
├── spec
└── templates
    └── list-of-fruits.erb

5 directories, 2 files
[root@puppetmaster modules]# cat user_account/manifests/init.pp
class user_account ($username) {

  user { $username:
    ensure => present,
    shell  => '/bin/bash',
  }

}

[root@puppetmaster modules]# cat /etc/puppet/manifests/site.pp
node 'puppetmaster' {
 class {user_account:}
}

So far nothing special. However the hiera.yaml file points to the following (and only) yaml file:

[root@puppetmaster modules]# cat /etc/hieradata/yaml/global.yaml
---

user_account::username:
  - homer
  - marge
  - bart
  - lisa
  - maggie

fruits:
  - apple
  - banana
  - carrot
[root@puppetmaster modules]#

As you can see, the user_account::username key is actually storing a array. Now if we run this, we get:

[root@puppetmaster user_account]# cat /etc/passwd | grep -E 'homer|marge|bart|lisa|maggie'
[root@puppetmaster user_account]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for puppetmaster.codingbee.dyndns.org
Info: Applying configuration version '1421710886'
Notice: /Stage[main]/User_account/User[marge]/ensure: created
Notice: /Stage[main]/User_account/User[lisa]/ensure: created
Notice: /Stage[main]/User_account/User[homer]/ensure: created
Notice: /Stage[main]/User_account/User[maggie]/ensure: created
Notice: /Stage[main]/User_account/User[bart]/ensure: created
Notice: Finished catalog run in 0.43 seconds
[root@puppetmaster user_account]# cat /etc/passwd | grep -E 'homer|marge|bart|lisa|maggie'
marge:x:503:506::/home/marge:/bin/bash
lisa:x:504:504::/home/lisa:/bin/bash
homer:x:505:507::/home/homer:/bin/bash
maggie:x:506:508::/home/maggie:/bin/bash
bart:x:507:509::/home/bart:/bin/bash
[root@puppetmaster user_account]#

That is how arrays are used in manifests. In the above example we created multiple user account, but you can also do the same think with any resource types, e.g. package, service, file….etc.

See also:

Puppet configuration variables and Hiera.

This makes reference to:

class myapplication ( $webname = hiera(“webname”) ) {

}

Which is a handy way to force a class to look up the data from hiera. In my case, I should use:

$username = hiera(“user_account::username”)