When working with Puppet and Puppet modules in particular you quickly notice that there are several ways to manage module dependencies. Today I learned about a new way introduced in Puppet Bolt 3.x, so I thought I’d compile a list that celebrates this diversity:

• metadata.json: used primarily to dependency metadata to Puppet Forge. Some tools like librarian-puppet and puppet-litmus use / can use this file to download dependency modules. The downside is that you can’t define the location of the modules, so you’re limited to Forge modules.

Puppet Litmus is a Puppet acceptance test harness that leverages on many existing, proven technologies. In the long run Litmus aims to replace Beaker. However, the latter is currently still the only way to run multi-machine (e.g. cluster) acceptance tests in a standardized way. For example we use Beaker for our Keycloak domain mode cluster tests in puppet-module-keycloak. On a high-level the test process is the same as with Beaker:

• Provision the SUT (“System Under Test”)

Puppet Litmus allows creating SUTs (“System Under Test”) with Vagrant using Virtualbox or Hyper-V as the virtualization backend. However, the default virtual machine settings like 1024MB of memory are only suitable for very simple modules and applications. For example, Litmus testing our puppetmaster-installer will fail with the default settings.

Litmus documentation tells us that you can define parameters in provision.yaml which get then converted to environment variables. What it does not tell what to use these environment variables for. Looking at the Vagrant code in Litmus you can see that those environment variables can be used to drive the Virtualbox settings, among other things:

Some Puppet modules like puppet-module-keycloak have hundreds of unit tests. That is good for test coverage, but waiting for test results hurts your productivity when you’re developing tests for your new code. There are at least two ways to (temporarily) limit the scope of the tests that you run. First method is baked in into the Puppet Development Kit command-line:

$ pdk test unit --tests=spec/classes/init_spec.rb

This runs only the tests in the specified file.

I recently had to add access control rules to OpenVPN Access Server to permit certain groups to access certain IP addresses. As we manage our Access Server configurations with Puppet using puppet-openvpnas (by the author), we needed a way to parameterize the allowed IP addresses while keeping the Puppet code readable.

This is where puppet-dnsquery came really handy. It allows using data that’s already available in DNS and using it in Puppet code. A simplistic example:

The main annoyance with agent-based configuration management is the management of the agents themselves. They usually don’t require any handholding after the initial install, except when you need to migrate them to a new environment. For example, migrate from a Puppet 5-based server to Puppet 6.

The migration process is not overly difficult, but in order to scale to more than a handful of nodes you will need to orchestrate the process. Historically we would have used Ansible for this task, but nowadays we prefer Puppet Bolt, which is more powerful and flexible in our use-cases. The Bolt plans typically end up being ugly, but based on my experience with Ansible Playbooks that tends to be the case anyways. I guess this is the natural result of mixing state-based configuration management, ad hoc commands and orchestration together.

Writing static facts is fairly easy even with a low level of Ruby skills. Below is an example of a fact that returns true or false depending on whether the node has a /boot partition:

Facter.add(:has_bootfs) do
  setcode do
    if Facter.value(:mountpoints)['/boot'].nil?
      false
    else
      true
    end
  end
end

As you can see above the required information was already inside the “mountpoints” fact of Facter.

Using the fact is easy:

Separating data from code in Puppet modules is advisable as it improves reusability of code. The separation can be accomplished with Hiera by having separate levels based on facts, organizational units, locations, etc.

Hiera can also be used for storing private data that needs to be protected and must not be readable by outsiders. Typically the data, like all other code, is stored in a version control system that may be located outside the organisation. In such a case Hiera encryption is a must.

If the workflow that includes the control repository, r10k and GitLab still feels too light, there are yet more ways to make the workflow heavier by adding more abstraction to the Puppet modules themselves. This is done by adopting the “Roles and Profiles pattern” that was developed by Craig Dunn and popularized in the blog posts by Gary Larizza.

Roles and profiles are just Puppet classes. Profiles are usually used for creating technology stacks. A good example is a web application whose installation requires a web server (nginx, apache), a database server (postgresql, mysql) and several other components. The configurations for such an application could of course be done with a “regular” Puppet module but the reusability of the module would be rather limited for two reasons:

In the previous post I discussed the control repository structure. In this post I’ll talk about r10k. That tool is used for deploying control repository branches into matching Puppet environments on the Puppet server. Installing r10k is very straightforward with the bundled gem:

$ /opt/puppetlabs/puppet/bin/gem install r10k

After this you’ll need a settings file for r10k, /etc/puppetlabs/r10k/r10k.yaml:

--- 
 :cachedir: /opt/puppetlabs/puppet/cache/r10k 
 :sources: 
 control: 
 basedir: /etc/puppetlabs/code/environments 
 prefix: false 
 remote: git@gitlab.com:myorg/control-repo.git

Several requirements must be met before the configuration above will work: