Separating data from code in Puppet modules is advisable as it improves reusability of code. The separation can be accomplished with Hiera by having separate levels based on facts, organizational units, locations, etc.

Hiera can also be used for storing private data that needs to be protected and must not be readable by outsiders. Typically the data, like all other code, is stored in a version control system that may be located outside the organisation. In such a case Hiera encryption is a must.

If the workflow that includes the control repository, r10k and GitLab still feels too light, there are yet more ways to make the workflow heavier by adding more abstraction to the Puppet modules themselves. This is done by adopting the “Roles and Profiles pattern” that was developed by Craig Dunn and popularized in the blog posts by Gary Larizza.

Roles and profiles are just Puppet classes. Profiles are usually used for creating technology stacks. A good example is a web application whose installation requires a web server (nginx, apache), a database server (postgresql, mysql) and several other components. The configurations for such an application could of course be done with a “regular” Puppet module but the reusability of the module would be rather limited for two reasons:

In the previous post I discussed the control repository structure. In this post I’ll talk about r10k. That tool is used for deploying control repository branches into matching Puppet environments on the Puppet server. Installing r10k is very straightforward with the bundled gem:

$ /opt/puppetlabs/puppet/bin/gem install r10k

After this you’ll need a settings file for r10k, /etc/puppetlabs/r10k/r10k.yaml:

--- 
 :cachedir: /opt/puppetlabs/puppet/cache/r10k 
 :sources: 
 control: 
 basedir: /etc/puppetlabs/code/environments 
 prefix: false 
 remote: git@gitlab.com:myorg/control-repo.git

Several requirements must be met before the configuration above will work:

In the previous posts of this series I discussed the control repository and r10k. The last component in fattening the workflow is Git provider such as GitLab, GitHub or Bitbucket. Nowadays all of them are reasonable choices for storing private Git modules, which Puppet control repositories and site-specific modules tend to be. One of the biggest advantages of GitLab over its competition is that it is open source and can be installed locally.

If editing Puppet code and Hiera directly with puppetmaster feels too easy, you can complicate the Puppet workflow as much you like by adding more components to the palette. Adding the so called control repository and r10k to Puppet environment maintenance instantly adds several phases to your initially simple workflow. There are also benefits that I’ll discuss later on.

The control repository (control repo for short) is a Git repository that is used to create dynamic Puppet environments on the Puppet master. Each Git branch can be turned into a matching Puppet environment. The name of the default branch, production, is the same as the environment the Puppet agents use by default. The content of the control repo branches look like this:

NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead.

Puppet Bolt is a designed to be an orchestration tool, but it can be used for configuration management as well. For example you may have a small environment of handful of nodes where having a dedicated Puppet server would be an overkill, or you have some nodes in the infrastructure where you can’t or don’t want to install a Puppet Agent.

Packer is an excellent tool for automating creation of machine images such as AWS AMIs or Docker container images. This is particularly useful when you need to scale up by spinning up instances on-demand or if you want to package software that is challenging to manage with a configuration management system. It is also extremely useful when creating actual packaged product for AWS Marketplace, for example.

In Packer you take a “base box” such as Ubuntu 18.04 base image and modify its contents with provisioning, for example to install and configure software on the image. Several different types of provisioners are supported, including but not limited to shell scripts, Puppet manifests and Ansible playbooks. The concept of provisioners is the same as in Vagrant and Terraform.

NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead.

Puppet Bolt is an orchestration tool similar to Ansible. It is suitable for multiple use-cases:

1. Running ad hoc commands on target nodes. We use this approach for testing Puppet feature branches before approving and merging them.

2. One-off configurations written in the Puppet language

3. Masterless Puppet environments (look here). You may not always have enough privileges, trust or will to manage a complete Puppet infrastructure including Puppetserver, PuppetDB and Puppet Agents.

As discussed in the introductory post one of the use-cases for Puppet Bolt is orchestration - running actions on multiple targets in a certain order, possibly using results of actions or data from some targets on other targets. Orchestration with Bolt plans is quite easy: you just create a plan with multiple TargetSpec parameters, each representing a target or a type of target. A classic example is two or more web servers behind a load balancer. In this case we do something even simpler and just take a fact from a node called “first” and create a file on node “second” that uses the value of that fact in a file.

It seems every other organization is using Jenkins these days. Jenkins is a continuous integration and continuous delivery server that can be used to automate building, testing, and delivering or deploying software. 

Many organizations also use Puppet for their configuration management needs. Puppet is, if not the de facto configuration management solution, at least one of the most mature and field tested ones. It consists of several server components, agents, a declarative programming language, amongst others. Puppet can be used to install, set up and manage large number of all kinds of resources.