Puppet on Puppeteers Oy

Convert a hash into JSON in Puppet

Thu, 22 Dec 2022 00:00:00 +0000

This article shows you how to convert a hash into JSON in Puppet using a simple ERB template that gets its data from Hiera.

Suppose you have this data in Hiera:

myhash:
  mykey:
    foo: one
    bar: two

Converting a hash into a JSON file on the target node is surprisingly easy. First look up the data:

$myhash = lookup('myhash', Hash)

Then create a simple ERB template (here: mymodule/templates/myhash.erb)

Using .sync.yml in Puppet Development Kit (PDK)

Tue, 09 Aug 2022 00:00:00 +0000

Puppet Development Kit is probably the best thing since sliced bread if you work a lot with Puppet. It makes adding basic validation and unit tests trivial with help from rspec-puppet. It also makes it very easy to build module packages for the Puppet Forge.

That said, there is a minor annoyance with it: whenever you run “pdk update” to update PDK to the latest version, all your local changes to files such as .gitignore get destroyed, because PDK creates those files from PDK templates and any local changes are mercilessly wiped. Fortunately there is a way to sync your local changes to the files generated from the templates with .sync.yml. While .sync.yml is is pretty well documented, documentation is lots and lots of words with very few examples. So, here’s a fairly trivial .sync.yml example that ensures local changes to .pdkignore and .gitignore persist across PDK updates:

Use content of a file as a variable with Puppet

Sat, 16 Jul 2022 00:00:00 +0000

Every now and then a need to use the content of a file as a variable on an agent node arises. Here’s one way to do it with the help of a custom fact.

First create a custom fact on the puppet server:

Facter.add(:my_file_content) do
  setcode do
    path = "/path/to/my_file"
    File.read(path) if File.exist?(path)
   end
end

You can confine this to restrict it to be available only on selected nodes, for example:

Debugging Puppet Bolt inventory plugins

Mon, 14 Mar 2022 00:00:00 +0000

Puppet Bolt handles inventories in a very flexible and powerful manner: you can combine static target definitions and different targets into a single inventory. For example, you can have an inventory which defines some static node names combined with the AWS inventory, or one that combines static nodes with the Vagrant inventory.

Puppet Bolt inventory plugins do their magic and then output what is essentially a static inventory generated dynamically. When dynamic inventory plugins do not work (as in my case) it can be challenging to figure out what is wrong, let alone how to fix it. However, there’s is a very handy way to print out a static version of the entire inventory, including the dynamically created content:

Hiera lookups in rspec-puppet

Wed, 09 Feb 2022 00:00:00 +0000

While rspec-puppet documentation is quite decent, it does not really explain how to test classes that get their parameters via Hiera lookups, such as profiles in the roles and profiles pattern. Several parameters related to Hiera are listed in the rspec-puppet configuration reference, but that’s all. The other documentation you find on the Internet is generally outdated and not applicable for modules converted with PDK.

In this article we show how to test an “ipa_client” class that uses lookups to get its parameter from Hiera. It used to be a profile that was later separated from the control-repo into a separate module, which means that rspec-puppet does not have access to the control-repo’s Hiera data.

Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt

Tue, 28 Dec 2021 00:00:00 +0000

The traditional way of managing systems with Puppet is to install Puppet agent on the nodes being managed and point those agents to a Puppet server (more details here). This approach works well for environments with tens or hundreds of nodes, but is an overkill for small environments with just a handful of nodes. Fortunately nowadays Puppet Bolt enables you to push configurations to the managed nodes via SSH and WinRM. When using Bolt the catalog (desired state) is created on the computer running Puppet Bolt using facts from managed nodes and then sent to the managed nodes to be applied.

Creating Puppet Bolt groups based on AWS tags

Thu, 01 Jul 2021 00:00:00 +0000

The Using tags in Puppet Bolt aws_inventory target_mapping showed how to use AWS “Name” tag as the target name for Puppet Bolt. Use of tags can be extended to creating Bolt target groups:

groups:
  - name: webservers
    targets:
     - _plugin: aws_inventory
       region:
         _plugin: env_var
         var: AWS_DEFAULT_REGION
       filters:
         - name: tag:Type
           values: [webserver]
       target_mapping:
         name: tags.Name
         uri: public_ip_address

All you need to do is add a “filter” section with one filter. The “name” parameter tells Bolt that the filter is based on tags and what tag to look for. The “Type” that follows “tag” is the name of the tag you want to filter by. The “values” array lists acceptable values for that tag. Any nodes that don’t have the tag (here: “Type”) or where tag value is not acceptable (e.g. “db”) will not be shown in the group.

Testing Puppet feature environments with Puppet Bolt

Fri, 18 Jun 2021 00:00:00 +0000

Puppet feature environments are an excellent way to test code before deploying it, typically to production. They allow testing Puppet runs on no-operation mode across the whole node population managed by Puppet. There are sometimes cases where your code changes potentially impact many nodes and you’re not exactly sure of their scope or effect. In that case it helps to test your code on all Puppet-managed nodes. The challenge is that Puppet open source does not have any built-in mechanism to do that. This is where we can make good use of Puppet Bolt.

Using tags in Puppet Bolt aws_inventory target_mapping

Sat, 05 Jun 2021 00:00:00 +0000

We’re migrating away from Ansible to Puppet Bolt and the fact that Ansible updates broke the old ec2.py inventory script expedited that process. While that inventory script was quite rudimentary, it was able to automatically add human-readable names to the EC2 instance names. So, for example, you could target a node using “server_example_org” if the Name tag was set to “server.example.org”.

Puppet Bolt’s aws_inventory seemed to lack that feature, but after taking a good look at its code this was just a case of missing documentation, which I fixed in this PR. Here’s an example of inventory.yaml for Bolt that maps the value of the “Name” tag to the name of the Bolt target:

Puppet Bolt: taking action based on exit codes

Tue, 13 Apr 2021 00:00:00 +0000

Puppet resources like Package, File or Service are idempotent. In other words, you define the state of the resource and Puppet enforces it. The resources can be used in Bolt plans as well, which frees you from having to do state checks yourself. However, if the thing you’re managing is not a Puppet resource you have to check its state manually and use that information to drive login in your plan. Here’s a crude but functional PoC for checking if a node’s certificate is present on the Puppetmaster and acting accordingly:

The four ways to install Puppet modules

Tue, 13 Apr 2021 00:00:00 +0000

When working with Puppet and Puppet modules in particular you quickly notice that there are several ways to manage module dependencies. Today I learned about a new way introduced in Puppet Bolt 3.x, so I thought I’d compile a list that celebrates this diversity:

metadata.json: used primarily to dependency metadata to Puppet Forge. Some tools like librarian-puppet and puppet-litmus use / can use this file to download dependency modules. The downside is that you can’t define the location of the modules, so you’re limited to Forge modules.

Notes about puppet-litmus testing

Sun, 07 Mar 2021 00:00:00 +0000

Puppet Litmus is a Puppet acceptance test harness that leverages on many existing, proven technologies. In the long run Litmus aims to replace Beaker. However, the latter is currently still the only way to run multi-machine (e.g. cluster) acceptance tests in a standardized way. For example we use Beaker for our Keycloak domain mode cluster tests in puppet-module-keycloak. On a high-level the test process is the same as with Beaker:

Provision the SUT (“System Under Test”)

Passing Vagrant parameters to Puppet Litmus

Sun, 07 Mar 2021 00:00:00 +0000

Puppet Litmus allows creating SUTs (“System Under Test”) with Vagrant using Virtualbox or Hyper-V as the virtualization backend. However, the default virtual machine settings like 1024MB of memory are only suitable for very simple modules and applications. For example, Litmus testing our puppetmaster-installer will fail with the default settings.

Litmus documentation tells us that you can define parameters in provision.yaml which get then converted to environment variables. What it does not tell what to use these environment variables for. Looking at the Vagrant code in Litmus you can see that those environment variables can be used to drive the Virtualbox settings, among other things:

Limiting the scope of puppet-rspec tests

Fri, 22 Jan 2021 00:00:00 +0000

Some Puppet modules like puppet-module-keycloak have hundreds of unit tests. That is good for test coverage, but waiting for test results hurts your productivity when you’re developing tests for your new code. There are at least two ways to (temporarily) limit the scope of the tests that you run. First method is baked in into the Puppet Development Kit command-line:

$ pdk test unit --tests=spec/classes/init_spec.rb

This runs only the tests in the specified file.

Querying DNS in Puppet code

Wed, 06 Jan 2021 00:00:00 +0000

I recently had to add access control rules to OpenVPN Access Server to permit certain groups to access certain IP addresses. As we manage our Access Server configurations with Puppet using puppet-openvpnas (by the author), we needed a way to parameterize the allowed IP addresses while keeping the Puppet code readable.

This is where puppet-dnsquery came really handy. It allows using data that’s already available in DNS and using it in Puppet code. A simplistic example:

Migrating from Puppet 5 to 6 with Puppet Bolt

Thu, 31 Dec 2020 00:00:00 +0000

The main annoyance with agent-based configuration management is the management of the agents themselves. They usually don’t require any handholding after the initial install, except when you need to migrate them to a new environment. For example, migrate from a Puppet 5-based server to Puppet 6.

The migration process is not overly difficult, but in order to scale to more than a handful of nodes you will need to orchestrate the process. Historically we would have used Ansible for this task, but nowadays we prefer Puppet Bolt, which is more powerful and flexible in our use-cases. The Bolt plans typically end up being ugly, but based on my experience with Ansible Playbooks that tends to be the case anyways. I guess this is the natural result of mixing state-based configuration management, ad hoc commands and orchestration together.

Creating custom facts programmatically

Fri, 18 Dec 2020 00:00:00 +0000

Writing static facts is fairly easy even with a low level of Ruby skills. Below is an example of a fact that returns true or false depending on whether the node has a /boot partition:

Facter.add(:has_bootfs) do
  setcode do
    if Facter.value(:mountpoints)['/boot'].nil?
      false
    else
      true
    end
  end
end

As you can see above the required information was already inside the “mountpoints” fact of Facter.

Using the fact is easy:

Fattening the workflow, part 5: Hiera and content encryption

Thu, 03 Dec 2020 00:00:00 +0000

Separating data from code in Puppet modules is advisable as it improves reusability of code. The separation can be accomplished with Hiera by having separate levels based on facts, organizational units, locations, etc.

Hiera can also be used for storing private data that needs to be protected and must not be readable by outsiders. Typically the data, like all other code, is stored in a version control system that may be located outside the organisation. In such a case Hiera encryption is a must.

Fattening the workflow, part 4: Roles and profiles

Wed, 02 Dec 2020 00:00:00 +0000

If the workflow that includes the control repository, r10k and GitLab still feels too light, there are yet more ways to make the workflow heavier by adding more abstraction to the Puppet modules themselves. This is done by adopting the “Roles and Profiles pattern” that was developed by Craig Dunn and popularized in the blog posts by Gary Larizza.

Roles and profiles are just Puppet classes. Profiles are usually used for creating technology stacks. A good example is a web application whose installation requires a web server (nginx, apache), a database server (postgresql, mysql) and several other components. The configurations for such an application could of course be done with a “regular” Puppet module but the reusability of the module would be rather limited for two reasons:

Fattening the workflow, part 2: r10k

Tue, 01 Dec 2020 00:00:00 +0000

In the previous post I discussed the control repository structure. In this post I’ll talk about r10k. That tool is used for deploying control repository branches into matching Puppet environments on the Puppet server. Installing r10k is very straightforward with the bundled gem:

$ /opt/puppetlabs/puppet/bin/gem install r10k

After this you’ll need a settings file for r10k, /etc/puppetlabs/r10k/r10k.yaml:

--- 
 :cachedir: /opt/puppetlabs/puppet/cache/r10k 
 :sources: 
 control: 
 basedir: /etc/puppetlabs/code/environments 
 prefix: false 
 remote: git@gitlab.com:myorg/control-repo.git

Several requirements must be met before the configuration above will work:

Fattening the Puppet workflow, part 1: The control repository

Thu, 26 Nov 2020 00:00:00 +0000

If editing Puppet code and Hiera directly with puppetmaster feels too easy, you can complicate the Puppet workflow as much you like by adding more components to the palette. Adding the so called control repository and r10k to Puppet environment maintenance instantly adds several phases to your initially simple workflow. There are also benefits that I’ll discuss later on.

The control repository (control repo for short) is a Git repository that is used to create dynamic Puppet environments on the Puppet master. Each Git branch can be turned into a matching Puppet environment. The name of the default branch, production, is the same as the environment the Puppet agents use by default. The content of the control repo branches look like this:

Using Puppet Bolt to apply roles to nodes, part 1

Sat, 17 Oct 2020 00:00:00 +0000

NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead.

Puppet Bolt is a designed to be an orchestration tool, but it can be used for configuration management as well. For example you may have a small environment of handful of nodes where having a dedicated Puppet server would be an overkill, or you have some nodes in the infrastructure where you can’t or don’t want to install a Puppet Agent.

Improving Vim's Puppet integration

Thu, 15 Oct 2020 00:00:00 +0000

A recent package upgrade seemed to break Puppet syntax highlighting horribly on my Fedora 32 laptop. Not only did syntax highlighting not work, it was actually completely wrong and confusing. It seemed as if the Puppet file type detection was missing, so vim was probably using some a syntax highlighter for a wrong type. To see which file types are enabled I used :set filetype followed by a couple of tabs (more details on StackExchange).

Integrating Puppet Bolt with PuppetDB

Fri, 24 Apr 2020 00:00:00 +0000

In the Puppet Bolt with a Puppet control repository blog post we showed how to integrate Puppet Bolt with an existing control repository. We used a static inventory file, which is ok for testing or when you have a few target nodes. However, if you have a Puppetserver running and are not using Bolt to manage the configurations of your nodes, then the chances are that you have a fairly large node population and maintaining a static inventory would be both tedious and error-prone. One option among many is to use the PuppetDB plugin to generate a dynamic inventory and that is what we describe here.

Puppet environment leakage

Wed, 01 Apr 2020 00:00:00 +0000

Occasionally when working with feature branch and updating types and providers you may run into environment leakage issues. They manifest themselves as issues that make no particular sense and are caused by the wrong version of a type/provider such as puppetlabs-apt (see this issue) leaking into your feature environment. This can happen when you update external modules in your Puppetfile. The solution (or band-aid) is fortunately fairly easy. On the Puppetmaster run this command:

Puppet Bolt with a Puppet control repository

Mon, 30 Mar 2020 00:00:00 +0000

NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead.

Puppet Bolt is an orchestration tool similar to Ansible. It is suitable for multiple use-cases:

Running ad hoc commands on target nodes. We use this approach for testing Puppet feature branches before approving and merging them.
One-off configurations written in the Puppet language
Masterless Puppet environments (look here). You may not always have enough privileges, trust or will to manage a complete Puppet infrastructure including Puppetserver, PuppetDB and Puppet Agents.

Simple orchestration with Puppet Bolt

Mon, 30 Mar 2020 00:00:00 +0000

As discussed in the introductory post one of the use-cases for Puppet Bolt is orchestration - running actions on multiple targets in a certain order, possibly using results of actions or data from some targets on other targets. Orchestration with Bolt plans is quite easy: you just create a plan with multiple TargetSpec parameters, each representing a target or a type of target. A classic example is two or more web servers behind a load balancer. In this case we do something even simpler and just take a fact from a node called “first” and create a file on node “second” that uses the value of that fact in a file.

Puppet types and providers development part 2: creating a resource

Sat, 07 Mar 2020 00:00:00 +0000

This blog post is a part of this blog post series:

In the previous blog post we created the Puppet type librenms_service and created a dummy skeleton for the provider implementation. We were able to use the type, but it did not yet do anything. In this blog post we move to implementing the create method, which in this case was the low-hanging fruit. At this point we do not need to worry about any existence checks (the exists? method) or trying to minimizing the number of API calls with self.prefetch or something similar.

Puppet types and providers development part 1: creating the type

Tue, 03 Mar 2020 00:00:00 +0000

This article is a part of this blog post series:

Writing Puppet types and providers looks like black magic even to those who have lots of experience in the Puppet language itself. While the official types development documentation is quite ok, provider documentation is definitely lacking. Not only are there no practical examples from real life, the important features such as prefetching, resource methods and flushing are described only on a very high level. The usual answer to lack of examples is “have a look at the built-in resources”. But when you look at any built-in resource, you realize that the code is very complex because

Giving special treatment to the last entry when iterating over a Puppet hash

Thu, 20 Feb 2020 00:00:00 +0000

When you’re iterating over a hash you may end up having to treat the last hash entry differently. For example, if you’re creating entries to a JSON file you should have “},” in between the entries until you reach the last entry, where you must have just “}” or JSON data will be invalid.

Now, hashes do not have indexes, nor does Puppet support variable reassignment along the lines of $x = $x +1, so you’re out of luck trying to get the length and comparing that to a counter. So, you need some other way to detect when iteration has reached the last entry in the hash.

Debugging rspec-puppet

Fri, 07 Feb 2020 00:00:00 +0000

Usually writing rspec-puppet tests is straightforward. Then you run into some problem that does not make any sense. In my case I ran into a couple of duplicate resource declaration issues when adding rspec tests to our puppetmaster-installer. The error message was not that useful:

$ pdk test unit --tests=spec/classes/puppetboard_spec.rb
--- snip ---
 Failures:
 1) puppetmaster::puppetboard minimal settings on ubuntu-18.04-x86_64 is expected
    to compile into a catalogue without dependency cycles
    Failure/Error: it { is_expected.to compile }
      error during compilation: Evaluation Error: Error while evaluating a
      Resource Statement, Duplicate declaration: Class[Apache] is already
      declared; cannot redeclare (file: /home/samuli/opt/puppeteers/puppet-
      puppetmaster/spec/fixtures/modules/puppetmaster/manifests/puppetboard.pp,
      line: 156) (file: /home/samuli/opt/puppeteers/puppet-puppetmaster
      /spec/fixtures/modules/puppetmaster/manifests/puppetboard.pp, line: 156,
      column: 3) on node puppet.example.org
--- snip ---

That particular block of code in puppetboard.pp did include a parameterized call to the apache class. But nothing else (afaics) included that class. Also, in Vagrant the same code worked just fine, which made things seem even more bizarre. To add insult to the injury the error message did not tell where else the apache class was included. That would probably have been way too helpful.

Managing Jenkins with Puppet, part 1: Basic installation

Fri, 31 Jan 2020 00:00:00 +0000

It seems every other organization is using Jenkins these days. Jenkins is a continuous integration and continuous delivery server that can be used to automate building, testing, and delivering or deploying software.

Many organizations also use Puppet for their configuration management needs. Puppet is, if not the de facto configuration management solution, at least one of the most mature and field tested ones. It consists of several server components, agents, a declarative programming language, amongst others. Puppet can be used to install, set up and manage large number of all kinds of resources.

Modifying XML files with Puppet's Augeas resource

Fri, 31 Jan 2020 00:00:00 +0000

This article is based on an earlier article written in Finnish.

Here I describe how to make changes to an XML file with Augeas - a useful approach when managing the whole file as a template is out of the question. The configuration file used as an example, web.xml, belong to Pwm, and we intend to set the webapp context parameter.

As discussed before it is important to get an idea of how Augeas sees the file before attempting to modify it with Puppet. First we tell which Augeas lens to use (here: Xml) and which file(s) to associate with the Xml lens. This can be done with in one step:

Using Augeas to manage resources in Puppet

Fri, 31 Jan 2020 00:00:00 +0000

This translated from the original Finnish article that is available here.

The Augeas resource in Puppet allows one to manage parts of a complex configuration file instead of managing the whole file using a template or a static file. For trivial cases stdlib’s file_line resource may be sufficient. While use of templates or static files is in general more reliable and easier than use of Augeas, the latter is quite useful in many cases. As using Puppet Augeas resource directly is quite challenging, a useful practice is to first make the changes manually with “augtool” and only then convert the results into Puppet Augeas resources. We use PostgreSQL’s pg_hba.conf file here as an example.

Allowing changes to a Puppet-managed resource outside of Puppet

Wed, 29 Jan 2020 00:00:00 +0000

When a resource is managed by Puppet it is typically managed fully, or not at all. Exceptions to this rule, such as the “replace” parameter in the File resource, are rare. However, sometimes you end up having to change the properties of a resource outside of Puppet without making Puppet overwrite those changes on the next run. A good example is a set of firewall rules managed with puppetlabs-firewall - you generally want to manage the whole ruleset with Puppet so that you can be sure of the state of the ruleset. For example, if you have libvirt installed, a bunch of virbr0-related rules may appear without your consent. On RedHat-based distros you may also see several rules getting removed by puppetlabs-firewall. Or a fellow sysadmin may have added some random iptables rules for debugging purposes. These are good reasons to have

Getting the interface that matches an IP in Puppet

Tue, 14 Jan 2020 00:00:00 +0000

I was recently in a situation where I needed to figure out which interface name matched a particular, static IPv4 address. On a single or a handful of nodes this could have been hardcoded in Hiera, for example, but there were dozens of nodes. The way I solved this was by creating a custom fact that uses the IPv4 address and loops through the interfaces list to find the matching interface name:

Why modules in Puppetfile should be sorted

Mon, 30 Dec 2019 00:00:00 +0000

This is probably one of the shortest blog postings in the history of this blog. But as this topic does not seem to be covered in more generic articles about avoiding Git merge conflicts (e.g. this, this and this), here it goes.

Adding modules to the Puppetfile is a quite common task. When you or your team has multiple feature branches at the same time many of those branches may add new Puppet modules to the Puppetfile. This is fine, unless the new modules are added to the bottom of the Puppetfile: Git will get confused because it sees that multiple commits have changed the exact same place in the Puppetfile.

Puppet rspec testing: understanding describe and context

Fri, 06 Dec 2019 00:00:00 +0000

In the Puppet module rspec testing with PDK blog post I covered the procedure of creating and running puppet-rspec tests with PDK. This blog post will go more in-depth to that topic, in particular describe and context which you see in all puppet-rspec tests.

When looking at the puppet-rspec test syntax the first time it looks very “rubyesque”. Somebody used to staring at Python code, for example, could call the syntax “odd” or “1337”. Fortunately reading the Ruby rspec documentation helps understand what is going on under the hood and thus making the spec files look less magical. In my case rspec documentation helped understand how contexts work. In the previous blog post a context was created for every supported operating system listed in metadata, and rspec tests were run for each of them by using facts from facterdb.

Testing Puppet feature branches in Foreman

Tue, 26 Nov 2019 00:00:00 +0000

With default settings Foreman does not allow Puppet Agents to specify their own environment when connecting to the Puppetserver. This makes it impossible to use r10k and feature branch workflow like you can with a normal Puppetserver. This behavior can fortunately be reverted easily by modifying Foreman configuration file, /etc/foreman/settings.yaml:

# Allow testing feature branches on nodes
:enc_environment: false

After modifying settings restart the webserver, e.g.

Explicit high-level dependencies in Puppet

Wed, 09 Oct 2019 00:00:00 +0000

I’ve noticed that some people - I won’t reveal their names here to protect them - love using contain and/or ordering arrows in Puppet. A typical use-case is for role classes:

class role::webserver { contain ::packetfilter::endpoint contain ::profile::monit contain ::profile::unixbase contain ::profile::ipa_client contain ::profile::webserver

Class[::packetfilter::endpoint] -> Class[::profile::monit] -> Class[::profile::unixbase] -> Class[::profile::ipa_client] -> Class[::profile::webserver] }

The above code nicely ensures that all the classes get applied in the exact order defined above. What happens under the hood is that Puppet generates dependencies between the classes and applies them in order.

Reprovisioning bare metal Linux servers with Foreman and IPMI

Wed, 25 Sep 2019 00:00:00 +0000

Recently we had to reprovision a bare metal server that was in Foreman and had IPMI support. The goal was to check if our automated “force-join to FreeIPA domain after reprovisioning” logic worked as intented. We did not have access to any graphical IPMI console, nor Foreman’s GUI, so we had to do everything from the command-line. The host we reprovisioned was a CentOS 7 instance. It is assumed that IPMI tools are installed. This post was inspired by the Automate SuperMicro Server Provisioning or Die Trying blog post.

Enabling PHP 7.4 on CentOS 7 with Puppet

Tue, 17 Sep 2019 00:00:00 +0000

The PHP version comes in default CentOS 7 repositories is rather horribly outdated. This is problematic when running Wordpress, which has to be upgraded constantly, and really old PHP versions may not be accepted, security-patched or not. The solution is to get updated PHP from the remi yum repositories. There are two sets of PHP packages as described here:

Install updated packages that replace the old PHP 5 packages (i.e. names are the same)

Puppet module rspec testing with PDK

Thu, 05 Sep 2019 00:00:00 +0000

If you’ve searched for instructions on how to write unit tests for Puppet code you’ve probably stumbled upon tutorials such as this that are of good quality, but suggest installing and using tools such as puppetlabs_rspec_helper directly. Or you may encounter references to rake, bundler, rvm and rspec. Then you start experimenting these tools, some of which (rvm and bundler) are aimed at managing the dependency hell that arises when you, or your team, works simultaneously on multiple projects with different dependency requirements. In the end you are maybe able to run spec tests on your Puppet module, but after all installations of gems and experiments you’re unsure what you did to make everything work. You also don’t want to become a hard-core Ruby developer who actually understands all these tools well - you just want to test your Puppet code.

Moving Jenkins user home directory

Thu, 29 Aug 2019 00:00:00 +0000

Jenkins is a continuous integration (CI) / continuous delivery (CD) platform consisting of a Jenkins master - essentially a controller - which launches jobs on Jenkins slaves. Often slaves are controlled via SSH.

Recently I ran into an issue with a legacy Jenkins slave where the “jenkins” where Puppet tried to enforce the home directory of “jenkins” to be /home/jenkins instead of /var/lib/jenkins:

Error: Could not set home on user[jenkins]: Execution of '/usr/sbin/usermod -d /home/jenkins jenkins' returned 8: usermod: user jenkins is currently used by p rocess 802

The usermod command failed because several Jenkins processes (ssh-agent, Java) locked the user from modification. The solution was relatively straightforward:

Lisää läskiä työnkulkuun, osa 4: Roolit ja profiilit

Wed, 31 Jan 2018 00:00:00 +0000

Mikäli hallintarepon, r10k:n ja GitLab:in sisältävä työnkulku tuntuu vielä liian kevyeltä, on läskiä ja abstraktiotasoja lisättävissä onneksi toisaalla, nimittäin itse Puppet-moduuleissa. Pelastuksen tarjoaa ns. “Roles and Profiles pattern”, jonka kehitti alunperin Craig Dunn, ja joka löi läpi todennäköisesti Gary Larizzan blogikirjoitusten myötä. Alla käytämme englanninkielisen termin sijaan lähes suomenkielistä termiä “roolit ja profiilit”.

Roolit ja profiilit ovat aivan tavallisia Puppet-luokkia. Ne eivät ole kuitenkaan ole Gary Larizzan termiä käyttäen ns. komponenttimoduuleja, jotka on suunniteltu jonkin yksittäisen asian tekoon, ja jotka olisivat siten (ainakin periaatteessa) jaettavissa muiden käyttäjien kesken esim. Puppet Forgessa.

Vagrant, Powershell ja parametrien välittäminen

Mon, 13 Nov 2017 00:00:00 +0000

Vagrant on näppärä väline myös Windows-virtuaalikoneiden luontiin esim. Puppet-resurssien testauksessa. Windows-virtuaalikoneille on luontevaa käyttää Powershell-provisiointia, minkä Vagrant tekee melkoisen yksinkertaiseksi. Aika nopeasti ilmenee tarve välittää Powershell-skriptille parametrejä Vagrantin kautta. Tästä on hieman hankalaa löytää esimerkkejä, joten tässä yksi.

Oletetaan että meillä on Powershell-skripti nimeltä bootstrap_windows.ps1 Vagrant-projektin alihakemistossa ‘provision’. Skripti asentaa ja konfiguroi Puppet-agentin Windows-virtuaalikoneelle. Skriptille pitää välittää kaksi parametria: Puppetserverin FQDN (serverName) ja mukavuusbonuksena virtuaalikoneelle haluttu FQDN (certName). (Käytän itse Vagrantille yaml-muotoista ns. data-driven vagrant-tiedostoa, joten syntaksi on hieman erilainen vakiomuotoiseen Vagrant-tiedostoon verrattuna).

Puppet-moduulien laadunvarmistus, osa 1: automaattiset tarkistustyökalut

Mon, 27 Apr 2015 00:00:00 +0000

Puppet-moduulien laatu pysyy korkeana, kun syntaksi- ja tyylivirheitä päästetä edes versionhallintaan asti. Puppet-koodin syntaksivirheet löytyvät helposti:

$ puppet parser validate tiedosto

ERB-templateiden virheet saa selvitettyä seuraavalla tavalla:

$ erb -P -x -T '-' tiedosto|ruby -c

JSON-tiedostojen (metadata.json) virheet etsiminen vaatii hieman kikkailua:

$ cat tiedosto|json_pp -f json -t null

Tyylivirheitä voi etsiä puppet-lint -työkalulla:

$ puppet-lint tiedosto
 $ puppet-lint .

Puppet-lint tunnistaa myös puppet-koodin, joka on eteenpäin tai taaksepäin epäyhteensopivaa. Onkin hyvin järkevää ajaa puppet-koodi aina puppet-lintin läpi ennen suuria Puppet-ympäristön päivityksiä.

Määriteltyjen resurssien luonti suoraan Puppet-luokista

Mon, 02 Feb 2015 00:00:00 +0000

Jotkin Puppet ENC:t (External Node Classifier), hyvänä esimerkkinä Foreman, eivät tue määriteltyjen resurssien (Defined Resource Types) käyttöä suoraan, vaan kaikki konfiguraatiot pitää sisällyttää Puppet-luokkiin.

Myös Hieran kanssa määritellyt resurssit ovat hieman ongelmallisia, koska ne pitää erikseen luoda create_resources funktiolla. Eräs ratkaisu on määrittää kaikki create_resources -funktiot site.pp-tiedostossa (sovellettu täältä):

create_resources('mysqldump::backup',hiera('mysqldump_backups', {}))

Tämän lähestymistavan ongelma on se, että uudet määritellyt resurssit pitää muistaa lisätä site.pp-tiedostoon.

Yksinkertaisempi ja myös Foremanin kanssa toimiva ratkaisu on luoda määritellyt resurssit suoraan Puppet-moduulin pääluokasta (sovellettu täältä):