Update: the problem described in this article seems to be resolved in Fedora 36 (NetworkManager 1.38.0-2.fc36). It is still present in Fedora 35 (NetworkManager 1.32.12-2.fc35).

The title of this blog post may seem familiar to some, as I solved this problem for OpenVPN earlier. I won’t repeat myself here, so please check the OpenVPN article to understand why this is happening.

Wireguard servers - unlike OpenVPN servers - are not able to push any settings such as DNS servers to their clients. However, Wireguard client itself is able to run PostUp commands to configure per-link DNS. The same approach can be used with OpenVPN “up” scripts as well, if you prefer having full control at the client-side. Here’s a Wireguard example:

Puppet resources like Package, File or Service are idempotent. In other words, you define the state of the resource and Puppet enforces it. The resources can be used in Bolt plans as well, which frees you from having to do state checks yourself. However, if the thing you’re managing is not a Puppet resource you have to check its state manually and use that information to drive login in your plan. Here’s a crude but functional PoC for checking if a node’s certificate is present on the Puppetmaster and acting accordingly:

Puppet Litmus allows creating SUTs (“System Under Test”) with Vagrant using Virtualbox or Hyper-V as the virtualization backend. However, the default virtual machine settings like 1024MB of memory are only suitable for very simple modules and applications. For example, Litmus testing our puppetmaster-installer will fail with the default settings.

Litmus documentation tells us that you can define parameters in provision.yaml which get then converted to environment variables. What it does not tell what to use these environment variables for. Looking at the Vagrant code in Litmus you can see that those environment variables can be used to drive the Virtualbox settings, among other things:

The vagrant-hostmanager plugin is a really nice plugin which helps Vagrant VMs communicate with each other using names instead of IP addresses. Essentially it manages hosts entries for the VMs, acting like a poor-man’s DNS. Now, on *NIX operating systems you typically use sudo, which caches your authorization. So, when hostmanager starts, you type your password once and then it is able to add all the host entries it needs to /etc/hosts. This is not so on Windows: you will get one UAC prompt per modification to c:\windows\system32\drivers\etc\hosts. This will eventually drive you crazy. To fix this you need to permit your Windows user account to modify the hosts-file without privilege elevation. To pull this off, you will need to modify that file’s access control lists. The process is quite convoluted imho, but here it goes:

We’re using the centos/7 base box in Vagrant a lot and often hit this problem when bringing up new VMs with Vagrant:

umount: /mnt: not mounted

This error comes from vagrant-vbguest plugin which tried to install the Virtualbox Guest Additions but failed. The error message is misleading, because the real issue is related to upstream (CentOS) yum repositories. As explained in this Vagrant bug report:

“Basically the CentOS that you’re installing is containing an old kernel version which has been removed from CentOS repositories. Vagrant is trying to install Guest Additions, which has as a dependency the kernel-devel package. kernel-devel package must be the same version as the kernel package that you have it installed in your CentOS box, version which is removed from the CentOS repos.”

We use Vagrant and Virtualbox heavily when developing Puppet code. We also do provisioning of baremetal servers with Foreman. While Vagrant is a really powerful tool, it is not designed for creating VMs that boot from the network (PXE boot). Fortunately Virtualbox itself provides a rich command-line with which you can create and configure virtual machines and configure them to boot from the network. It is assumed below that there is a PXE boot server (e.g. Foreman) created into a hostonly network with Vagrant. Here’s an example script that create a VM with vboxmanage and adds it the same hostonly network interface (vboxnet15) as the PXE boot server:

Sometimes you’d like to pass a dynamic value to a Terraform resource’s provider parameter. This can be done, but some background first.

Terraform allows you to define multiple providers of the same type using provider aliases. This is useful when you’re working with a provider that is tied to a region, the AWS provider being a prime example. The moment you want to create resources in multiple regions you will have to learn provider aliases.

This could possibly be the most esoteric blog post I’ve ever written, but here it goes. So, for reasons even we have difficulty comprehending (hint: related to PXE booting in a Vagrant/Virtualbox environment) we had to find out which inactive vboxnet (hostonly) interface had the highest index. We wanted the code to (hopefully) run on all platforms (Linux, MacOS, Windows) so we wanted to write this code in Ruby. So, this was the brutal but working end-result:

Grafana is a common tool to visualize data from multiple datasources. Perhaps the most common datasource is Prometheus. If an organization has a Single-Sign On solution, it makes sense to authenticate users centrally with that solution That will make authentication easier and friendlier for end users (authenticate once and then access multiple services), and also enable stronger authentication by adding multifactor authentication.

In addition to authenticating user, it is also possible to authorize Grafana users based on their group membership. 

Some Puppet modules like puppet-module-keycloak have hundreds of unit tests. That is good for test coverage, but waiting for test results hurts your productivity when you’re developing tests for your new code. There are at least two ways to (temporarily) limit the scope of the tests that you run. First method is baked in into the Puppet Development Kit command-line:

$ pdk test unit --tests=spec/classes/init_spec.rb

This runs only the tests in the specified file.