Introduction

In the “Packer QEMU: Building Ubuntu 20.04/22.04/24.04 VM images” blog post we briefly mentioned ways to improve Packer QEMU performance. In this article we’ll go into the details on how this is implemented. The key in improving Packer QEMU performance is to split the build into two distinct stages:

1. Operating system installation stage

2. Provisioning stage

This setup cuts down development time considerably as repeating the provisioning stage becomes a lot faster.

Introduction

Packer QEMU builder allows you to build virtual machine images from scratch. You need a Packerfile, an automatic installation configuration file (autoinstall, kickstart or preseed) and the operating system installer (e.g. ISO). The benefit of Packer QEMU in particular are that you can have fine-grained control over the virtual machine image content such as disk partitioning. Moreover you don’t need to rely on official base images produced by other people, such as the official Ubuntu Cloud images.

The Using tags in Puppet Bolt aws_inventory target_mapping showed how to use AWS “Name” tag as the target name for Puppet Bolt. Use of tags can be extended to creating Bolt target groups:

groups:
  - name: webservers
    targets:
     - _plugin: aws_inventory
       region:
         _plugin: env_var
         var: AWS_DEFAULT_REGION
       filters:
         - name: tag:Type
           values: [webserver]
       target_mapping:
         name: tags.Name
         uri: public_ip_address

All you need to do is add a “filter” section with one filter. The “name” parameter tells Bolt that the filter is based on tags and what tag to look for. The “Type” that follows “tag” is the name of the tag you want to filter by. The “values” array lists acceptable values for that tag. Any nodes that don’t have the tag (here: “Type”) or where tag value is not acceptable (e.g. “db”) will not be shown in the group.

Scheduling meetings with email is annoying. When everyone is on the same timezone that kind of works, but tends to be slow. When you factor in timezone differences the process becomes really error-prone and slows down even more, often to several days. For this reason we wanted to provide people - including our customers - a way to easily schedule meetings with us.

We’re special in the sense that we self-host almost everything we use. For email and calendars we use Zimbra and for our website we use Wordpress. Most people seem to use Calendly as a meeting scheduler, but we ended up choosing wpcal.io which is a native Wordpress plugin and seems to work quite nicely.

Puppet feature environments are an excellent way to test code before deploying it, typically to production. They allow testing Puppet runs on no-operation mode across the whole node population managed by Puppet. There are sometimes cases where your code changes potentially impact many nodes and you’re not exactly sure of their scope or effect. In that case it helps to test your code on all Puppet-managed nodes. The challenge is that Puppet open source does not have any built-in mechanism to do that. This is where we can make good use of Puppet Bolt.

I recently checked the pricing model for Amazon Fargate to see if migrating a fair number of EC2 instance-based workloads to containers would save money. In theory this should have been the case, as a container has less “fat” compared to a full virtual machine. In this case the workload itself was perfectly suited for a container: a single process with single configuration file that was mostly network-bound running on t3.micro instances.

Vagrant has built-in support for connecting to Windows VMs using the vagrant rdp command. It does basically does the same as vagrant ssh for Linux VMs. There are a few shortcomings, though: vagrant rdp fails to automatically accept the host key of the VM it is connecting to, at least when using it from Linux with xfreerdp. So, you first need to connect to the VM directly once with raw xfreerdp, then vagrant rdp will start working. Then, when you wipe out the VM, you need to repeat the process again. If you have a high resolution screen you will also notice that vagrant rdp will give you a tiny Windows desktop where you need a magnifying glass to read the text.

I’ve been doing lots of Windows automation recently. My goal has been to be able to reproducibly create a Windows Server 2019-based Buildbot workers, first in a Vagrant environment, then later in AWS EC2 using pre-built images created with Packer. This task requires installing Visual Studio 2019 build tools automatically and fetching the project’s dependencies with vcpkg automatically. It is possible to install Visual Studio 2019 with Chocolatey quite easily, but what if you need to install additional components to Visual Studio without using the Visual Studio installer GUI? Looking around did not quickly reveal any de facto instructions on how to do it, so this article aims to fill that gap.

We’re migrating away from Ansible to Puppet Bolt and the fact that Ansible updates broke the old ec2.py inventory script expedited that process. While that inventory script was quite rudimentary, it was able to automatically add human-readable names to the EC2 instance names. So, for example, you could target a node using “server_example_org” if the Name tag was set to “server.example.org”.

Puppet Bolt’s aws_inventory seemed to lack that feature, but after taking a good look at its code this was just a case of missing documentation, which I fixed in this PR. Here’s an example of inventory.yaml for Bolt that maps the value of the “Name” tag to the name of the Bolt target:

We use Vagrant and Virtualbox a lot because it is an excellent tool for infrastructure code development. Unfortunately it is also very prone to breakage during operating system upgrades, especially if you depend on many plugins. When I upgraded from Fedora 32 to 34 Vagrant broken even more than usual due to conflicting rspec dependencies. Basically every Vagrant plugin refused to install with this error message:

$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...

Vagrant failed to properly resolve required dependencies. These
errors can commonly be caused by misconfigured plugin installations
or transient network issues. The reported error is:

conflicting dependencies rspec (= 3.10.0) and rspec (= 2.14.1)
  Activated rspec-2.14.1
  which does not match conflicting dependency (= 3.10.0)

  Conflicting dependency chains:
    rspec (= 2.14.1), 2.14.1 activated

  versus:
    rspec (= 3.10.0)

  Gems matching rspec (= 3.10.0):
    rspec-3.10.0

The rspec dependency conflict does not sound particularly scary for somebody who is not developing Vagrant or Vagrant plugins. Therefore ignoring the dependencies is likely to be just fine: