Introduction on Puppeteers Oy

Installing OpenBSD 7.5 with virt-install on Linux

Wed, 12 Jun 2024 00:00:00 +0000

Introduction

OpenBSD 7.5 is a UNIX-like operating system. The project’s “efforts emphasize portability, standardization, correctness, proactive security and integrated cryptography”. Compared to modern Linux distributions OpenBSD is very compact, but comes with the usual packages and ports tree you would expect from any *BSD operating system. This article shows the process of installing OpenBSD 7.5 with virt-install, making it a virtualization guest on Linux.

Downloading the ISO installer

First you need to download the OpenBSD 7.5 ISO installer on your Linux system. On most systems the amd64 install75.iso is the correct choice. Check the OpenBSD download page for other architectures.

Red Hat subscriptions quick start

Mon, 15 Jan 2024 00:00:00 +0000

What follows is an minimalistic, bare-bones guide to get you started with Red Hat subscriptions quickly. No spoon-feeding, no introductions, just get your first RHEL* system into a blessed state quickly, while your attention can still keep up.

We assume you are starting from scratch and generally have no glue. We also assume that your boss has punished you with this task, not your mom, and therefore you represent a company, not a Sapiens specimen that looks like you. If this is not the case, then it isn’t.

Options for building self-service infrastructure with Ansible and Terraform

Wed, 22 Nov 2023 00:00:00 +0000

Introduction

Being able to build IT infrastructure always having to involve an IT, operations or platform team seems be a big priority for many organizations. This is understandable because processes typically slow down to a crawl when they have to pass organizational boundaries. Also, teams responsible for infrastructure would not want to spend time on routine requests if possible. Provisioning infrastructure is no exception here. It is possible to create a custom solution to enable self-service infrastructure with Ansible and Terraform, but that takes a fair amount of effort. In this article we focus on open source and commercial open source offerings that make enabling self-service infrastructure provisioning easier.

Linux Azure AD authentication options

Wed, 26 Apr 2023 00:00:00 +0000

There are a myriad of ways to login to Linux systems using Azure AD credentials. Photo credit.

Introduction

There are several ways to do Linux Azure AD authentication. Note that Azure AD was renamed “Microsoft Entra ID” recently, but in this article we’ll use its old name, at least for now. In other words you can log in to your Linux hosts Azure Active Directory (“Azure AD”) credentials in various ways. Azure, Microsoft’s public Cloud, builds on top of Azure AD. In fact, your Azure users, groups, roles and role assignments are stored in Azure AD.

Openshift versions: all you hopefully need to know

Wed, 05 Apr 2023 00:00:00 +0000

Introduction

Red Hat Openshift is essentially an opinionated Kubernetes distribution that comes with a large number of features such as CI/CD and container registry built-in; for a full list of differences look at the Red Hat OpenShift vs. Kubernetes. Openshift comes in a number of versions, some commercial and some open source. The naming history of Openshift is particularly confusing. That’s why I wrote this blog post where I attempt to list most Openshift versions and variants and outline their main differences.

Automation use-cases in the Cloud: Red Hat Open Tour 2022 Tallinn

Fri, 14 Oct 2022 00:00:00 +0000

Automation use-cases in the Cloud

Johan Wennerberg, a Solution Architect for Red Hat Nordics in Stockholm gave presentation in Red Hat Open Tour 2022 Tallinn. In his presentation titled “Gain robust repeatability as selfservice, by automating your automation” he listed several automation use-cases in the Cloud. Each of these automation use-cases is made possible by automation stacks. The automation stacks are technology stacks that produce business value through automation. In practice you need to build your automation stacks on top of infrastructure as code.

Ansible variable validation with ansible.utils.assert

Thu, 06 Oct 2022 00:00:00 +0000

Overview of Ansible quality assurance

Ansible is an IT automation engine which you can use for configuration management, orchestration and device management, among other things. While you can get started fast with Ansible, ensuring high-quality, bug-free code can be challenging. Moreover, there’s not that much official, high-quality or coherent documentation available on Ansible quality assurance best practices. While low hanging fruit like Ansible variable validation are available, they are not emphasized in official documentation.

Azure function monitoring with Azure Monitor and Application Insights

Mon, 26 Sep 2022 00:00:00 +0000

Microsoft Azure provides a metrics and monitoring framework called Azure Monitor. With it you can monitor your Cloud infrastructure and services running there. You can view graphs of the metrics, alert on threshold and all that usual stuff, just like in AWS Cloudwatch.

Some Cloud resources like Azure Functions expose “a limited number of useful metrics” in Azure Monitor. This is a polite way of saying that the metrics are “completely useless” for most intents and purposes. For example, an Azure Function that gets triggered a few times a week on an irregular schedule by real people you don’t really have any useful metrics in Azure Monitor which you could alert on if something is off.

Hiera lookups in rspec-puppet

Wed, 09 Feb 2022 00:00:00 +0000

While rspec-puppet documentation is quite decent, it does not really explain how to test classes that get their parameters via Hiera lookups, such as profiles in the roles and profiles pattern. Several parameters related to Hiera are listed in the rspec-puppet configuration reference, but that’s all. The other documentation you find on the Internet is generally outdated and not applicable for modules converted with PDK.

In this article we show how to test an “ipa_client” class that uses lookups to get its parameter from Hiera. It used to be a profile that was later separated from the control-repo into a separate module, which means that rspec-puppet does not have access to the control-repo’s Hiera data.

Windows domain in Azure

Tue, 11 Jan 2022 00:00:00 +0000

I wrote this article to better understand all the pieces called “AD” or “Active Directory” in Microsoft Azure fit together. The pieces are as follows:

Active Directory (AD): The is the classic on-premise Active Directory with LDAP, Kerberos, group policies and all that. Traditionally Windows machines in larger environments have been domain-joined to on-premise AD.
Active Directory Federation services (AD FS): adds support for modern web-SSO protocols (SAML, OAuth 2.0 and OIDC) to on-premise Active Directory.

Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt

Tue, 28 Dec 2021 00:00:00 +0000

The traditional way of managing systems with Puppet is to install Puppet agent on the nodes being managed and point those agents to a Puppet server (more details here). This approach works well for environments with tens or hundreds of nodes, but is an overkill for small environments with just a handful of nodes. Fortunately nowadays Puppet Bolt enables you to push configurations to the managed nodes via SSH and WinRM. When using Bolt the catalog (desired state) is created on the computer running Puppet Bolt using facts from managed nodes and then sent to the managed nodes to be applied.

Short introduction to Packer and Vagrant

Sun, 31 Oct 2021 00:00:00 +0000

This article is a short introduction to Packer and Vagrant - tools that we often recommend to our customers but which may be a bit hard to understand if you have no previous expose to them.

Packer

Packer is used to “create identical machine images for multiple platforms from a single source configuration”. Packer works in a very simple fashion:

It reads its configuration file
It launches a virtual machine based on the “base box” you’ve defined. The base box would be a “vanilla” Linux distribution such as Ubuntu 20.04.
It runs the provisioning scripts you’ve written on the base box (via SSH)
It removes deployment-specific things (e.g. SSH host keys)
It shuts down the VM
It creates a VM image out of base box + your changes

After this you can distribute the base box to people and they will have an identical system at their disposal. If you need to modify the image you can just change the provisioning scripts and/or the base image and rebuild. The configuration file can and should be stored in version control repository.

OpenVPN server options in Azure

Thu, 21 Oct 2021 00:00:00 +0000

OpenVPN is among the best VPN solutions, especially when you need a cross-platform (Windows, Mac, Linux, Android, iOS) solution that just works. It is best suited for point to site (P2S) or “road warrior” setups where users access internal resources using a VPN client when on the road.

When setting up OpenVPN server in Azure you have several options at your disposal:

Set up your own open source OpenVPN 2 server

What Vagrant plugins actually are?

Tue, 01 Jun 2021 00:00:00 +0000

This was a revelation that came to me when fixing Vagrant on my Fedora 34 laptop: Vagrant plugins seem to be just Ruby gems installed into an isolated runtime environment, with “vagrant plugin” ensuring that the gems are installed in the correct place.

The same gems that you can install with “vagrant plugiin install” can be found using vanilla gem command:

$ vagrant plugin list
rexml (3.2.5, global)
vagrant-hostmanager (1.8.9, global)
vagrant-vbguest (0.29.0, global)
winrm (2.3.6, global)
winrm-elevated (1.2.3, global)
winrm-fs (1.3.5, global)

$ gem info --remote rexml

*** REMOTE GEMS ***

rexml (3.2.5)
    Author: Kouhei Sutou
    Homepage: https://github.com/ruby/rexml

    An XML toolkit for Ruby

$ gem info --remote vagrant-vbguest

*** REMOTE GEMS ***

vagrant-vbguest (0.29.0)
    Author: Robert Schulze
    Homepage: https://github.com/dotless-de/vagrant-vbguest

    A Vagrant plugin to install the VirtualBoxAdditions into the guest
    VM

$ gem list|grep vagrant
<no output>

On the remote side you can find all the familiar Vagrant plugins as gems, but vanilla gem won’t find them installed locally, because they’re in a different path.

Managing self-hosted GitLab with Terraform

Sun, 16 May 2021 00:00:00 +0000

GitLab has an extensive API which allows managing its internal configuration such as users, groups and projects. The official GitLab Terraform provider builds on top of this API.

It assumed here that the Terraform controller (e.g. your laptop) is able to reach your GitLab instance’s TCP port 443 - if it can’t there’s no way Terraform’s API calls will work. The same approach should work with gitlab.com - you can probably (=untested) skip the base_url parameter in that case.

The four ways to install Puppet modules

Tue, 13 Apr 2021 00:00:00 +0000

When working with Puppet and Puppet modules in particular you quickly notice that there are several ways to manage module dependencies. Today I learned about a new way introduced in Puppet Bolt 3.x, so I thought I’d compile a list that celebrates this diversity:

metadata.json: used primarily to dependency metadata to Puppet Forge. Some tools like librarian-puppet and puppet-litmus use / can use this file to download dependency modules. The downside is that you can’t define the location of the modules, so you’re limited to Forge modules.

Notes about puppet-litmus testing

Sun, 07 Mar 2021 00:00:00 +0000

Puppet Litmus is a Puppet acceptance test harness that leverages on many existing, proven technologies. In the long run Litmus aims to replace Beaker. However, the latter is currently still the only way to run multi-machine (e.g. cluster) acceptance tests in a standardized way. For example we use Beaker for our Keycloak domain mode cluster tests in puppet-module-keycloak. On a high-level the test process is the same as with Beaker:

Provision the SUT (“System Under Test”)

Configuring ejabberd multi-user chatrooms

Fri, 12 Feb 2021 00:00:00 +0000

Ejabberd is a very flexible and scalable XMPP server. We use it because it can be configured using a simple yaml configuration file and managed via ejabberdctl commands. This makes it a good fit for our infrastructure as code approach. That said, ejabberd does require one to understand the XMPP protocol/jargon as it does not abstract away any of it. So simple things like creating chatrooms, limiting access to them and playing history backlog to users who connected them can be an effort if you’ve never done it.

Ejabberd message archiving and history playback

Fri, 12 Feb 2021 00:00:00 +0000

When using ejabberd in a company for multi-user chat (see my earlier blog post) people will quickly start complaining that they missed discussions because they had been offline. Ejabberd does play back last 20 messages when user joins a room, but that is too little for an active chatroom. The simplistic approach is to set history_size in mod_muc settings in ejabberd.yml to a higher value:

mod_muc:
  history_size: 200
  --- snip ---

This simplistic approach should work with any XMPP clients.

Creating custom facts programmatically

Fri, 18 Dec 2020 00:00:00 +0000

Writing static facts is fairly easy even with a low level of Ruby skills. Below is an example of a fact that returns true or false depending on whether the node has a /boot partition:

Facter.add(:has_bootfs) do
  setcode do
    if Facter.value(:mountpoints)['/boot'].nil?
      false
    else
      true
    end
  end
end

As you can see above the required information was already inside the “mountpoints” fact of Facter.

Using the fact is easy:

Fattening the workflow, part 5: Hiera and content encryption

Thu, 03 Dec 2020 00:00:00 +0000

Separating data from code in Puppet modules is advisable as it improves reusability of code. The separation can be accomplished with Hiera by having separate levels based on facts, organizational units, locations, etc.

Hiera can also be used for storing private data that needs to be protected and must not be readable by outsiders. Typically the data, like all other code, is stored in a version control system that may be located outside the organisation. In such a case Hiera encryption is a must.

Fattening the workflow, part 4: Roles and profiles

Wed, 02 Dec 2020 00:00:00 +0000

If the workflow that includes the control repository, r10k and GitLab still feels too light, there are yet more ways to make the workflow heavier by adding more abstraction to the Puppet modules themselves. This is done by adopting the “Roles and Profiles pattern” that was developed by Craig Dunn and popularized in the blog posts by Gary Larizza.

Roles and profiles are just Puppet classes. Profiles are usually used for creating technology stacks. A good example is a web application whose installation requires a web server (nginx, apache), a database server (postgresql, mysql) and several other components. The configurations for such an application could of course be done with a “regular” Puppet module but the reusability of the module would be rather limited for two reasons:

Fattening the workflow, part 2: r10k

Tue, 01 Dec 2020 00:00:00 +0000

In the previous post I discussed the control repository structure. In this post I’ll talk about r10k. That tool is used for deploying control repository branches into matching Puppet environments on the Puppet server. Installing r10k is very straightforward with the bundled gem:

$ /opt/puppetlabs/puppet/bin/gem install r10k

After this you’ll need a settings file for r10k, /etc/puppetlabs/r10k/r10k.yaml:

--- 
 :cachedir: /opt/puppetlabs/puppet/cache/r10k 
 :sources: 
 control: 
 basedir: /etc/puppetlabs/code/environments 
 prefix: false 
 remote: git@gitlab.com:myorg/control-repo.git

Several requirements must be met before the configuration above will work:

Fattening the workflow, part 3: GitLab and similar

Tue, 01 Dec 2020 00:00:00 +0000

In the previous posts of this series I discussed the control repository and r10k. The last component in fattening the workflow is Git provider such as GitLab, GitHub or Bitbucket. Nowadays all of them are reasonable choices for storing private Git modules, which Puppet control repositories and site-specific modules tend to be. One of the biggest advantages of GitLab over its competition is that it is open source and can be installed locally.

Fattening the Puppet workflow, part 1: The control repository

Thu, 26 Nov 2020 00:00:00 +0000

If editing Puppet code and Hiera directly with puppetmaster feels too easy, you can complicate the Puppet workflow as much you like by adding more components to the palette. Adding the so called control repository and r10k to Puppet environment maintenance instantly adds several phases to your initially simple workflow. There are also benefits that I’ll discuss later on.

The control repository (control repo for short) is a Git repository that is used to create dynamic Puppet environments on the Puppet master. Each Git branch can be turned into a matching Puppet environment. The name of the default branch, production, is the same as the environment the Puppet agents use by default. The content of the control repo branches look like this:

Using Puppet Bolt to apply roles to nodes, part 1

Sat, 17 Oct 2020 00:00:00 +0000

NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead.

Puppet Bolt is a designed to be an orchestration tool, but it can be used for configuration management as well. For example you may have a small environment of handful of nodes where having a dedicated Puppet server would be an overkill, or you have some nodes in the infrastructure where you can’t or don’t want to install a Puppet Agent.

Debugging Packer provisioning

Wed, 05 Aug 2020 00:00:00 +0000

Packer is an excellent tool for automating creation of machine images such as AWS AMIs or Docker container images. This is particularly useful when you need to scale up by spinning up instances on-demand or if you want to package software that is challenging to manage with a configuration management system. It is also extremely useful when creating actual packaged product for AWS Marketplace, for example.

In Packer you take a “base box” such as Ubuntu 18.04 base image and modify its contents with provisioning, for example to install and configure software on the image. Several different types of provisioners are supported, including but not limited to shell scripts, Puppet manifests and Ansible playbooks. The concept of provisioners is the same as in Vagrant and Terraform.

Puppet Bolt with a Puppet control repository

Mon, 30 Mar 2020 00:00:00 +0000

NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead.

Puppet Bolt is an orchestration tool similar to Ansible. It is suitable for multiple use-cases:

Running ad hoc commands on target nodes. We use this approach for testing Puppet feature branches before approving and merging them.
One-off configurations written in the Puppet language
Masterless Puppet environments (look here). You may not always have enough privileges, trust or will to manage a complete Puppet infrastructure including Puppetserver, PuppetDB and Puppet Agents.

Simple orchestration with Puppet Bolt

Mon, 30 Mar 2020 00:00:00 +0000

As discussed in the introductory post one of the use-cases for Puppet Bolt is orchestration - running actions on multiple targets in a certain order, possibly using results of actions or data from some targets on other targets. Orchestration with Bolt plans is quite easy: you just create a plan with multiple TargetSpec parameters, each representing a target or a type of target. A classic example is two or more web servers behind a load balancer. In this case we do something even simpler and just take a fact from a node called “first” and create a file on node “second” that uses the value of that fact in a file.

Managing Jenkins with Puppet, part 1: Basic installation

Fri, 31 Jan 2020 00:00:00 +0000

It seems every other organization is using Jenkins these days. Jenkins is a continuous integration and continuous delivery server that can be used to automate building, testing, and delivering or deploying software.

Many organizations also use Puppet for their configuration management needs. Puppet is, if not the de facto configuration management solution, at least one of the most mature and field tested ones. It consists of several server components, agents, a declarative programming language, amongst others. Puppet can be used to install, set up and manage large number of all kinds of resources.

Using Augeas to manage resources in Puppet

Fri, 31 Jan 2020 00:00:00 +0000

This translated from the original Finnish article that is available here.

The Augeas resource in Puppet allows one to manage parts of a complex configuration file instead of managing the whole file using a template or a static file. For trivial cases stdlib’s file_line resource may be sufficient. While use of templates or static files is in general more reliable and easier than use of Augeas, the latter is quite useful in many cases. As using Puppet Augeas resource directly is quite challenging, a useful practice is to first make the changes manually with “augtool” and only then convert the results into Puppet Augeas resources. We use PostgreSQL’s pg_hba.conf file here as an example.

Puppet rspec testing: understanding describe and context

Fri, 06 Dec 2019 00:00:00 +0000

In the Puppet module rspec testing with PDK blog post I covered the procedure of creating and running puppet-rspec tests with PDK. This blog post will go more in-depth to that topic, in particular describe and context which you see in all puppet-rspec tests.

When looking at the puppet-rspec test syntax the first time it looks very “rubyesque”. Somebody used to staring at Python code, for example, could call the syntax “odd” or “1337”. Fortunately reading the Ruby rspec documentation helps understand what is going on under the hood and thus making the spec files look less magical. In my case rspec documentation helped understand how contexts work. In the previous blog post a context was created for every supported operating system listed in metadata, and rspec tests were run for each of them by using facts from facterdb.

Migrating from Fabric 1 to Fabric 2

Thu, 10 Oct 2019 00:00:00 +0000

Fabric is a Python library for executing shell commands remotely over SSH in serial or parallel mode. I used Fabric 1 for years and it was - and still is - an excellent tool. While Fabric’s use-cases overlap somewhat with those of Ansible, the difference is that Fabric is not state-based. In other words, in Fabric you don’t tell that “file a should have contents n and mode m”. Instead, you just run commands on the target host which hopefully result in the end state you’re looking for. Or maybe you’re not even interested in end-state per se, but just need to run some commands. Both approaches have their use-cases. Fabric and Ansible are quite powerful as orchestration tools: they allow running task a on server x, then task b on server y. For enforcing infrastructure to be in a certain state we prefer Puppet and Terraform over Ansible.

Puppet module rspec testing with PDK

Thu, 05 Sep 2019 00:00:00 +0000

If you’ve searched for instructions on how to write unit tests for Puppet code you’ve probably stumbled upon tutorials such as this that are of good quality, but suggest installing and using tools such as puppetlabs_rspec_helper directly. Or you may encounter references to rake, bundler, rvm and rspec. Then you start experimenting these tools, some of which (rvm and bundler) are aimed at managing the dependency hell that arises when you, or your team, works simultaneously on multiple projects with different dependency requirements. In the end you are maybe able to run spec tests on your Puppet module, but after all installations of gems and experiments you’re unsure what you did to make everything work. You also don’t want to become a hard-core Ruby developer who actually understands all these tools well - you just want to test your Puppet code.