The main annoyance with agent-based configuration management is the management of the agents themselves. They usually don't require any handholding after the initial install, except when you need to migrate them to a new environment. For example, migrate from a Puppet 5-based server to Puppet 6. The migration process is not overly difficult, but in […]
When moving from older versions of Terraform 0.12.x to latest 0.12.x (now: 0.12.29) you may notice the following warnings: These make sense, but may come as a surprise as earlier 0.12.x versions were perfectly fine with them. You may have tons of these, so fixing them manually gets tiresome. The fix is not too difficult, […]
Writing static facts is fairly easy even with a low level of Ruby skills. Below is an example of a fact that returns true or false depending on whether the node has a /boot partition: As you can see above the required information was already inside the "mountpoints" fact of Facter. Using the fact is […]
Do you want to work on the front line of IT? Do you want to use the tools and methods of tomorrow, before the masses do? Read on! Puppeteers Oy is a small, growing IT company that focuses on building modern IT infrastructure for organizations. We’ve got a long experience in IT, especially in DevOps […]
We recently hit an interesting issue with an "almost in production" Keycloak high-availability clustered domain mode setup that was behind an Amazon EC2 load balancer with sticky sessions enabled. The symptom was that changes from the domain controller did not propagate to the slave. This included: Changes made to realms and clients using the Admin […]
Our company is about infrastructure automation and products built on top of it. That means we really don't want to do manually what a computer can do. Therefore it was not an option to publish our blog posts first in WordPress, then copy-and-paste them to LinkedIn. Rather, let a WordPress plugin configured as a LinkedIn […]
We - like many others - have the problem that our work gets easily interrupted. Some interruptions come from the inside. For example, you remember "a thing you need to do" and without thinking start doing it immediately. Other interruptions are external, for example a colleague ask you for advise or a customer calls you […]
Keycloak is an excellent Open Source Identity and Access Management solution that builds on top of the Wildfly application server. We manage several Keycloak installations for providing SSO with FreeIPA credentials for self-hosted and SaaS services via SAML and OIDC. To keep our Keycloak configurations healthy and in a known-good state we manage their configurations […]
Here's the next episode in our Duplo visualization series, have a look! This time we put Puppet agents to work. Featuring: Puppet agents will retrieve their desired state definition (catalog) from Puppet master for the Puppet environment they're in. Puppet agent reads the catalog: Puppet agent compares the current state to the catalog and notices […]
We maintain a Prometheus and Alertmanager setup where both services are running on the same host and an Apache reverse proxy handles TLS and authentication, authorization and single sign-on with Apache mod_auth_mellon configured as a Keycloak SAML client. This way we can limit access to Prometheus and Alertmanager to people who belong to a certain […]
Separating data from code in Puppet modules is advisable as it improves reusability of code. The separation can be accomplished with Hiera by having separate levels based on facts, organizational units, locations, etc. Hiera can also be used for storing private data that needs to be protected and must not be readable by outsiders. Typically […]
If the workflow that includes the control repository, r10k and GitLab still feels too light, there are yet more ways to make the workflow heavier by adding more abstraction to the Puppet modules themselves. This is done by adopting the "Roles and Profiles pattern" that was developed by Craig Dunn and popularized in the blog […]
We've been having DIY visualization sessions on Puppet and sysadmin related topics and we wanted to share some of them with you. We had a lot of fun doing this, hope you'll enjoy this even a bit as much as we did! Let me introduce the characters involved in the first chapter: R10k is started […]
In the previous posts of this series I discussed the control repository and r10k. The last component in fattening the workflow is Git provider such as GitLab, GitHub or Bitbucket. Nowadays all of them are reasonable choices for storing private Git modules, which Puppet control repositories and site-specific modules tend to be. One of the […]
In the previous post I discussed the control repository structure. In this post I'll talk about r10k. That tool is used for deploying control repository branches into matching Puppet environments on the Puppet server. Installing r10k is very straightforward with the bundled gem: After this you'll need a settings file for r10k, /etc/puppetlabs/r10k/r10k.yaml: Several requirements […]
If editing Puppet code and Hiera directly with puppetmaster feels too easy, you can complicate the Puppet workflow as much you like by adding more components to the palette. Adding the so called control repository and r10k to Puppet environment maintenance instantly adds several phases to your initially simple workflow. There are also benefits that […]
Introduction The purpose of this post is to show that search engine marketing (SEM) can be started rather easily. Small businesses, like our company, may not have resources (money, knowledge, time etc) to spend on marketing and consultancy so I'm hoping that sharing this experience will help and encourage other SEM newbies! This blog post […]
Update: the problem described in this article seems to be resolved in Fedora 36 (NetworkManager 1.38.0-2.fc36). It is still present in Fedora 35 (NetworkManager 1.32.12-2.fc35). Update: there a companion article about solving this problem for Wireguard in here. In an the Manage Linux client DNS settings in OpenVPN and Wireguard article we described how to […]
We maintain a rather complex AWS environment built with Terraform that uses nested modules and lots of per-resource provider configurations. Recently the pain of staying with Terraform 0.11 became too much, so we had to start the migration to Terraform 0.12, with 0.13 going to follow soon after. In general the official upgrade instructions were […]
NOTE: this article is somewhat outdated. Please refer to Serverless Puppet with control repo, Hiera, roles and profiles and Puppet Bolt instead. Puppet Bolt is a designed to be an orchestration tool, but it can be used for configuration management as well. For example you may have a small environment of handful of nodes where […]
