Fattening the workflow, part 5: Hiera and content encryption

Separating data from code in Puppet modules is advisable as it improves reusability of code. The separation can be accomplished with Hiera by having separate levels based on facts, organizational units, locations, etc.

Hiera can also be used for storing private data that needs to be protected and must not be readable by outsiders. Typically the data, like all other code, is stored in a version control system that may be located outside the organisation. In such a case Hiera encryption is a must.

The development of hiera-eyaml is an improvement to the usability compared to previous encryption backends: it allows encryption of just the desired data inside yaml files. Besides hiera-eyaml there are other options as well, like using hiera-vault for fetching secrets dynamically from Hashicorp Vault, but those are outside of the scope of this article.

First you'll need to set up hiera-yaml:

$ sudo gem install hiera-eyaml

This is the installation command for a Puppet server:

$ sudo puppetserver gem install hiera-eyaml

Hiera-eyaml uses asymmetric encryption, PKCS#7 being the default. So it needs a key pair for using it: one key, the public key is used for encryption, and the other, the private key, is used for decryption. The key pair is created by:

$ eyaml createkeys

The key pair is created to the keys sub folder of the current folder.

Puppetserver needs both the public and private key for decryption and reading the values. A good location to store the keys on Puppetserver is for example:

/etc/puppetlabs/puppet/eyaml

So let's create the folder and minimize the permissions:

$ mkdir /etc/puppetlabs/puppet/eyaml
$ chown -R puppet:puppet /etc/puppetlabs/puppet/eyaml
$ chmod -R 0500 /etc/puppetlabs/puppet/eyaml
$ chmod 0400 /etc/puppetlabs/puppet/eyaml/*.pem

Let's verify that the owners and permissions are what they're supposed to be:

$ ls -lha /etc/puppetlabs/puppet/eyaml

The owner should be "puppet", group owner "puppet" and read permission should be granted to the user "puppet".

If the user wants to edit the values in the existing Hiera file, typically in a copy of the control repository, both keys are needed. The keys need to be transferred to your own computer with a safe protocol, for example with scp.

In order to ease up the editing it's useful to create a new folder .eyaml in the home folder and in it a configuration file config.yaml which states where the keys will be located:

---
pkscs7_public_key: '/Users/tunnus/keys/public_key.pkcs7.pem'
pkcs7_private_key: '/Users/tunnus/keys/private_key.pkcs7.pem'

Make sure that you configure environment variable EYAML_CONFIG to point to that configuration file - otherwise hiera-eyaml might not be able to find them.

An alternative is to store the keys at the root of the control repo inside ./keys and add that directory to .gitignore. This works, but you need to launch all eyaml commands from the root of the repository or they will fail.

When the eyaml keys are on the server you can configure the Hiera configuration file to use these keys. The new Hiera version 5 has independent hierarchy settings for each environment and module. Let's suppose that the private key is called private_key.pkcs7.pem and the public key public_key.pkcs7.pem:

---
version: 5
defaults:
  datadir: data
hierarchy:
  - name: "Secret data"
    lookup_key: eyaml_lookup_key
    paths:
      - "nodes/%{trusted.certname}.eyaml"
      - "common.eyaml"
    options:
      pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
      pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
  - name: "Normal data"
    data_hash: yaml_data 
    paths:
      - ”nodes/%{trusted.certname}.yaml"
      - "common.yaml"

In *nix type operating systems it's possible to create an environment variable EDITOR that determines the desired default editor. The environmental variable can be placed in the shell initialization script (e.g.~/.bashrc). The editor can be given directly on the command line as well:

$ export EDITOR=emacs

When that environmental variable exists, the data sections of the eyaml file defined in Hiera be edited followingly:

$ eyaml edit common.eyaml

Eyaml opens the file and prints out helpful instructions. The value to be encrypted will be placed between square brackets: DEC::PKCS7[]!

#| This is eyaml edit mode. This text (lines starting with #| at the top of the
 #| file) will be removed when you save and exit.
 #| - To edit encrypted values, change the content of the DEC(<num>)::PKCS7[]!
 #| block.
 #| WARNING: DO NOT change the number in the parentheses.
 #| - To add a new encrypted value copy and paste a new block from the
 #| appropriate example below. Note that:
 #| * the text to encrypt goes in the square brackets
 #| * ensure you include the exclamation mark when you copy and paste
 #| * you must not include a number when adding a new block
 #| e.g. DEC::PKCS7[]!
 ---
 # This value will not be encrypted
 plaintextvalue: plaintext
 # This value will be encrypted
 encryptedvalue: DEC::PKCS7[encrypted]!

If we now look at the file, we can see that the previously define value is encrypted:

$ cat common.eyaml
 ---
 # This value will not be encrypted
 plaintextvalue: plaintext
 # This value will be encrypted
 encryptedvalue: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAK/YGRbR2qbgpxHxrCic6ywXG6x0w0hZksNQqJPBYT

Links:

• https://github.com/voxpupuli/hiera-eyaml
• https://puppet.com/blog/encrypt-your-data-using-hiera-eyaml

Fattening the workflow series:

• Part 1: The control repository
• Part 2: r10k
• Part 3: GitLab and similar
• Part 4: Roles and profiles
• Part 5: Hiera and content encryption