Overview of Ansible quality assurance
Ansible is an IT automation engine which you can use for configuration management, orchestration and device management, among other things. While you can get started fast with Ansible, ensuring high-quality, bug-free code can be challenging. Moreover, there's not that much official, high-quality or coherent documentation available on Ansible quality assurance best practices. While low hanging fruit like Ansible variable validation are available, they are not emphasized in official documentation.
This is the first part of our "Ansible quality assurance" series of blog posts.
Why you should do Ansible variable validation?
Here in part 1 we cover validation of variables. In particular, we focus on variable validation in Ansible roles, although the same approach works anywhere. Variable validation helps avoid playbook failures and hard to debug runtime errors and side-effects caused by
- Undefined variables
- Invalid variable values
Ansible variable validation with ansible.builtin.assert
Ansible does not a have built-in data types. As a result, you need to construct assertions manually. This is contrast to typed languages like Puppet where data types are first class citizens. The main tool you can use for Ansible variable validation is the ansible.builtin.assert module which builds on top of Jinja2 tests and filters.
Fail as early as possible
You can minimize misconfigurations by failing as early as possible. Typically misconfigurations are caused by missing or wrongly time variable validation and fall into two categories
- Partial configurations: failure happens after Ansible has already executed some tasks. Sometimes you may have to do cleanup if this happens. For example, if your tasks are not idempotent, you may not be able to run them twice in a row without side-effects.
- Hidden misconfigurations: this happens when Ansible thinks everything is ok, but a wrong (or missing) variable value ended up in, say, a configuration file. These can be very tricky to debug afterwards.
For this reason you should not only fail on invalid variables, but fail as early as possible. Correct location for Ansible variable validation depends on your use-case:
- Roles: top of <role>/tasks/main.yml
- (Orchestration) playbooks: top of the playbook
This is why you should not in most cases rely on variable validation done at task execution time.
Additionally you can avoid late failures by preferring imports over includes. When you Import a role or task it is added to your code statically. In other words, importing is about the same as if you had copied-and-pasted the imported role or task into your own code. A positive side-effect of an import is that you can validate variables before the Ansible run starts. In contrast includes are evaluated at runtime when the playbook is already running, so missing/invalid variables may go unnoticed until they cause a failure.
If you want to learn more about includes and imports please refer to official Ansible documentation.
Minimal Ansible variable validation: is the variable defined?
The minimal check you should do for every variable is to check for variable presence. We focus on roles, of which here is an example:
- ansible.builtin.include_role:
name: myrole
vars:
myrole_myvar: foobarThe myrole/tasks/main.yml has the assert(s) at top. You should at minimum check that you've set all the variables a role needs:
---
- name: validate parameters
ansible.builtin.assert:
that:
- myrole_myvar is definedNow if you forget to pass a value to the role Ansible will error out immediately with a reasonable error message. This is definitely progress, but it is far from optimal. As you can see, you could still pass, for example, "foobar" instead of an IP address.
Validating that a variable is of certain type
Validating that a variable is of certain type is one step up from the "is variable defined" check:
- name: validate parameters
ansible.builtin.assert:
that:
- my_string is string
- my_integer is number
- my_float is float
- my_boolean is booleanQuite confusingly validating dictionaries and lists requires testing "features" of the variable's datatype:
- name: validate dictionaries and lists
ansible.builtin.assert:
that:
- my_dictionary is not string and my_dictionary is iterable and my_dictionary is mapping
- my_list is not string and my_list is not mapping and my_list is iterableMore details are available here.
Validating that a variable belongs to a predefined set
For string variables that take a limited set of values using a regular expression matches are very useful:
- name: validate parameters
ansible.builtin.assert:
that:
- myrole_state is match ('^(present|absent)$')Validating that a variable matches a value
Normally checking that a variable matches a hardcoded value makes little sense. However, it can be useful when combined with boolean operators. For example, you may want to validate that a variable either matches a domain name or has a special value of 'none':
- name: Validate DNS domain name
ansible.builtin.assert:
that: domain_name is match ('^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\\.)[A-Za-z]{2,6}') or domain_Name == 'none'This is particularly useful when you can't just pass an empty string. One example is set_fact which chokes if you try to create a list with empty values in it.
Regular expressions for complex string validation
Regular expressions also help you validate DNS names, for example like this:
- name: validate parameters
ansible.builtin.assert:
that:
- myrole_dnsname is match ('^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\\.)[A-Za-z]{2,6}')As a regular expression grows more complex, the likelihood of it being buggy grows.
Validating numeric values
Checking numeric values like port numbers is easy:
- name: validate parameters
ansible.builtin.assert:
that:
- myrole_port >= 1 and myrole_port <= 65535Doing asserts on multiple variable values in one place
Validating multiple asserts in one place is trivial as well:
---
- name: validate parameters
ansible.builtin.assert:
that:
- myrole_state is match ('^(present|absent)$')
- myrole_dnsname is match ('^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\\.)[A-Za-z]{2,6}')
- myrole_port >= 1 and myrole_port <= 65535Validating multiple variables in a loop
Sometimes you have a large number of variables that require the same set of complex validation rules. In that case using a loop saves code and reduces code repetition. Here is an example where we validate a list of domain names with a regular expression pattern:
- name: Create list of DNS domain parameters for validation
ansible.builtin.set_fact:
dns_domains:
- foo.example.org
- bar.example.org
- baz.example.org
- name: Validate DNS domains
ansible.builtin.assert:
that: dns_domain is match ('^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\\.)[A-Za-z]{2,6}')
loop: "{{ dns_domains }}"
loop_control:
loop_var: dns_domainValidating list length
If you expect a list variable to be of certain length you can use the following assert:
- name: Assert that list length is 1
assert:
that:
- mylist|length == 1Validating task results
The validation strategies I demonstrated above are focused on input validation. When you develop Ansible modules you should do integration testing and in that context it is very important to test task outputs as well. When an Ansible module has finished running, it returns a dictionary which can be registered as a variable and inspected with ansible.builtin.assert. The exact format of the dictionary varies module by module, but there are some common dictionary keys that have special significance - reserved words of sorts.
Here is an example from one of our Ansible modules, keycloak_authz_permission, on how you can use asserts to validate task results using task return values. The task definition looks like this:
- name: Create scope permission
community.general.keycloak_authz_permission:
auth_keycloak_url: "{{ url }}"
auth_realm: "{{ admin_realm }}"
auth_username: "{{ admin_user }}"
auth_password: "{{ admin_password }}"
state: present
name: "ScopePermission"
description: "Scope permission"
permission_type: scope
scopes:
- "file:delete"
policies:
- "Default Policy"
client_id: "{{ client_id }}"
realm: "{{ realm }}"
register: resultThe task returns a dictionary like this:
TASK [keycloak_authz_permission : Create scope permission] ********************* changed: [testhost] =>
{
"changed": true,
"end_state": {
"decisionStrategy": "UNANIMOUS",
"description": "Scope permission",
"logic": "POSITIVE",
"name": "ScopePermission",
"policies": ["570d477f-ac97-4c7b-8b9f-ae91454cc22d"],
"resources": [],
"scopes": ["ebbc7ca8-8f62-4822-87a6-6a05b5efb5d2"],
"type": "scope"
},
"msg": "Permission created",
} To verify the results we can register the returned dictionary into variable and check its contents like this:
- name: Assert that scope permission was created
assert:
that:
- result is changed
- result.end_state != {}
- result.end_state.name == "ScopePermission"
- result.end_state.description == "Scope permission"
- result.end_state.type == "scope"
- result.end_state.resources == []
- len(result.end_state.policies) == 1
- len(result.end_state.scopes) == 1If there is a discrepancy between the task definition and the actual result object the assert will thus fail. While this is probably most useful for writing tests, it can definitely be used in normal playbooks as well.
More on Ansible quality assurance from Puppeteers
- Ansible variable validation with ansible.utils.assert
- Version locking Ansible Collections
- Writing Ansible modules: when and why?
