Data-driven Terraform: Kubernetes cluster in Hetzner Cloud

April 15, 2022

Terraform does not have a particularly strong decoupling between data and code, at least not from a best practices perspective. It is possible and useful, however, to use data to define Terraform resources - if not for any other reason but to reduce code repetition for common resources that require defining lots of parameters. Here’s an example of setting up a Kubernetes cluster in Hetzner Cloud using Terraform with a “data-driven” approach:

variable "k8s_instances" {
  type = map
  default = { "k8s-master-01.example.org" = { ip = "10.41.86.101", server_type = "cx21", worker = false },
              "k8s-master-02.example.org" = { ip = "10.41.86.102", server_type = "cx21", worker = false },
              "k8s-master-03.example.org" = { ip = "10.41.86.103", server_type = "cx21", worker = false },
              "k8s-worker-01.example.org" = { ip = "10.41.86.111", server_type = "cx31", worker = true  },
              "k8s-worker-02.example.org" = { ip = "10.41.86.112", server_type = "cx31", worker = true  },
              "k8s-worker-03.example.org" = { ip = "10.41.86.113", server_type = "cx31", worker = true  }
            }
}

locals {
  k8s_master_firewall_ids = [hcloud_firewall.standard.id]
  k8s_worker_firewall_ids = [hcloud_firewall.standard.id, hcloud_firewall.k8s_common.id, hcloud_firewall.k8s_worker.id]
}

module "k8s_instance" {
  for_each = var.k8s_instances

  source          = "github.com/Puppet-Finland/terraform-hcloud_server_wrapper?ref=1.0.0"
  hostname        = each.key
  image           = "ubuntu-20.04"
  puppetmaster_ip = var.puppet7_example_org_private_ip
  ssh_keys        = [ data.hcloud_ssh_key.johndoe.id ]
  backups         = "true"
  server_type     = each.value["server_type"]
  floating_ip     = "false"
  firewall_ids    = each.value["worker"] ? local.k8s_worker_firewall_ids : local.k8s_master_firewall_ids
}

resource "hcloud_server_network" "k8s_network" {
  for_each = var.k8s_instances

  server_id  = module.k8s_instance[each.key].id
  network_id = hcloud_network.example_org.id
  ip         = each.value["ip"]
}

resource "hcloud_firewall" "k8s_common" {
  name = "k8s-common"

  dynamic "rule" {
    for_each = var.k8s_instances

    content {
      description = "Any TCP from ${rule.key}"
      direction   = "in"
      protocol    = "tcp"
      port        = "any"
      source_ips  = ["${rule.value["ip"]}/32"]
    }
  }
}

resource "hcloud_firewall" "k8s_worker" {
  name = "k8s-worker"
  rule {
    description = "HTTPS from anywhere"
    direction   = "in"
    protocol    = "tcp"
    port        = 443
    source_ips  = [
      "0.0.0.0/0",
      "::/0"
    ]
  }
}

The code above does two main things:

Creates all the Kubernetes cluster nodes (three masters, three workers) using a single “hcloud_server” resource.
Opens up holes in each cluster node’s Hetzner Cloud firewall for each other cluster node to allow intra-cluster communication.

The looping of data is based on for_each function, which is used to for creating multiple resources on one go from a map, as well as for creating dynamic embedded blocks inside a single resource definition.

This same approach can be used for other purposes. Note that if you’re refactoring code then you will need to move resources in the Terraform state file after implementing this kind of data-driven approach, as resource paths will change.

Deep-Dive

Terraform