Packer QEMU: Building Ubuntu 20.04/22.04/24.04 VM images

A person closing a large cardboard box with tape.

Introduction

Packer QEMU builder allows you to build virtual machine images from scratch. You need a Packerfile, an automatic installation configuration file (autoinstall, kickstart or preseed) and the operating system installer (e.g. ISO). The benefit of Packer QEMU in particular are that you can have fine-grained control over the virtual machine image content such as disk partitioning. Moreover you don’t need to rely on official base images produced by other people, such as the official Ubuntu Cloud images.

This article focuses on building Ubuntu 20.04/22.04/24.04 images with Packer QEMU builder. However, even if you’re interested in building other operating systems such as Rocky Linux or Debian the same basic principles still apply. We assume that you run Packer on a Linux system due to the Linux-centric dependencies (libvirt and qemu). Although it seems possible to build QEMU images with Packer on MacOS, but at least on M1 ARM64 Macs the process is very, very far from straightforward.

Our video course about Packer QEMU builder

If you wish to save time you might be interested in our video course “Creating Cloud images from scratch with Packer and QEMU” which covers this topic in much more detail. Additionally you will learn how to build Ubuntu 24.04 and Rocky Linux 9 VM images from scratch and how to import them into AWS and Azure. The course students also get access to a Git repository with tested, ready-to-use code that makes the whole build and publishing process trivial.

How Packer QEMU builder works

Packer QEMU builder can use two types of sources:

  • Start from an installation ISO: do a true hands-free automated installation with autoinstall.

  • Start from an Ubuntu Cloud image: modify an existing disk image to suit your needs without using autoinstall

Because the former provides most flexibility in how things are setup, thus that is the approach we describe in this article.

Packer QEMU builder automates several parts of the operating system installation process:

  • Sends keystrokes over VNC to launch the installer similarly to what a human would do.

  • Servers the autoinstall file on its built-in HTTP server

It is your responsibility to tell Packer the correct keystrokes for the installer and to provide an autoinstall file that actually automates the installation. It is important to realize that Packer does not solve any automation challenges by itself - it only makes your work a lot easier.

Prerequisites for Packer QEMU builder

Packer QEMU builder depends on libvirt and qemu system packages. For the most part these are installed by default on modern Linux distributions. Additionally you need to install Packer and the Packer QEMU builder plugin.

Also the user running Packer needs additional permissions to create virtual machine images. In most cases running this command is enough:

usermod -a -G libvirt <your_username>

The relevant sections of the Packerfile

In this section we review the most important options in the Packerfile. At first we ensure that Packer knows that we want to launch an ISO-based installation instead of building on top of a disk image:

disk_image = false

Next we ensure that the temporary virtual machine Packer creates has enough memory to work:

memory = 1500

Rather surprisingly you might get strange errors almost immediately when the installer starts up should you use the default values.

Another key point is to wait a bit before letting Packer start typing:

boot_wait = "3s"

The correct keystrokes to run over VNC depend on the operating system. Here’s an example for Ubuntu 24.04:

  boot_command         = [
    "e<wait>",
    "<down><down><down><end>",
    " autoinstall ds=\"nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/\" ",
    "<f10>"
    ]

The “ds” line tells the Ubuntu installer where to find the HTTP server serving user-data and meta-data. The parameters in curly braces are automatically populated by Packer with its built-in webserver’s IP and port. In order to make file serving work you need to define the webroot for Packer’s internal web server:

http_directory = "http"

Secondly you need to create the http directory next to your Packerfile:

mkdir -p http

Now, when you add a meta-data and user-data files to the http directory the Ubuntu installer will find and use them. To begin with empty files should be sufficient for an automated Ubuntu installation with all default values.

A successful automated installation does not, however, guarantee that Packer’s SSH provisioning will work. Take the following Packer SSH settings as an example:

ssh_username = "packer"
ssh_password = "packer"
ssh_timeout  = "60m"

This example assumes that password authentication is enabled and that a user called packer is present on the system. However, this is not the case by default. Luckily you can add configurations to your user-data (autoinstall) file to fix that.

Creating the user-data file

The user-data file is an Ubuntu autoinstall file. To get started you have two options:

  • Write the user-data file from scratch using the autoinstall reference as a guide

  • Install Ubuntu manually and use the generated user data file (/var/log/installer/autoinstall-user-data) as a basis for your own, customized/generalized user data file.

For Packer provisioners we create a packer user with the identity module:

#cloud-config
autoinstall:
  identity:
    hostname: ubuntu
    password: <password hash copied from /etc/shadow> 
    realname: packer
    username: packer

You can create a password hash by various means. For example, you can create a new user on an existing system and then get the password hash from /etc/shadow as root.

We also need to ensure that SSH server is installed and enabled and allows password logins:

  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: true

This should be enough for running Packer provisioners. Check the autoinstall reference to see what other things you can configure with autoinstall.

Sample files

Ubuntu 24.04: Packerfile

source "qemu" "iso" {
  vm_name              = "ubuntu-2404-amd64.raw"
  iso_url              = "https://www.releases.ubuntu.com/24.04/ubuntu-24.04-live-server-amd64.iso"
  iso_checksum         = "8762f7e74e4d64d72fceb5f70682e6b069932deedb4949c6975d0f0fe0a91be3"
  memory               = 1500
  disk_image           = false
  output_directory     = "build/os-base"
  accelerator          = "kvm"
  disk_size            = "12000M"
  disk_interface       = "virtio"
  format               = "raw"
  net_device           = "virtio-net"
  boot_wait            = "3s"
  boot_command         = [
    "e<wait>",
    "<down><down><down><end>",
    " autoinstall ds=\"nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/\" ",
    "<f10>"
    ]
  http_directory       = "http"
  shutdown_command     = "echo 'packer' | sudo -S shutdown -P now"
  ssh_username         = "packer"
  ssh_password         = "packer"
  ssh_timeout          = "60m"
}

build {
  name    = "iso"
  sources = ["source.qemu.iso"]
}

Ubuntu 24.04: autoinstall file

#cloud-config
autoinstall:
  apt:
    disable_components: []
    fallback: abort
    geoip: true
    mirror-selection:
      primary:
      - country-mirror
      - arches: &id001
        - amd64
        - i386
        uri: http://archive.ubuntu.com/ubuntu/
      - arches: &id002
        - s390x
        - arm64
        - armhf
        - powerpc
        - ppc64el
        - riscv64
        uri: http://ports.ubuntu.com/ubuntu-ports
    preserve_sources_list: false
    security:
    - arches: *id001
      uri: http://security.ubuntu.com/ubuntu/
    - arches: *id002
      uri: http://ports.ubuntu.com/ubuntu-ports
  codecs:
    install: false
  drivers:
    install: false
  identity:
    hostname: ubuntu
    password: <password hash>
    realname: packer
    username: packer
  kernel:
    package: linux-generic
  keyboard:
    layout: us
    toggle: null
    variant: ''
  locale: en_US.UTF-8
  source:
    id: ubuntu-server-minimal
    search_drivers: false
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: true
  storage:
    layout:
      name: direct
  updates: security
  version: 1

Ubuntu 22.04: Packerfile

packer {
  required_version = "= 1.11.1"
  required_plugins {
    qemu = {
      version = "= 1.1.0"
      source = "github.com/hashicorp/qemu"
    }
  }
}

source "qemu" "iso" {
  vm_name              = "ubuntu-2204-amd64.raw"
  iso_url              = "https://www.releases.ubuntu.com/22.04/ubuntu-22.04.4-live-server-amd64.iso"
  iso_checksum         = "45f873de9f8cb637345d6e66a583762730bbea30277ef7b32c9c3bd6700a32b2"
  memory               = 1500
  disk_image           = false
  output_directory     = "build/os-base"
  accelerator          = "kvm"
  disk_size            = "8000M"
  disk_interface       = "virtio"
  format               = "raw"
  net_device           = "virtio-net"
  boot_wait            = "3s"
  boot_command         = [
    "e<wait>",
    "<down><down><down><end>",
    " autoinstall ds=\"nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/\" ",
    "<f10>"
    ]
  http_directory       = "http"
  shutdown_command     = "echo 'packer' | sudo -S shutdown -P now"
  ssh_username         = "packer"
  ssh_password         = "packer"
  ssh_timeout          = "60m"
}

build {
  name = "iso"
}

Ubuntu 22.04: user data

#cloud-config
autoinstall:
  apt:
    disable_components: []
    fallback: abort
    geoip: true
    mirror-selection:
      primary:
      - country-mirror
      - arches: &id001
        - amd64
        - i386
        uri: http://archive.ubuntu.com/ubuntu/
      - arches: &id002
        - s390x
        - arm64
        - armhf
        - powerpc
        - ppc64el
        - riscv64
        uri: http://ports.ubuntu.com/ubuntu-ports
    preserve_sources_list: false
    security:
    - arches: *id001
      uri: http://security.ubuntu.com/ubuntu/
    - arches: *id002
      uri: http://ports.ubuntu.com/ubuntu-ports
  codecs:
    install: false
  drivers:
    install: false
  identity:
    hostname: ubuntu
    password: <password hash> 
    realname: packer
    username: packer
  kernel:
    package: linux-generic
  keyboard:
    layout: us
    toggle: null
    variant: ''
  locale: en_US.UTF-8
  source:
    id: ubuntu-server-minimal
    search_drivers: false
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: true
  storage:
    layout:
      name: direct
  updates: security
  version: 1

Ubuntu 20.04: Packerfile

source "qemu" "iso" {
  vm_name              = "ubuntu-2004-amd64-iso.qcow2"
  iso_url              = "https://www.releases.ubuntu.com/20.04/ubuntu-20.04.6-live-server-amd64.iso"
  iso_checksum         = "sha256:b8f31413336b9393ad5d8ef0282717b2ab19f007df2e9ed5196c13d8f9153c8b"
  memory               = 1280
  disk_image           = false
  output_directory     = "build/iso"
  accelerator          = "kvm"
  disk_size            = "12000M"
  disk_interface       = "virtio"
  format               = "qcow2"
  net_device           = "virtio-net"
  boot_wait            = "3s"
  boot_command         = [
    # Make the language selector appear...
    " <up><wait>",
    # ...then get rid of it
    " <up><wait><esc><wait>",

    # Go to the other installation options menu and leave it
    "<f6><wait><esc><wait>",

    # Remove the kernel command-line that already exists
    "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
    "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
    "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",

    # Add kernel command-line and start install
    "/casper/vmlinuz ",
    "initrd=/casper/initrd ",
    "autoinstall ",
    "ds=nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/ ",
    "<enter>"
    ]
  http_directory       = "http"
  shutdown_command     = "echo 'packer' | sudo -S shutdown -P now"
  # These are required or Packer will panic, even if no provisioners are not
  # configured
  ssh_username         = "packer"
  ssh_password         = "packer"
  ssh_timeout          = "60m"
}

build {
  name = "iso"

  sources = ["source.qemu.iso"]
}

Ubuntu 20.04: user data

#cloud-config
autoinstall:
  version: 1
  # Fetch latest version of autoinstall before starting
  refresh-installer:
    update: true
  # Prevent packer from connecting to the _installer_ with SSH and trying to
  # start provisioning.
  #
  # https://tlhakhan.medium.com/ubuntu-server-20-04-autoinstall-2e5f772b655a
  #
  early-commands:
    - "systemctl stop ssh"
  locale: en_US.UTF-8
  keyboard:
    layout: us
  apt:
    preserve_sources_list: false
    primary:
        - arches: [i386, amd64]
          uri: "http://archive.ubuntu.com/ubuntu"
        - arches: [default]
          uri: "http://ports.ubuntu.com/ubuntu-ports"
    geoip: true
  storage:
    layout:
      name: direct
  identity:
    hostname: ubuntu
    username: packer
    password: <password-hash> 
  ssh:
    allow-pw: true
    install-server: true
  late-commands:
    - "echo 'packer ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/packer"
    - "chmod 440 /target/etc/sudoers.d/packer"
  user-data:
    timezone: UTC
    disable_root: true

Launching the build

Assuming your system is set up correctly you can build with just:

packer build

If you split your Packer configuration into multiple .pkr.hcl file you need to place all the files into a directory and use

packer build <directory>

Tips

Using custom cloud-init configuration with autoinstall

Autoinstall uses the cloud-init file format. You must place the autoinstall configurations go under the _a_utoinstall top-level key. You can place additional cloud-init modules under the user-data subkey. Here’s a simple example that shows how to enable the root user with highly insecure password and to set the timezone:

#cloud-config
autoinstall:
  user-data:
    timezone: Europe/Helsinki
    disable_root: false
    chpasswd:
      list: |
        root:secret

Anything you can do with cloud-init should be possible to do from the user-data section. When you boot the system the first time cloud-init runs and applies these changes.Therefore you won’t see any changes from cloud-init on the disk image.

Mounting qcow2 images

You can mount the disk images created by Packer. This technique is very useful whenever you need to debug your autoinstall file. Make sure that you have libguestfs-tools or equivalent package installed, then as root do something like this:

LIBGUESTFS_BACKEND=direct guestmount -a output/ubuntu-2404-amd64-qemu-build -m /dev/sda2 /mnt/guestmount/

If you give guestmount an invalid partition it will helpfullly print out a list of valid partition names:

guestmount: ‘/dev/sda3’ could not be mounted.
guestmount: Did you mean to mount one of these filesystems?
guestmount:     /dev/sda1 (unknown)
guestmount:     /dev/sda2 (ext4)
guestmount:     /dev/ubuntu-vg/ubuntu-lv (ext4)

Guestmount supports logical volumes as well, but debugging is easier with normal partitions. To unmount a guestmounted partition:

umount /mnt/guestmount

Note that making modifications to the mounted disk image will change the image checksum. This may be a problem if subsequent builds assume a certain image checksum.

Using two-stage builds to save time

Provisioning operating systems is a very slow process and autoinstall is no exception. In the Packer context you’re doing two things:

  1. Automating the installation of the operating system

  2. Running provisioning scripts

You definitely don’t want to run the entire operating system installation when making changes to the provisioning scripts. If you did, you’d waste 10 minutes of your time on every try. To alleviate this problem you can split your Packer builds into two phases. For details please have a look at our Two-stage Qemu builds with Packer blog post.

For further information please refer to the official documentation:

These external web pages were used as a source: