Introduction
Packer QEMU builder allows you to build virtual machine images from scratch. You need a Packerfile, an automatic installation configuration file (autoinstall, kickstart or preseed) and the operating system installer (e.g. ISO). The benefit of Packer QEMU in particular are that you can have fine-grained control over the virtual machine image content such as disk partitioning. Moreover you don't need to rely on official base images produced by other people, such as the official Ubuntu Cloud images.
This article focuses on building Ubuntu 20.04/22.04/24.04 images with Packer QEMU builder. However, even if you're interested in building other operating systems such as Rocky Linux or Debian the same basic principles still apply. We assume that you run Packer on a Linux system due to the Linux-centric dependencies (libvirt and qemu). Although it seems possible to build QEMU images with Packer on MacOS, but at least on M1 ARM64 Macs the process is very, very far from straightforward.
Our video course about Packer QEMU builder
If you wish to save time you might be interested in our video course "Creating Cloud images from scratch with Packer and QEMU" which covers this topic in much more detail. Additionally you will learn how to build Ubuntu 24.04 and Rocky Linux 9 VM images from scratch and how to import them into AWS and Azure. The course students also get access to a Git repository with tested, ready-to-use code that makes the whole build and publishing process trivial.
How Packer QEMU builder works
Packer QEMU builder can use two types of sources:
- Start from an installation ISO: do a true hands-free automated installation with autoinstall.
- Start from an Ubuntu Cloud image: modify an existing disk image to suit your needs without using autoinstall
Because the former provides most flexibility in how things are setup, thus that is the approach we describe in this article.
Packer QEMU builder automates several parts of the operating system installation process:
- Sends keystrokes over VNC to launch the installer similarly to what a human would do.
- Servers the autoinstall file on its built-in HTTP server
It is your responsibility to tell Packer the correct keystrokes for the installer and to provide an autoinstall file that actually automates the installation. It is important to realize that Packer does not solve any automation challenges by itself - it only makes your work a lot easier.
Prerequisites for Packer QEMU builder
Packer QEMU builder depends on libvirt and qemu system packages. For the most part these are installed by default on modern Linux distributions. Additionally you need to install Packer and the Packer QEMU builder plugin.
Also the user running Packer needs additional permissions to create virtual machine images. In most cases running this command is enough:
usermod -a -G libvirt <your_username>
The relevant sections of the Packerfile
In this section we review the most important options in the Packerfile. At first we ensure that Packer knows that we want to launch an ISO-based installation instead of building on top of a disk image:
disk_image = false
Next we ensure that the temporary virtual machine Packer creates has enough memory to work:
memory = 1500
Rather surprisingly you might get strange errors almost immediately when the installer starts up should you use the default values.
Another key point is to wait a bit before letting Packer start typing:
boot_wait = "3s"
The correct keystrokes to run over VNC depend on the operating system. Here's an example for Ubuntu 24.04:
boot_command = [
"e<wait>",
"<down><down><down><end>",
" autoinstall ds=\"nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/\" ",
"<f10>"
]
The "ds" line tells the Ubuntu installer where to find the HTTP server serving user-data and meta-data. The parameters in curly braces are automatically populated by Packer with its built-in webserver's IP and port. In order to make file serving work you need to define the webroot for Packer's internal web server:
http_directory = "http"
Secondly you need to create the http directory next to your Packerfile:
mkdir -p http
Now, when you add a meta-data and user-data files to the http directory the Ubuntu installer will find and use them. To begin with empty files should be sufficient for an automated Ubuntu installation with all default values.
A successful automated installation does not, however, guarantee that Packer's SSH provisioning will work. Take the following Packer SSH settings as an example:
ssh_username = "packer"
ssh_password = "packer"
ssh_timeout = "60m"
This example assumes that password authentication is enabled and that a user called packer is present on the system. However, this is not the case by default. Luckily you can add configurations to your user-data (autoinstall) file to fix that.
Creating the user-data file
The user-data file is an Ubuntu autoinstall file. To get started you have two options:
- Write the user-data file from scratch using the autoinstall reference as a guide
- Install Ubuntu manually and use the generated user data file (/var/log/installer/autoinstall-user-data) as a basis for your own, customized/generalized user data file.
For Packer provisioners we create a packer user with the identity module:
#cloud-config
autoinstall:
identity:
hostname: ubuntu
password: <password hash copied from /etc/shadow>
realname: packer
username: packer
You can create a password hash by various means. For example, you can create a new user on an existing system and then get the password hash from /etc/shadow as root.
We also need to ensure that SSH server is installed and enabled and allows password logins:
ssh:
allow-pw: true
authorized-keys: []
install-server: true
This should be enough for running Packer provisioners. Check the autoinstall reference to see what other things you can configure with autoinstall.
Sample files
Ubuntu 24.04: Packerfile
source "qemu" "iso" {
vm_name = "ubuntu-2404-amd64.raw"
iso_url = "https://www.releases.ubuntu.com/24.04/ubuntu-24.04-live-server-amd64.iso"
iso_checksum = "8762f7e74e4d64d72fceb5f70682e6b069932deedb4949c6975d0f0fe0a91be3"
memory = 1500
disk_image = false
output_directory = "build/os-base"
accelerator = "kvm"
disk_size = "12000M"
disk_interface = "virtio"
format = "raw"
net_device = "virtio-net"
boot_wait = "3s"
boot_command = [
"e<wait>",
"<down><down><down><end>",
" autoinstall ds=\"nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/\" ",
"<f10>"
]
http_directory = "http"
shutdown_command = "echo 'packer' | sudo -S shutdown -P now"
ssh_username = "packer"
ssh_password = "packer"
ssh_timeout = "60m"
}
build {
name = "iso"
sources = ["source.qemu.iso"]
}
Ubuntu 24.04: autoinstall file
#cloud-config
autoinstall:
apt:
disable_components: []
fallback: abort
geoip: true
mirror-selection:
primary:
- country-mirror
- arches: &id001
- amd64
- i386
uri: http://archive.ubuntu.com/ubuntu/
- arches: &id002
- s390x
- arm64
- armhf
- powerpc
- ppc64el
- riscv64
uri: http://ports.ubuntu.com/ubuntu-ports
preserve_sources_list: false
security:
- arches: *id001
uri: http://security.ubuntu.com/ubuntu/
- arches: *id002
uri: http://ports.ubuntu.com/ubuntu-ports
codecs:
install: false
drivers:
install: false
identity:
hostname: ubuntu
password: <password hash>
realname: packer
username: packer
kernel:
package: linux-generic
keyboard:
layout: us
toggle: null
variant: ''
locale: en_US.UTF-8
source:
id: ubuntu-server-minimal
search_drivers: false
ssh:
allow-pw: true
authorized-keys: []
install-server: true
storage:
layout:
name: direct
updates: security
version: 1
Ubuntu 22.04: Packerfile
packer {
required_version = "= 1.11.1"
required_plugins {
qemu = {
version = "= 1.1.0"
source = "github.com/hashicorp/qemu"
}
}
}
source "qemu" "iso" {
vm_name = "ubuntu-2204-amd64.raw"
iso_url = "https://www.releases.ubuntu.com/22.04/ubuntu-22.04.4-live-server-amd64.iso"
iso_checksum = "45f873de9f8cb637345d6e66a583762730bbea30277ef7b32c9c3bd6700a32b2"
memory = 1500
disk_image = false
output_directory = "build/os-base"
accelerator = "kvm"
disk_size = "8000M"
disk_interface = "virtio"
format = "raw"
net_device = "virtio-net"
boot_wait = "3s"
boot_command = [
"e<wait>",
"<down><down><down><end>",
" autoinstall ds=\"nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/\" ",
"<f10>"
]
http_directory = "http"
shutdown_command = "echo 'packer' | sudo -S shutdown -P now"
ssh_username = "packer"
ssh_password = "packer"
ssh_timeout = "60m"
}
build {
name = "iso"
}
Ubuntu 22.04: user data
#cloud-config
autoinstall:
apt:
disable_components: []
fallback: abort
geoip: true
mirror-selection:
primary:
- country-mirror
- arches: &id001
- amd64
- i386
uri: http://archive.ubuntu.com/ubuntu/
- arches: &id002
- s390x
- arm64
- armhf
- powerpc
- ppc64el
- riscv64
uri: http://ports.ubuntu.com/ubuntu-ports
preserve_sources_list: false
security:
- arches: *id001
uri: http://security.ubuntu.com/ubuntu/
- arches: *id002
uri: http://ports.ubuntu.com/ubuntu-ports
codecs:
install: false
drivers:
install: false
identity:
hostname: ubuntu
password: <password hash>
realname: packer
username: packer
kernel:
package: linux-generic
keyboard:
layout: us
toggle: null
variant: ''
locale: en_US.UTF-8
source:
id: ubuntu-server-minimal
search_drivers: false
ssh:
allow-pw: true
authorized-keys: []
install-server: true
storage:
layout:
name: direct
updates: security
version: 1
Ubuntu 20.04: Packerfile
source "qemu" "iso" {
vm_name = "ubuntu-2004-amd64-iso.qcow2"
iso_url = "https://www.releases.ubuntu.com/20.04/ubuntu-20.04.6-live-server-amd64.iso"
iso_checksum = "sha256:b8f31413336b9393ad5d8ef0282717b2ab19f007df2e9ed5196c13d8f9153c8b"
memory = 1280
disk_image = false
output_directory = "build/iso"
accelerator = "kvm"
disk_size = "12000M"
disk_interface = "virtio"
format = "qcow2"
net_device = "virtio-net"
boot_wait = "3s"
boot_command = [
# Make the language selector appear...
" <up><wait>",
# ...then get rid of it
" <up><wait><esc><wait>",
# Go to the other installation options menu and leave it
"<f6><wait><esc><wait>",
# Remove the kernel command-line that already exists
"<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
"<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
"<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
# Add kernel command-line and start install
"/casper/vmlinuz ",
"initrd=/casper/initrd ",
"autoinstall ",
"ds=nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/ ",
"<enter>"
]
http_directory = "http"
shutdown_command = "echo 'packer' | sudo -S shutdown -P now"
# These are required or Packer will panic, even if no provisioners are not
# configured
ssh_username = "packer"
ssh_password = "packer"
ssh_timeout = "60m"
}
build {
name = "iso"
sources = ["source.qemu.iso"]
}
Ubuntu 20.04: user data
#cloud-config
autoinstall:
version: 1
# Fetch latest version of autoinstall before starting
refresh-installer:
update: true
# Prevent packer from connecting to the _installer_ with SSH and trying to
# start provisioning.
#
# https://tlhakhan.medium.com/ubuntu-server-20-04-autoinstall-2e5f772b655a
#
early-commands:
- "systemctl stop ssh"
locale: en_US.UTF-8
keyboard:
layout: us
apt:
preserve_sources_list: false
primary:
- arches: [i386, amd64]
uri: "http://archive.ubuntu.com/ubuntu"
- arches: [default]
uri: "http://ports.ubuntu.com/ubuntu-ports"
geoip: true
storage:
layout:
name: direct
identity:
hostname: ubuntu
username: packer
password: <password-hash>
ssh:
allow-pw: true
install-server: true
late-commands:
- "echo 'packer ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/packer"
- "chmod 440 /target/etc/sudoers.d/packer"
user-data:
timezone: UTC
disable_root: true
Launching the build
Assuming your system is set up correctly you can build with just:
packer build
If you split your Packer configuration into multiple .pkr.hcl file you need to place all the files into a directory and use
packer build <directory>
Tips
Using custom cloud-init configuration with autoinstall
Autoinstall uses the cloud-init file format. You must place the autoinstall configurations go under the autoinstall top-level key. You can place additional cloud-init modules under the user-data subkey. Here's a simple example that shows how to enable the root user with highly insecure password and to set the timezone:
#cloud-config
autoinstall:
user-data:
timezone: Europe/Helsinki
disable_root: false
chpasswd:
list: |
root:secret
Anything you can do with cloud-init should be possible to do from the user-data section. When you boot the system the first time cloud-init runs and applies these changes.Therefore you won't see any changes from cloud-init on the disk image.
Mounting qcow2 images
You can mount the disk images created by Packer. This technique is very useful whenever you need to debug your autoinstall file. Make sure that you have libguestfs-tools or equivalent package installed, then as root do something like this:
LIBGUESTFS_BACKEND=direct guestmount -a output/ubuntu-2404-amd64-qemu-build -m /dev/sda2 /mnt/guestmount/
If you give guestmount an invalid partition it will helpfullly print out a list of valid partition names:
guestmount: ‘/dev/sda3’ could not be mounted.
guestmount: Did you mean to mount one of these filesystems?
guestmount: /dev/sda1 (unknown)
guestmount: /dev/sda2 (ext4)
guestmount: /dev/ubuntu-vg/ubuntu-lv (ext4)
Guestmount supports logical volumes as well, but debugging is easier with normal partitions. To unmount a guestmounted partition:
umount /mnt/guestmount
Note that making modifications to the mounted disk image will change the image checksum. This may be a problem if subsequent builds assume a certain image checksum.
Using two-stage builds to save time
Provisioning operating systems is a very slow process and autoinstall is no exception. In the Packer context you're doing two things:
- Automating the installation of the operating system
- Running provisioning scripts
You definitely don't want to run the entire operating system installation when making changes to the provisioning scripts. If you did, you'd waste 10 minutes of your time on every try. To alleviate this problem you can split your Packer builds into two phases. For details please have a look at our Two-stage Qemu builds with Packer blog post.
External links
For further information please refer to the official documentation:
- Automated Server Installs
- Automated Server Installs Config File Reference
- Packer QEMU Builder
- Subiquity project
- Curtin project
These external web pages were used as a source: