Installing the KVM Stack on Raspberry Pi OS: What Each Package Does

The previous post confirmed that the kernel’s side is sound. This post installs the programs that run on top of it.

Most guides give you an apt install command with nine package names and no explanation. That works, but it leaves you unable to work out problems later, because you do not know which package provides which program.

Part 1 of this series explained what each layer of the stack does, so the packages can be placed on that structure instead of memorised. Then this post runs libvirt’s own readiness check, which prints a wall of warnings and failures on a perfectly healthy Raspberry Pi. Reading that output correctly is a useful skill in itself.

As in the rest of Part 2, the output was captured on home1, an SD card for my Raspberry Pi 5.

The install command

sudo apt update
sudo apt install qemu-system-arm qemu-utils qemu-efi-aarch64 \
    libvirt-daemon-system libvirt-clients virtinst virt-manager \
    bridge-utils dnsmasq

Read the rest of this post before running it: one of these packages causes a problem later, which is much easier to understand if you know it is coming.

Which package provides what

The VM itself

qemu-system-arm provides /usr/bin/qemu-system-aarch64, the program that is each VM. Everything from post 1.4 and post 1.5 happens inside it: the device models, the virtual processor threads, and the loop that runs the guest on KVM. One running copy is one virtual machine.

The name is a lasting nuisance. It says -arm, but it contains the 64-bit qemu-system-aarch64 program. There is no separate qemu-system-aarch64 package to look for.

qemu-utils provides qemu-img, the tool for disk images, which Part 4 covers in detail. It creates, converts, resizes and snapshots image files on the host, with no VM involved. When virt-install needs a new disk, it asks libvirt, and libvirt uses qemu-img.

qemu-efi-aarch64 is the package most specific to ARM64. ARM64 VMs boot with UEFI firmware, rather than the old BIOS, and this package installs that firmware, called AAVMF, the ARM equivalent of OVMF on x86.

It matters more than its size suggests. It installs more than one version of the firmware, and libvirt’s automatic choice between them causes the most confusing failure in this part of the series, in post 2.3. Post 3.3 explains the firmware files themselves.

Management

libvirt-daemon-system installs libvirtd and sets it up as a system service. This is the manager from post 1.3: it keeps each VM’s XML definition, turns it into a QEMU command, starts the process, and keeps track of it.

Installing it also sets up the libvirt group, defines the default NAT network, and creates the default storage pool, a folder for VM disks, at /var/lib/libvirt/images.

Newer versions of libvirt can split libvirtd into separate services, such as virtqemud and virtnetworkd. They do the same jobs, packaged differently. Debian 12, which Raspberry Pi OS Bookworm is based on, uses the single libvirtd by default, which is what the commands below assume.

bridge-utils provides brctl, a tool for Linux bridges, the software network switches covered in Part 5. Today ip link and bridge do the same jobs, but brctl is still widely referred to.

dnsmasq is a small DHCP and DNS server. libvirt runs its own copy of it for the default NAT network, to give guests their addresses.

This is the package that causes a problem. Installing dnsmasq on its own also installs it as a system service, and starts it, listening on the DNS port, 53, on every interface. libvirt’s own copy then cannot get the port it needs, and the default network fails to start. Post 2.3 shows the failure and the fix. To avoid it altogether, install dnsmasq-base instead of dnsmasq: it provides the program libvirt needs, without the system service.

What you type

libvirt-clients provides the tools you use directly:

Tool	What it does
virsh	The main command-line tool, used for most of this series
virt-host-validate	Checks whether the host is ready, below
virt-xml-validate	Checks that an XML definition is correct

Everything virsh does goes to libvirtd, through libvirt’s interface. It never talks to QEMU directly.

virtinst provides virt-install, which creates a new VM from a set of options, writing its XML and starting the installer in one command. It also provides virt-clone and virt-xml, both used later in the series. virt-install builds the first VM in post 2.4.

virt-manager is a desktop program for managing VMs in a window. It is useful for seeing a VM’s hardware laid out, but everything it does can also be done with virsh. On a Pi without a screen it is optional.

To see what any package installed, or which package a file came from:

dpkg -L qemu-system-arm      # every file the package installed
dpkg -S /usr/bin/qemu-img    # which package a file belongs to

virt-host-validate: is the host ready?

With the stack installed, libvirt can check the host itself:

home1@home1:~ $ virt-host-validate
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : WARN (Enable 'memory' in kernel Kconfig file or mount/enable cgroup controller in your system)
  QEMU: Checking for cgroup 'devices' controller support                     : WARN (Enable 'devices' in kernel Kconfig file or mount/enable cgroup controller in your system)
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for device assignment IOMMU support                         : WARN (Unknown if this platform has IOMMU support)
  QEMU: Checking for secure guest support                                    : WARN (Unknown if this platform has Secure Guest support)
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : PASS
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : PASS
   LXC: Checking for cgroup 'memory' controller support                      : FAIL (Enable 'memory' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'devices' controller support                     : FAIL (Enable 'devices' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'freezer' controller support                     : FAIL (Enable 'freezer' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'blkio' controller support                       : PASS
   LXC: Checking if device /sys/fs/fuse/connections exists                   : PASS

Four warnings and three failures. This host is fine, and here is how to tell.

The four checks that decide everything

Check	Result	Meaning
/dev/kvm exists	PASS	KVM is present
/dev/kvm is accessible	PASS	Your user can use it
/dev/vhost-net exists	PASS	The fast path for networking is available
/dev/net/tun exists	PASS	TAP devices, the host’s end of a VM’s network cable, can be created

These four decide whether accelerated VMs are possible, and all four pass.

Two are worth a moment. /dev/kvm is accessible confirms that the group and ACL permissions from the previous post work for your user, so VMs will not need sudo. And /dev/vhost-net exists is the kernel’s fast path for networking, from post 1.6, already in place before a single VM exists. Post 5.11 later confirms a running VM using it.

Reading the warnings

A WARN means “be aware”, not “broken”.

The memory cgroup warning. cgroups are the kernel’s way of limiting and accounting for the resources a group of processes uses, and libvirt uses them to enforce limits for each VM. Raspberry Pi OS switches the memory controller off by default. VMs still run normally, but libvirt cannot enforce a hard memory limit through cgroups: a VM gets the memory its XML gives it, and QEMU allocates that much. For learning and lab use, this changes nothing. To turn it on, add cgroup_enable=memory cgroup_memory=1 to /boot/firmware/cmdline.txt and restart.

The devices cgroup warning is not specific to the Pi. Raspberry Pi OS Bookworm uses version 2 of cgroups, which has no separate devices controller: access to devices is controlled in a different way. So this check warns on many modern Linux systems, and there is nothing to fix.

The IOMMU warning is a real limit, not a fault. An IOMMU is the hardware that makes it safe to hand a real device to a guest, by limiting which memory the device can reach. Without one, a guest given a real device could make that device write anywhere in the host’s memory. The Pi cannot confirm it has a usable IOMMU, which is why passing real devices to guests is only discussed in theory in this series.

The secure guest warning does not apply here. “Secure guests” means hardware that encrypts a VM’s memory so that even the host cannot read it, such as AMD SEV or Intel TDX. These are server features, not present on the Pi, and not needed for anything in this series.

The LXC failures are not your concern

This is the part that looks alarming: three lines near the end of the output report FAIL, for the memory, devices and freezer cgroup controllers.

virt-host-validate checks two separate technologies. libvirt can manage QEMU and KVM virtual machines, and also LXC containers, so the tool checks the host for both. Every line starting with LXC: is about containers.

This series does not use LXC, so that whole section can be ignored.

The difference in severity is itself informative. The same missing cgroup controllers are a WARN for QEMU and a FAIL for LXC, because containers rely on cgroups to keep themselves apart. Without them, containers cannot do their job. Full VMs are kept apart by the hypervisor instead, so for them the same gap is only a limitation.

The verdict

Part	Result	What to do
The four checks for acceleration	PASS	Nothing: ready
cgroup memory	WARN	Nothing, unless you need hard memory limits
cgroup devices	WARN	Nothing: expected with cgroups version 2
IOMMU	WARN	Nothing: a real hardware limit
Secure guest	WARN	Nothing: not relevant
Every LXC line	FAIL	Nothing: this series does not use containers

Nothing here needs fixing.

Two housekeeping steps

Check the service is running:

systemctl status libvirtd

Look for Active: active (running), and enabled on the Loaded: line, which means it starts with the Pi. If it is not running: sudo systemctl enable --now libvirtd.

Check your groups:

groups

You need libvirt and kvm. If either is missing:

sudo usermod -aG libvirt,kvm $USER

The -a matters. It means “add to”. Without it, usermod -G replaces your whole list of groups with the ones given, which can remove you from groups you need, such as sudo. It is a classic, and damaging, mistake.

Group changes take effect when you next log in, so log out and back in, or restart the Pi, which is simpler.

The final check

virsh list --all

 Id   Name   State
--------------------

An empty table, which is the result you want. It shows three things at once: libvirtd is running and answering, your user can reach it without sudo, and virsh is connected to a working libvirt.

An error here, particularly one saying it failed to connect to the socket, usually means your new group membership has not taken effect yet, and you need to log out and back in.

One caution, which matters straight away. virsh connects to libvirt through an address called a URI, and the one it chooses by default is not always the one you want. An empty table does not tell you which libvirt answered. That is the first of the problems in the next post, and the one that wastes the most time.

Summary

• Nine packages, in three layers: the VM itself (QEMU and its firmware), management (libvirt and networking), and the tools you type.
• qemu-system-arm contains the 64-bit qemu-system-aarch64 program, despite its name.
• qemu-efi-aarch64 provides the AAVMF UEFI firmware that ARM64 VMs boot with.
• Installing dnsmasq rather than dnsmasq-base starts a system service that takes port 53 from libvirt’s own copy.
• virt-host-validate checks both QEMU and KVM, and LXC. Only the QEMU lines matter unless you use containers.
• The four checks for devices decide whether acceleration works. Everything else is advice.
• The memory warning is a Raspberry Pi OS default, the devices warning comes from cgroups version 2, and the IOMMU warning is a real limit for passing devices to guests.
• Use usermod -aG. Without -a, your groups are replaced rather than added to.
• An empty virsh list --all shows that the service, your permissions and the connection all work.