3.1 · Inside the VM definition
What a libvirt VM Really Is: The XML File Behind Every Tool
The previous post
ended with a guest running, created by one long virt-install command. It also
mentioned, almost in passing, that virt-install never talks to QEMU or KVM
directly. It generates a document and hands it to libvirt.
Part 3 is about that document. This first post explains what it is, how every libvirt tool relates to it, and a few terms that cause confusion later. The rest of Part 3 reads the document block by block, follows the firmware it boots, and looks at the commands that move a VM between states.
Most people who run libvirt VMs only ever use tools: virt-install to create
one, virsh to start and stop it, and perhaps virt-manager to change its
settings in a window. The tools work, so the question of what a VM actually is
rarely comes up. It is worth answering, because the answer is simple, and it
makes the rest of the stack much easier to understand.
A VM is a document
libvirt calls a virtual machine a domain. The word comes from Xen, an older
hypervisor, where it meant a running guest. In libvirt, “domain” and “VM” mean
the same thing. That is why commands have names like dumpxml, dominfo and
domifaddr.
Each domain is described by one XML document. XML is a text format that stores
information as nested, labelled elements, such as <memory> or <disk>.
libvirt stores the document for a VM called lab1 at
/etc/libvirt/qemu/lab1.xml.
To see it, ask libvirt rather than opening the file:
virsh dumpxml lab1
Here is the start of the output for lab1, the Debian guest on my Raspberry
Pi 5. The full document is 180 lines, so this is shortened to the lines that
matter here:
<domain type='kvm' id='1'>
<name>lab1</name>
<uuid>5648e099-729a-4036-b292-bff7a28298c0</uuid>
...
<memory unit='KiB'>2097152</memory>
<currentMemory unit='KiB'>2097152</currentMemory>
<vcpu placement='static'>2</vcpu>
...
<os>
<type arch='aarch64' machine='virt-7.2'>hvm</type>
<loader readonly='yes' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader>
<nvram template='/usr/share/AAVMF/AAVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/lab1_VARS.fd</nvram>
<boot dev='hd'/>
</os>
...
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/lab1.qcow2' index='1'/>
...
</domain>
Even this short extract describes a complete machine. It has 2,097,152 KiB of
memory, which is 2 GiB, and two virtual CPUs. It is an ARM64 machine
(arch='aarch64'). It boots from UEFI firmware, and its hard disk is a file
called lab1.qcow2.
Notice that this is a description, not a set of instructions. The document does not say “allocate 2 GiB of memory”. It says “this machine has 2 GiB of memory”. Turning that description into a running machine is libvirt’s job.
Why virsh dumpxml and not the file itself
The file in /etc/libvirt/qemu/ belongs to libvirt. libvirt does not watch it
for changes, so if you edit it by hand while libvirt is running, nothing
happens, and libvirt may later overwrite your edit. The safe way to read the
document is virsh dumpxml, and the safe way to change it is virsh edit,
which opens it in your text editor and checks your changes before saving them.
What the document points to
The document is the definition of the VM, but it does not contain the VM’s
data. Instead, it points to files that do. For lab1, there are two:
| File | What it holds | Where the XML points to it |
|---|---|---|
/var/lib/libvirt/images/lab1.qcow2 | The virtual hard disk: the operating system and everything installed on it | <source file=...> inside <disk> |
/var/lib/libvirt/qemu/nvram/lab1_VARS.fd | The firmware’s settings, including its list of boot entries | <nvram> inside <os> |
The second file is easy to overlook. Every VM that boots with UEFI firmware has one, and on ARM64 that means every VM. Post 3.3 explains why the firmware comes as two files, and why that matters.
So a more precise answer to “what is a VM?” is: one XML document, plus the files it points to. There is no hidden database behind it. If you want to know what a VM is, read its XML. If you want to change it, change its XML. If you want to move it to another machine, copy the XML and the files it points to. (Snapshots add records of their own, which Part 4 covers.)
Where the tools fit
Once you know the document exists, every libvirt tool can be described by what it does to it:
virt-install ─────────► writes a new document from command-line options
virt-manager ─────────► a graphical program that reads and writes the document
virsh edit ───────────► opens the document in your text editor
virsh dumpxml ────────► prints the document
virsh define ─────────► registers a document as a domain
virsh start ──────────► reads the document and builds a running machine from it
virt-install, which most people meet first, is a generator. It turns its
options into an XML document, hands the document to libvirt, and starts the
result. Everything it does could also be done by writing the XML by hand and
running virsh define. It exists because writing that document from scratch
is tedious, not because it can do something the document cannot express.
This reverses the usual way of thinking about it. The tools are not the real interface, with the XML as a hidden detail. The XML is the real interface, and the tools are conveniences built on top of it.
Persistent and transient domains
libvirt has two kinds of domain, and the difference is simply whether the document is saved:
- A persistent domain has its XML stored on disk. It continues to exist as a
definition when it is not running, and after the host restarts.
lab1is persistent, which is why it still appears invirsh list --allafter it has been shut down. - A transient domain exists only while it is running. Its XML is never stored, so when it stops, it disappears completely.
virt-install creates persistent domains. Transient domains are less common
when you work by hand, but automated systems use them to start machines on
demand without leaving definitions behind.
define, create and start
These three commands are often confused, and the difference follows directly from persistent and transient domains:
| Command | What it does |
|---|---|
virsh define lab1.xml | Registers a persistent domain from an XML file. It saves the definition, but does not start anything. |
virsh create lab1.xml | Starts a transient domain directly from an XML file. It runs the machine now, and saves nothing. |
virsh start lab1 | Starts a domain that libvirt already knows about, by its name. |
In short, define means “remember this machine”, create means “run this
machine now, without remembering it”, and start means “run the machine you
already remember”.
The confusing one is create. It sounds like the command for making a new VM,
but it actually runs a VM that libvirt has not saved. The command that makes a
VM, in the sense most people mean, is define.
The opposites follow the same pattern, and one of them has an unfortunate name.
The opposite of define is undefine, which deletes the saved definition. The
opposite of create is destroy, which stops a running machine immediately,
like pulling its power cable. Despite its name, virsh destroy deletes nothing:
the XML, the disk and the firmware settings all remain. Post 3.7 looks at this
in more detail.
So there are two separate questions, each with its own pair of commands:
| Question | Commands |
|---|---|
| Does a saved definition exist? | define and undefine |
| Is the machine running right now? | create and destroy (and start for a defined machine) |
What libvirt turns the document into
There is one more fact that makes the whole document make sense: libvirt does not run virtual machines itself.
When you start a domain, libvirt reads its XML, translates it into a very long
qemu-system-aarch64 command, and runs that command. The QEMU process that
results is the virtual machine. libvirt keeps a connection to that process so
that it can control it and report its state.
domain XML ──libvirt translates──► qemu-system-aarch64 -machine virt ...
│
▼
the QEMU process is the VM
This means every element in the document corresponds to something QEMU can be
told on its command line. <memory> becomes the -m option, and <vcpu>
becomes -smp. A <disk> becomes a set of -blockdev options, which describe
the file and its format, and a -device option, which describes the virtual
disk controller the guest sees. The XML is an easier, descriptive way of writing
that command, plus a long list of sensible defaults you would otherwise have to
write yourself.
libvirt can show you the translation:
sudo virsh domxml-to-native qemu-argv /etc/libvirt/qemu/lab1.xml
For lab1, the output is a single line of several thousand characters, and
every option in it can be traced back to a block in the document. Post 3.9
works through that translation, and then starts the VM from a QEMU command
written by hand, with libvirt out of the loop.
Why it pays to read the XML
The practical reason to learn this document, rather than staying with the tools, is that every new capability appears as something in the document.
Adding a second disk is a new <disk> block. Moving a VM from the default NAT
network to your real network is a change to its <interface> block. Pinning
virtual CPUs to particular physical cores is a <cputune> block. Giving a guest
a real PCI device from the host is a <hostdev> block.
So the question “how do I do this with a libvirt VM?” usually becomes “which
element does this, and where does it go?”. That is easier to answer than
searching for the right tool option, and the answer is the same whether you use
virsh, virt-manager, Terraform, or a Python script that talks to libvirt
directly. They all end up writing the same document.
Summary
- libvirt calls a VM a domain. Each domain is defined by one XML document,
which libvirt stores in
/etc/libvirt/qemu/. - The document points to the files that hold the VM’s data: its disk image and, for UEFI VMs (which on ARM64 means every VM), its firmware settings file.
- Read the document with
virsh dumpxml, and change it withvirsh edit. libvirt does not notice edits made directly to the file. - Every tool, including
virt-install,virt-managerandvirsh, works by reading or writing this document. - A persistent domain has a saved definition. A transient domain exists only while it runs.
definesaves a definition without starting it,createruns a VM without saving it, andstartruns a VM libvirt already knows.destroystops a running VM and deletes nothing.- libvirt translates the document into a
qemu-system-aarch64command, and the resulting QEMU process is the VM.
The next post steps back from libvirt to something more basic: what happens when any computer, real or virtual, is switched on, before an operating system exists.
Comments