1.6 · How the stack works
Why VirtIO Is Faster Than Emulated Devices: Virtqueues and vhost-net Explained
The previous article established what a VM exit costs, and that the expensive ones are full round trips to QEMU userspace.
With that cost understood, the obvious question is what to do about it. VirtIO is the answer, and it’s a genuinely elegant one — not a micro-optimization but a rethinking of what a virtual device should be.
To see why it works, you first have to feel the problem it solves.
The problem: emulating real hardware means exit storms
Take the serial console from the previous article. The guest writes one character to a device address, that traps, QEMU’s UART model prints it, back into the guest. One character, one round trip.
For a serial console that’s completely fine. Humans type slowly, and terminal output is measured in kilobytes.
Now apply the same approach to a network card.
A real NIC — say an Intel e1000, a common emulated model — is controlled through dozens of hardware registers. Its driver was written for physical silicon, and to send a single packet it performs a sequence something like this:
guest driver sends ONE packet through an emulated e1000:
write register: descriptor ring base address → VM EXIT
write register: descriptor length → VM EXIT
write register: head pointer → VM EXIT
write register: tail pointer (the "kick") → VM EXIT
read register: status → VM EXIT
read register: interrupt cause → VM EXIT
... and moreEvery one of those register accesses is MMIO. Every one traps. Every one is an expensive round trip out to QEMU’s C code and back.
Several exits to send one packet. At a hundred thousand packets per second, the guest spends more time exiting than computing. The VM doesn’t just underperform — it collapses.
And here’s the part worth sitting with: none of that cost buys anything. QEMU is faithfully reproducing the behaviour of a chip designed for electrical signals on a circuit board. The register-at-a-time protocol exists because that’s how a CPU talks to physical hardware across a bus. In a VM there is no bus, no chip, and no electrical anything — both sides are software running on the same machine. You are emulating the inefficiency of real hardware, perfectly, for no benefit whatsoever.
That observation is what produces VirtIO.
The flip: a device the guest knows is virtual
Every device so far works by deception. The guest believes it’s talking to real hardware; QEMU intercepts the accesses and pretends convincingly. The guest’s driver is the standard, unmodified driver for real silicon, and it must never learn the truth.
VirtIO works by cooperation. The guest is told outright that the device is virtual, and both sides use a protocol designed for the situation they’re actually in — software talking to software.
EMULATED DEVICE (e1000) VIRTIO DEVICE (virtio-net)
─────────────────────── ──────────────────────────
guest: "I'll talk to this guest: "I KNOW this is a
real Intel chip" virtual device. Let's use
a protocol built for speed."
QEMU: secretly intercepts QEMU: openly agrees on a
every register poke shared structure with the guest
│ │
many exits per operation ~one exit per BATCH of operationsThe trade-off is that the guest needs a driver that speaks VirtIO. It isn’t fooled — it’s a willing participant, and it has to know the protocol.
In practice this is not a burden. VirtIO drivers have been in the mainline Linux kernel for many years, so any modern Linux guest has them built in. This is precisely why Debian and Alpine aarch64 guests “just work” with bus=virtio in this series. Windows needs drivers installed separately, which is the one common case where it matters.
The virtqueue
Here’s the mechanism, and its simplicity is the point.
Instead of poking registers one at a time, the guest and host share a region of memory containing a ring of descriptors. The guest writes requests into it; the host reads them out. Both sides are just reading and writing ordinary memory — and memory access doesn’t trap.
A virtqueue has three parts, all living in memory the guest allocated and the host can see:
THE VIRTQUEUE (shared memory, guest writes / host reads)
┌─────────────────────────────────────────────────────┐
│ 1. DESCRIPTOR TABLE │
│ the entries themselves: "buffer at address X, │
│ length Y, readable/writable, next in chain" │
│ ┌────┬────┬────┬────┬────┬────┐ │
│ │ d0 │ d1 │ d2 │ d3 │ .. │ dN │ │
│ └────┴────┴────┴────┴────┴────┘ │
├─────────────────────────────────────────────────────┤
│ 2. AVAILABLE RING (guest → host) │
│ "guest has placed these descriptors for you; │
│ please process them" │
├─────────────────────────────────────────────────────┤
│ 3. USED RING (host → guest) │
│ "host has finished these; here are the results" │
└─────────────────────────────────────────────────────┘A restaurant makes the roles concrete. The descriptor table is the pool of order slips, each describing one buffer of data. The available ring is the rail where the waiter clips orders for the kitchen — the guest saying “these are ready for you.” The used ring is the counter where the kitchen puts finished plates — the host saying “these are done, come collect.”
Both the waiter and the kitchen read the same rails. Nobody pokes a register. The rings are circular buffers with index counters, which is what makes them lock-free: the guest only advances the available index, the host only advances the used index, and neither needs to block the other.
What sending packets costs now
Watch the exit count.
1. guest driver writes packet buffers into memory, and adds
descriptors pointing at them into the AVAILABLE ring
│ (ordinary memory writes — NO exits)
│
...it can queue MANY packets this way, all exit-free...
│
▼
2. guest does ONE "kick" — a single notification meaning
"there's work in the queue"
│ ◄──── THIS is the one VM exit
▼
3. VM EXIT → the host side wakes, reads the available ring,
and processes the WHOLE BATCH at once
│
▼
4. host places results in the USED ring and signals the guest
once — one interrupt for the entire batch
│
▼
5. guest reads the used ring and sees everything completedDozens of packets: one exit and one interrupt. Against the e1000’s several exits per packet.
That is the whole performance story. VirtIO doesn’t make an exit cheaper — an exit costs exactly what article 5 said it costs. It amortizes that cost across a batch of work.
The general principle is worth extracting, because it recurs throughout systems work: when an operation has a high fixed cost and a low marginal cost, the win comes from doing more per operation, not from doing the operation faster.
There’s a further refinement in the same spirit. Under load, the host can suppress notifications entirely — if it’s already processing the queue, the guest can skip the kick, because the host will pick up new entries on its next pass. Sustained high throughput can approach zero exits for stretches. The available ring carries flags for exactly this negotiation.
vhost-net: shortening the path further
One more optimization, and it explains a device file you’ll see on the host.
In the flow above, the kick exits out to QEMU in userspace. For networking, even that detour can be removed. vhost-net moves the host end of the virtqueue into the kernel:
WITHOUT vhost-net WITH vhost-net
──────────────── ──────────────
guest kicks → VM exit guest kicks → handled by a
│ KERNEL thread reading the
▼ virtqueue directly
QEMU (userspace) reads │
the queue, then calls packets go guest → kernel →
into the kernel to send NIC without the userspace
│ detour through QEMU
kernel sends packetThe virtqueue is the same structure. vhost-net simply lets a kernel thread be the one reading the available ring and writing the used ring, so packets never travel up to QEMU userspace at all. QEMU sets the arrangement up once, then steps out of the data path entirely.
The host exposes this as /dev/vhost-net, and its presence is one of the things virt-host-validate checks — which appears in article 8 of this series. It’s a good example of a concept you can confirm exists on your host before ever building a VM.
The same idea extends elsewhere: vhost-user moves the backend to another userspace process, which is how high-performance software switches integrate with VMs.
Self-description: the pattern underneath
One last piece, because it’s a pattern that recurs across the whole field.
How does a guest know which VirtIO devices exist and what they support? The device describes itself, and the two sides negotiate:
1. guest finds a virtio device (on a virtual PCI or MMIO bus)
2. guest reads its DEVICE TYPE → "I'm a network card" (or block, etc.)
3. host advertises FEATURE BITS → "I support checksum offload,
multiqueue, indirect descriptors…"
4. guest acknowledges the subset it also supports
5. both sides set up virtqueues and beginFeature negotiation is what lets the standard evolve without breaking anything: a new host feature is simply not acknowledged by an old guest, and both proceed with the common subset.
That sequence — a device describes itself, the other side trusts the description and loads a matching driver, then both exchange data through a shared queue — is not unique to VirtIO. USB enumeration works this way. PCI configuration space works this way. Device passthrough works this way. VirtIO is the cleanest example of a pattern that runs through the entire field, which is why recognizing it here pays off repeatedly later.
Where you’ll see this
VirtIO isn’t abstract — it’s a choice you make explicitly every time you define a VM.
In virt-install, bus=virtio on a disk and model=virtio on a network interface select it. In domain XML, the same choices appear as <target dev='vda' bus='virtio'/> and <model type='virtio'/>.
There’s a visible consequence inside the guest. A VirtIO disk appears as /dev/vda — the v is for virtio — while an emulated SATA or SCSI controller yields /dev/sda. Seeing vda in a guest is immediate confirmation that the fast path is in use.
The common VirtIO device types, all following the same virtqueue model:
virtio-blkandvirtio-scsi— storagevirtio-net— networkingvirtio-balloon— reclaiming guest memory back to the host without a rebootvirtio-gpu— graphicsvirtio-rng— feeding host entropy to the guest, which matters because fresh VMs otherwise start entropy-starved
Summary
- Emulated devices are slow because they faithfully reproduce hardware protocols designed for physical buses. An emulated NIC costs several VM exits per packet, and that cost buys nothing.
- VirtIO replaces deception with cooperation: the guest knows the device is virtual and uses a protocol designed for software-to-software communication.
- The virtqueue is shared memory containing a descriptor table, an available ring (guest → host), and a used ring (host → guest). Reading and writing it costs no exits.
- Work is batched and released with a single kick, so one exit and one interrupt can cover dozens of operations. VirtIO amortizes exit cost rather than reducing it.
- vhost-net moves the host end of the virtqueue into the kernel, removing the QEMU userspace detour from the network data path.
- VirtIO devices describe themselves and negotiate feature bits — a pattern shared with USB, PCI, and passthrough.
- A VirtIO disk shows up in the guest as
/dev/vda, which is a quick visual confirmation the fast path is active.
Comments