Why VirtIO Is Faster Than Emulated Devices: Virtqueues and vhost-net Explained

The previous post showed what a VM exit costs, and that the expensive ones are full round trips out to QEMU.

With that cost understood, the obvious question is what to do about it. virtio is the answer, and a neat one: not a small optimization, but a rethink of what a virtual device should be. To see why it works, start with the problem it solves.

The problem: imitating real hardware means many exits

Take the serial console from the previous post. The guest writes one character to a device address, that traps, QEMU’s serial port model prints it, and the guest carries on. One character, one round trip.

For a serial console, that is completely fine. People type slowly, and a console’s output is measured in kilobytes.

Now apply the same approach to a network card.

A real network card, such as Intel’s e1000, a common one for QEMU to imitate, is controlled through dozens of hardware registers. Its driver was written for a physical chip, and sending a single packet involves a series of register accesses, along these lines:

Each of those register accesses is memory-mapped I/O, so each one traps, and each is an expensive round trip out to QEMU’s code and back. The exact sequence depends on the driver, but the pattern holds: several exits to send one packet. At a hundred thousand packets a second, the guest spends more time exiting than doing work.

And the important point is that none of that cost buys anything. QEMU is faithfully reproducing how a chip designed for electrical signals on a circuit board behaves. Accessing registers one at a time exists because that is how a processor talks to physical hardware across a bus. In a VM there is no bus and no chip: both sides are software on the same machine. You are imitating the inefficiency of real hardware, perfectly, for no benefit.

That observation is what led to virtio.

The turn: a device the guest knows is virtual

Every device so far works by deception. The guest believes it is talking to real hardware, and QEMU intercepts its accesses and pretends convincingly. The guest uses its ordinary driver for the real chip, and must never find out.

virtio works by cooperation. The guest is told plainly that the device is virtual, and both sides use a method designed for what they actually are: software talking to software.

	An imitated device, such as the e1000	A virtio device, such as virtio-net
What the guest believes	It is using a real Intel chip	It knows the device is virtual
How QEMU works	Secretly intercepts every register access	Openly shares a structure in memory with the guest
Exits	Several for every operation	About one for each batch of operations

The cost is that the guest needs a driver that understands virtio: it is a willing participant, not fooled, so it has to know the method.

In practice, that is no burden. virtio drivers have been in the Linux kernel for many years, so any modern Linux guest has them already. That is why the Debian and Alpine guests in this series just work with virtio. Windows needs drivers installed separately, which is the one common case where it matters.

The virtqueue

Here is the mechanism, and its simplicity is the point.

Instead of accessing registers one at a time, the guest and the host share an area of memory holding a ring of entries. The guest writes requests into it, and the host reads them out. Both sides are only reading and writing ordinary memory, and ordinary memory accesses do not trap.

A virtqueue has three parts, all in memory the guest set aside and the host can see:

Part	Written by	Holds
The descriptor table	The guest	The entries themselves: “a buffer at this address, this long, to be read or written, and the next one in the chain is…”
The available ring	The guest	“These entries are ready for you: please process them.”
The used ring	The host	“These are finished, and here are the results.”

A restaurant makes the roles concrete. The descriptor table is the pile of order slips, each describing one buffer of data. The available ring is the rail where the waiter clips orders for the kitchen: the guest saying “these are ready for you”. The used ring is the counter where the kitchen puts finished plates: the host saying “these are done, come and collect them”.

Both sides read the same rails, and nobody touches a register. The rings are circular, with a counter for each: the guest only moves the available ring’s counter on, and the host only moves the used ring’s, so neither has to wait for the other.

What sending packets costs now

The figure above shows the difference. With virtio:

1. The guest puts the packets’ buffers in memory, and adds entries pointing at them to the available ring. These are ordinary memory writes, with no exits, and it can queue many packets this way.
2. The guest sends one notification, called a kick, meaning “there is work in the queue”. That is the one VM exit.
3. The host side wakes, reads the available ring, and processes the whole batch at once.
4. It puts the results in the used ring, and signals the guest once: one interrupt for the whole batch.
5. The guest reads the used ring, and sees everything completed.

Dozens of packets cost one exit and one interrupt, against several exits for each packet with the imitated e1000.

That is the whole performance story. virtio does not make an exit cheaper: an exit costs exactly what the previous post said. It shares that cost across a batch of work.

The general principle is worth keeping, because it appears throughout computing: when an operation has a high fixed cost and a low cost for each extra item, the gain comes from doing more per operation, not from doing the operation faster.

There is a further refinement in the same spirit. Under heavy load, the host can tell the guest not to send kicks at all: if it is already working through the queue, it will find new entries on its next pass anyway. With a steady, heavy flow of traffic, there can be stretches with almost no exits. The rings carry flags for exactly this.

vhost-net: shortening the path again

One more optimization, which explains a device file you will find on the host.

In the steps above, the kick goes out to QEMU, an ordinary program. For networking, even that can be removed. vhost-net moves the host’s end of the virtqueue into the kernel:

	Path of a packet
Without vhost-net	The guest kicks, and QEMU reads the queue, then asks the kernel to send the packet
With vhost-net	The guest kicks, and a vhost-net worker reads the queue directly, inside the kernel, and sends the packet. QEMU is not involved.

The virtqueue is the same structure. vhost-net only changes who reads the available ring and writes the used ring, so packets never travel up to QEMU at all. QEMU sets the arrangement up once, and then steps out of the path every packet takes.

On older kernels, that worker was a separate kernel thread. On recent ones, including the Raspberry Pi 5’s 6.12 kernel, it runs as a thread belonging to the QEMU process itself, and post 5.11 finds it switched on for a running guest on my Pi.

The host offers vhost-net as /dev/vhost-net, and checking for it is one of the things virt-host-validate does, in post 2.2. It is a good example of something you can confirm on your host before building a VM.

The same idea goes further: vhost-user moves the host’s end of the queue into a separate ordinary program, which is how high-speed networking software works with VMs.

Self-description: the pattern underneath

One last piece, because it is a pattern that appears throughout computing.

How does a guest know which virtio devices exist, and what they can do? The device describes itself, and the two sides agree on what to use:

1. The guest finds a virtio device, on a virtual PCI bus or as a memory-mapped device.
2. The guest reads its device type: “I am a network card”, or a disk, and so on.
3. The host offers a list of features, as bits: “I can calculate checksums for you, use several queues, …”
4. The guest accepts the ones it also supports.
5. Both sides set up their virtqueues, and begin.

Agreeing on features is what lets the standard grow without breaking anything: an older guest simply does not accept a new feature, and both sides carry on with what they have in common.

That sequence, a device describing itself, the other side loading a matching driver, and then both exchanging data through a shared queue, is not unique to virtio. USB devices are discovered this way, and so are PCI devices. virtio is the cleanest example of a pattern found across the whole field.

Where you will see this

virtio is not abstract: it is a choice you make every time you define a VM.

With virt-install, bus=virtio on a disk and model=virtio on a network card choose it, as post 2.4 explains. In a VM’s XML, the same choices appear as <target dev='vda' bus='virtio'/> and <model type='virtio'/>.

It is also visible inside the guest. A virtio-blk disk appears as /dev/vda, with v for virtio, so seeing vda confirms the fast path. The reverse does not hold, though: a virtio-scsi disk appears as /dev/sda, just like an imitated SATA disk, so sda on its own does not mean a slow disk.

The common virtio devices all use the same virtqueues:

Device	What it is for
virtio-blk and virtio-scsi	Disks
virtio-net	Networking
virtio-balloon	Taking memory back from a guest for the host, without restarting it
virtio-gpu	Graphics
virtio-rng	Passing random numbers from the host to the guest, which helps a newly started guest that has little randomness of its own

Summary

• Imitated devices are slow because they faithfully reproduce hardware methods designed for physical buses. An imitated network card costs several VM exits for every packet, and that cost buys nothing.
• virtio replaces deception with cooperation: the guest knows the device is virtual, and uses a method designed for software talking to software.
• A virtqueue is shared memory holding a descriptor table, an available ring from guest to host, and a used ring from host to guest. Reading and writing it causes no exits.
• Work is put in the queue, then released with a single kick, so one exit and one interrupt can cover dozens of operations. virtio shares the cost of an exit rather than reducing it.
• vhost-net moves the host’s end of the virtqueue into the kernel, taking QEMU out of the path packets take. On recent kernels, its worker is a thread of the QEMU process.
• virtio devices describe themselves and agree on features, a pattern shared with USB and PCI.
• A virtio-blk disk appears in the guest as /dev/vda. A virtio-scsi disk appears as /dev/sda.