My Experiences with virtio-9p
When I was scaling up my home lab, I started thinking more about data management. I hadn’t (and still haven’t) set up any form of network storage. I have, however, set up a backup mechanism using Borg. Still, I want to operate lots of virtual machines, and backing up each one of them separately seemed excessive. So I started thinking, what if I just let the host machines back up the data? After all, the amount of physical hosts I have in my home lab is unlikely to increase drastically.
The Use Case for Sharing Directories
I started working out this idea further. Without network storage, I needed a way for guest VMs to access the host’s disks. Here there are two possibilities, either expose some block device or a file system. Creating a whole virtual disk for just the data of some VMs seemed wasteful, and from my experiences also increases backup times dramatically. I therefore searched for a way to mount a directory from the host OS on the guest VM. This is when I stumbled upon this blog post talking about sharing directories with virtual machines.
Sharing Directories with virtio-9p
virtio-9p is a way to map a directory on the host OS to a special device on the virtual machine. In virt-manager
, it looks like the following: Under the hood, virtio-9p uses the 9pnet protocol. Originally developed at Bell Labs, support for this is available in all modern Linux kernels. If you share a directory with a VM, you can then mount it. Below is an extract of my /etc/fstab
to automatically mount the directory:
data /mnt/data 9p trans=virtio,rw 0 0
The first argument (data
) refers to the name you gave this share from the host With the trans
option we specify that this is a virtio share.
Problems with virtio-9p
At first I had no problems with my setup, but I am now contemplating just moving to a network storage based setup because of two problems.
The first problem is that some files have suddenly changed ownership from libvirt-qemu
to root
. If the file is owned by root
, the guest OS can still see it, but cannot access it. I am not entirely sure the problem lies with virtio, but I suspect it is. For anyone experiencing this problem, I wrote a small shell script to revert ownership to the libvirt-qemu
user:
find -printf "%h/%f %u\n" | grep root | cut -d ' ' -f1 | xargs chown libvirt-qemu:libvirt-qemu
Another problem that I have experienced, is guests being unable to mount the directory at all. I have only experienced this problem once, but it was highly annoying. To fix it, I had to reboot the whole physical machine.
Alternatives
virtio-9p seemed like a good idea, but as discussed, I had some problems with it. It seems virtioFS might be a an interesting alternative as it is designed specifically for sharing directories with VMs.
As for me, I will probably finally look into deploying network storage either with NFS or SSHFS.