ArgoBox
ArgoBox Infrastructure & Automation
user@argobox:~/journal/2025-11-04-the-vm-that-couldnt-find-its-disk
$ cat entry.md

The VM That Couldn't Find Its Disk

[Tuesday, November 4, 2025] MOOD: DEBUGGING
○ NOT REVIEWED

18:15 - Gentoo VM won’t boot. Dracut emergency shell:

dracut Warning: Could not boot.
dracut Warning: /dev/nbd0p3 does not exist

The kernel loads. Hardware initializes. Then dracut can’t find the root filesystem.

18:16 - Tried to modify the VM in virt-manager. Different error:

XML error: Invalid PCI address 0000:04:00.0. slot must be >= 1

Now we have two problems.

18:17 - Checked the VM XML. Found three PCI devices using slot 0x00:

<address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>

Slot 0 is reserved for the PCI bridge. Devices need slots 1-31.

18:20 - Fixed PCI slots, but then the opposite error:

XML error: Invalid PCI address 0000:04:01.0. slot must be <= 0

Turns out PCIe root ports can only have slot 0. The solution was moving devices to bus 0x10 (the pcie-to-pci-bridge), which allows multiple slots.

18:33 - Back to the original problem. The boot disk is /dev/vda, not /dev/nbd0p3. Someone configured this VM for NBD (Network Block Device) boot, but the actual disk is a local qcow2 file.

Checked GRUB:

linux /vmlinuz-6.12.47-gentoo-dist root=UUID=dbe636a9-... rootflags=subvol=@ root=/dev/nbd0p3 rootflags=subvol=@ ro

Two root= parameters. Two rootflags= parameters. GRUB was confused.

18:36 - The fix. Single root parameter pointing to the actual disk:

linux /vmlinuz-6.12.47-gentoo-dist root=/dev/disk/by-path/... ro rootflags=subvol=@

Regenerated grub.cfg. VM boots.

The Problem: This VM was set up for NBD boot testing, then switched back to local storage. The GRUB config kept remnants of both configurations. And the PCI addresses had been manually tweaked, leaving some devices in invalid slots.

The Lesson: When you repurpose a VM, check the bootloader config. Legacy parameters from previous configurations don’t just disappear - they accumulate and conflict.

Also: PCIe topology is confusing. Root ports want slot 0. Behind a pci-to-pci bridge, you can use slots 1-31. Virt-manager error messages don’t explain which.

The VM had been sitting broken for days. I assumed it was something complicated. It was two duplicate parameters in GRUB.