CoreOS Image fails to load ignition file via libvirt (permission denied)

Question

I'm testing out a new network config using VM's and got libvirt and qemu setup on my system. Following the documentation for Fedora CoreOS to help get started.

Steps Taken

• I've updated my user to be a member of kvm and libvirt groups

  > groups
  [ .. ] video kvm fuse libvirt
  

• Created Ignition YAML and exported to the JSON .ign file

  {"ignition":{"version":"3.1.0"},"passwd":{"users":[{ ...
  

• Running the kvm command directly using the documentation works as expected. I am able to ssh into the system
• libvirt is able to generate an XML using the steps in the original CoreOS Documentation (https://coreos.com/os/docs/latest/booting-with-libvirt.html#virtual-machine-startup) with the qemu commands attached

  ...
  <qemu:commandline xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0">
    <qemu:arg value="-fw_cfg"/>
    <qemu:arg value="name=opt/com.coreos/config,file=/var/lib/libvirt/machine/main.ign"/>
  </qemu:commandline>
  

• Tried setting ignition file's permissions to "kvm" and "libvirt" with no change. Currently all files using libvirt:

  $ ls -l /var/lib/libvirt/machine/
  total 8
  -rw-r--r-- 1 root libvirt 2242 Sep 20 17:16 domain.xml
  -rwxr-xr-x 1 root libvirt  658 Sep 20 17:15 main.ign
  

• Machine is loaded and displayed in virt-manager

  $ virsh list --all
  Id   Name      State
  -----------------------------
  -    machine   shut off
  

Only issue is the machine will not work from virt-manager or virsh:

$ virsh start machine
error: Failed to start domain machine
error: internal error: qemu unexpectedly closed the monitor: 2020-09-21T00:34:59.972390Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-09-21T00:34:59.972427Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-09-21T00:34:59.973278Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-09-21T00:34:59.973287Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-09-21T00:34:59.980931Z qemu-system-x86_64: -fw_cfg name=opt/com.coreos/config,file=/var/lib/libvirt/machine/main.ign: can't load /var/lib/libvirt/machine/main.ign: Failed to open file “/var/lib/libvirt/machine/main.ign”: Permission denied

Now out of ideas on what to do. Someone have an idea?

Answer 1

Setting

security_driver = "none"

in /etc/libvirt/qemu.conf and restart libvirtd worked for me to solve a similar problem. (Debian 11, neither AppArmor or SELinux activated / used at all). Error here looked like:

Error: error creating libvirt domain: internal error: process exited while connecting to monitor: ...

and

Could not open '/var/lib/libvirt/images/d1test4-centos7.qcow2': Permission denied

Answer 2

Modify the following line in /etc/libvirt/qemu.conf

security_driver = "none"

Answer 3

Try this instead

# mv main.ign /var/lib/libvirt/images/
# restorecon /var/lib/libvirt/images/main.ign

/var/lib/libvirt/images is properly permissioned and selinux labelled for VMs to access. /var/lib/libvirt/machines is not a standard libvirt directory so depending on how you created it the qemu process may not have access. Also on Fedora the qemu process is launched as 'qemu' user, not 'kvm' or 'libvirt', so it could be due to that too, but if the file is in /var/lib/libvirt/images and global read permissions then I think that should be enough