CODEWITHSUNDEEP
macosqemulibvirtsilicon
fish lein
fish lein

Reputation: 41

libvirt on Apple Silicon with qemu-system-aarch64

Compiled qemu from this manual

Try to run this on Apple Silicon with qemu-system-aarch64, but got an error:

Error: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-aarch64 for probing: qemu-system-aarch64: invalid accelerator kvm Could not allocate dynamic translator buffer

This is the XML:

<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <name>ubuntu</name>
    <uuid>2005CB24-522A-4485-9B9A-E60A61D9F8CF</uuid>
    <memory unit='GB'>2</memory>
    <cpu mode='custom'>
        <model>Westmere</model>
    </cpu>
    <vcpu>2</vcpu>
    <features>
        <acpi/>
        <apic/>
    </features>
    <os>
        <type arch='aarch64' machine='cortex-a57'>hvf</type>
        <bootmenu enable='yes'/>
    </os>
    <clock offset='localtime'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <pm>
        <suspend-to-mem enabled='no'/>
        <suspend-to-disk enabled='no'/>
    </pm>
    <devices>
        <emulator>/usr/local/bin/qemu-system-aarch64</emulator>
        <controller type='usb' model='ehci'/>
        <disk type='file' device='disk'>
            <driver name='qemu' type='qcow2'/>
            <source file='/Users/matthias/VM/Ubuntu_20.04-LTS/disk.qcow2'/>
            <target dev='vda' bus='virtio'/>
        </disk>
        <!--disk type='file' device='cdrom'>
            <source file='/Users/matthias/VM/Ubuntu_20.04-LTS/ubuntu-20.04.2-live-server-arm64.iso'/>
            <target dev='sdb' bus='sata'/>
        </disk-->
        <console type='pty'>
            <target type='serial'/>
        </console>
        <input type='tablet' bus='usb'/>
        <input type='keyboard' bus='usb'/>
        <graphics type='vnc' port='5900' listen='127.0.0.1'/>
        <video>
            <model type='virtio' vram='16384'/>
        </video>
    </devices>
    <seclabel type='none'/>
    <qemu:commandline>
        <!--qemu:arg value='-machine'/>
        <qemu:arg value='type=q35,accel=hvf'/>
        <qemu:arg value='-netdev'/>
        <qemu:arg value='user,id=n1,hostfwd=tcp::2222-:22'/>
        <qemu:arg value='-device'/>
        <qemu:arg value='virtio-net-pci,netdev=n1,bus=pcie.0,addr=0x19'/-->
    <qemu:arg value='-accel hvf -m 2048 -cpu cortex-a57 -M virt,highmem=off'/>
    <qemu:arg value='-drive file=/usr/local/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on'/>
    <qemu:arg value='-drive file=ovmf_vars.fd,if=pflash,format=raw'/>
    <qemu:arg value='-serial telnet::4444,server,nowait'/>
    <qemu:arg value='-device virtio-blk-device,drive=hd0,serial="dummyserial"'/>
    <qemu:arg value='-device virtio-net-device,netdev=net0'/>
    <qemu:arg value='-netdev user,id=net0,hostfwd=tcp:127.0.0.1:2222-0.0.0.0:22'/>
    <qemu:arg value='-vga none -device ramfb'/>
    <qemu:arg value='-device usb-ehci -device usb-kbd -device usb-mouse -usb'/>
    <qemu:arg value='-nographic -serial mon:stdio'/>
    </qemu:commandline>
</domain>

Thx for any feedback!

Upvotes: 4

Views: 7360

Answers (4)

Bill
Bill

Reputation: 73

Instead of using XML and virsh, I found it is much easier to use shell to call QEMU directly. Here is the script I am using:

qemu-system-aarch64 \
         -machine virt,accel=hvf,highmem=off \
         -cpu cortex-a72 -smp 2 -m 4G \
         -device intel-hda -device hda-output \
         -device qemu-xhci \
         -device virtio-gpu-gl-pci \
         -device usb-kbd \
         -device virtio-mouse-pci \
         -display cocoa,gl=es \
             -device e1000,netdev=net0 \
             -netdev user,id=net0 \
         -drive "if=pflash,format=raw,file=#{firmware_path}/edk2-aarch64-code.fd,readonly=on" \
         -drive "if=pflash,format=raw,file=#{firmware_path}/edk2-arm-vars.fd,discard=on" \
         -drive "if=virtio,format=qcow2,file=#{disk_file_location},discard=on" \
             -chardev qemu-vdagent,id=spice,name=vdagent,clipboard=on \
             -device virtio-serial-pci \
             -device virtserialport,chardev=spice,name=com.redhat.spice.0

If you happen to use vagrant, you may find my plugin useful: https://github.com/billyan2018/vagrant-qemu

Upvotes: 3

ihsakashi
ihsakashi

Reputation: 31

You need to pass -machine accel=hvf,highmem=off as a QEMU argument. I've got a fully working Libvirt + QEMU solution in https://github.com/ihsakashi/VM . Libvirt couldn't identify the aarch64 architecture using uname since macOS gave something different. That's been patched.

Upvotes: 3

Darren Smith
Darren Smith

Reputation: 23

You've got to apply this patch: https://bugs.launchpad.net/qemu/+bug/1914849/comments/3

And re-compile. This seems to stop this error from happening. However, I'm working off of the same tutorial that you are, and there are some other changes that need to be made too.

So far, all I've figured out is that I need to reference this EFI firmware: https://www.sevarg.net/images/2021-qemu-m1/QEMU_EFI.fd (referenced from this tutorial - https://www.sevarg.net/2021/01/09/arm-mac-mini-and-boinc/)

I've tried it in the OS XML Block, but it cannot seem to find it:

    <os firmware='efi'>
        <type>hvm</type>
        <loader readonly='yes' secure='no' type='pflash'>/Users/darren/Documents/VMs/QEMU_EFI.fd</loader>
        <boot dev='hd'/>
        <bootmenu enable='yes'/>
        <smbios mode='sysinfo'/>
        <bios useserial='yes' rebootTimeout='0'/>
    </os>

So instead I referenced the UEFI BIOS as a command line parameter.

Anyway, here is my full XML:

<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <name>ubuntu1</name>
    <uuid>2005CB24-522A-4485-9B9A-E60A61D9F8CF</uuid>
    <memory unit='GB'>2</memory>
    <cpu mode='custom'>
        <model>cortex-a72</model>
    </cpu>
    <vcpu>1</vcpu>
    <features>
        <apic/>
    </features>
    <os>
        <type arch='aarch64' machine='virt'>hvm</type>
        <bootmenu enable='yes'/>
    </os>
    <clock offset='localtime'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
        <emulator>/usr/local/bin/qemu-system-aarch64</emulator>
        <controller type='usb' model='ehci'/>
        <disk type='file' device='disk'>
            <driver name='qemu' type='qcow2'/>
            <source file='/Users/darren/Documents/VMs/ubuntu1.qcow2'/>
            <target dev='vda' bus='virtio'/>
        </disk>
        <console type='pty'>
            <target type='serial'/>
        </console>
        <input type='tablet' bus='usb'/>
        <input type='keyboard' bus='usb'/>
        <graphics type='vnc' port='5900' listen='127.0.0.1'/>
        <video>
            <model type='virtio' vram='16384'/>
        </video>
    </devices>
    <seclabel type='none'/>
    <qemu:commandline>
        <qemu:arg value='-accel'/>
        <qemu:arg value='hvf'/>
        <qemu:arg value='-netdev'/>
        <qemu:arg value='user,id=n1,hostfwd=tcp::2222-:22'/>
        <qemu:arg value='-device'/>
        <qemu:arg value='virtio-gpu-pci,netdev=n1,bus=pcie.0,addr=0x19'/>
        <qemu:arg value='-bios'/>
        <qemu:arg value='/Users/darren/Documents/VMs/QEMU_EFI.fd'/>
    </qemu:commandline>
</domain>

Now I am getting this error: error: Failed to start domain 'ubuntu1' error: internal error: undefined hardware architecture

Upvotes: 0

Peter Maydell
Peter Maydell

Reputation: 11383

QEMU appears to be trying use the KVM accelerator, which obviously does not work on a macos host, so QEMU reports "invalid accelerator kvm". It then tries to fall back to TCG (pure emulation), which also fails, because of a macos bug involving mprotect(), hence "Could not allocate dynamic translator buffer". (Latest upstream QEMU has a workaround for this in commit c118881ee607dcac, but you don't need that since you're only running into it because hvf is not actually being used.)

I think that this is happening because your XML syntax is incorrect and so QEMU is not seeing your attempt to tell it to use hvf. Specifically, each "qemu:arg" tag in the XML must specify one single command line argument word, so if you want to say "-accel hvf" you must write

 <qemu:arg value='-accel'>
 <qemu:arg value='hvf'>

and so on for all the other options and arguments you want to specify. (You can see that the q35 options you've commented out are all specified in this way, unlike the ones you've added for the virt machine: look at the difference between the -netdev handling there and what you have, for example.)

If you try to pass a longer string in the XML as you have done, then QEMU will see it as a single long commandline argument with embedded spaces, which it will then probably treat as a very odd disk image filename or complain about as invalid.

Upvotes: 0

Related Questions