Most Vagrant boxes out there are not compatible with Apple Silicon arm64 chips so we have to make our own. Should also come in handy when testing systems deployed to arm64 VMs in the cloud. This folder contains automated builders of base images and boxes for Vagrant and other hardware or virtual systems using Packer, Virtualbox, VMWare, QEMU, and Parallels.
Current state of the builders
| Builder/App | Status | Reason |
|---|---|---|
| Virtalbox | Blocked |
Virtualbox does not fully support arm64 on Apple Silicon yet |
| VMWare | Done-ish |
Still has a manual "reboot using VMware Fusion before it'll fully provision |
| QEMU | Blocked |
Currently blocked by lack of support for Apple's vmnet on Apple's hypervisor framework, hvf |
| Parallels | Not started |
Apple moved away from traditional TUN/TAP infra for virtualization starting with the M1. They created their own hypervisor framework (hvf) and added entitlements that signed apps need to have in order to use this framework in user space, w/o running as root. When combined these work together to automate the creation of network interfaces and bridges that you can attach to VMs like normal. Apple asks you to talk to your sales rep to get official access to the entitlement. This has created another hurdle to overcome when trying to virtualize arm64 machines on macOS.
Current state of virtualization software
- Virtualbox is in beta, but currently crashes when booting a barebones Ubuntu 20.04 arm64 VM.
- QEMU supports
hvfandvmnet, but it is not signed and doesn't seem to have plans to be signed.- Currently it does not ship with
vmnetsupport built in nor does it seem you can build from source to bake it in easily.--enable-vnmetdid not produce a build that includedvmnet-*devices. - Libvirt has the same signing and entitlement issues when trying to create network interfaces
- The UTM app has custom built QEMU and other tools and has full signed support of
vmnet, but you can't use it with Packer. Manual box builds to raw or qcow2 - I'm uncertain if qcow2 or raw images are supported by Vagrant.
- Currently it does not ship with
- VMWare: TBD
- Parallels: TBD - Might be the best option right now given that they are close partners with Apple and claim they support arm64 on Apple Silicon, including automating with Packer.
Packer
Linux
QEMU
KVM, Libvirt, and friends
Can't use vmnet-(host|bridged|shared) devices on macOS.
Can't create a properly named bridge interface on macOS
There is a really strange issue where I need to manually restart the VM using VMware itsself before the NoCloud Cloud-Init configuration will run. It hangs right after (or during?) the network interfaces are loaded and up. Seems to be the metadata crawler gets stuck with infinite timeouts?
When not using auto-conf and using plain cloud-init via mounted cdrom the same issues happen. Pretty sure its VMware being really wonky. Even when trying Floppy, the floppy won't mount even when explicitly told to via vmx_data.
Failing to unmount cdrom (ISOs) leading to installer boot loops
- https://bugs.launchpad.net/subiquity/+bug/1901397
- hashicorp/packer-plugin-qemu#66 (comment)
Note that
sedis also available so it doesn't have to beperldoing the regex - https://www.linuxbabe.com/command-line/how-to-use-linux-efibootmgr-examples
Failing to unmount cdrom (ISOs) leading to installer boot loops
Failing to apply a proper Netplan config leading to a hung boot at cloudinit[678]