Running VMs on OpenWRT

Ten64 Forum
1

Hi,
I realise that muvirt is an option, but I’m currently running OpenWrt 22.03.5+traverse, 852551817. I’d like to try to run HomeAssistant as a VM as the Ten64 has plenty of RAM free.

What would be the best way to run an aarch64 VM with the ability to passthrough one of the USB devices? (a Zigbee radio)

Thanks again,
Adam

2

The Traverse OpenWrt 22.03 doesn’t have KVM/virtualization enabled.

However, in the new upstreamed OpenWrt (armsr) I have a pending commit to enable KVM, and once that is in the upstream OpenWrt, I will submit the rest of the stuff that goes with it (e.g vhost-user, vfio and qemu host). Hopefully these will all make it into the upcoming 23.05 stable release.

Once that is done, you will be able to run qemu-system-aarch64 on an OpenWrt image (ours or OpenWrt.org) as normal. Theoretically, you will be able to install the muvirt core package (to run VMs with an OpenWrt init script and UCI configuration) onto normal OpenWrt as well.

2 Likes
3

Fantastic news. It’s great to see it all coming together.

4

Update: The 23.05-rc4 release has all the bits needed for QEMU/KVM to work! Including extras like vhost-net and vfio-pci. As has the main branch for a few weeks now…

The following packages are needed:

opkg update
opkg install qemu-aarch64-softmmu qemu-bridge-helper kmod-vhost-net kmod-tun

A virtual machine (Arm64 EFI guest) can be started like so:

$ cat startvm.sh
#!/bin/sh
qemu-system-aarch64 -nographic -M virt -cpu host --enable-kvm \
        -bios u-boot.bin -smp 1 -m 1024 \
        -drive file=openwrt-armsr-armv8-generic-ext4-combined.img,format=raw,index=0,media=disk \
        -device "virtio-net,netdev=landev,disable-legacy=off,disable-modern=off" \
        -netdev "tap,id=landev,helper=/usr/lib/qemu-bridge-helper --br=br-lan,vhost=on"
$ ./startvm.sh

If the guest VM is EFI based, you will need a EDK or U-Boot binary for the QEMU virt machine. A workable u-boot binary can be found in the armsr/armv8/u-boot-qemu_armv8 folder on the OpenWrt archive server, or an EDK from somewhere like Unofficial EDK2 nightly build | EDK2 Nightly Build (unlike x86, there is no qemu-firmware-* package for arm64 right now)

The muvirt package does not work on mainline OpenWrt yet, it needs a small function to be added to the procd shell library (can be edited onto a running system). (see procd: allow jail parameters to be set from initscripts by mcbridematt · Pull Request #11859 · openwrt/openwrt · GitHub )
I might just re-implement it without the procd patch instead, it is just used to easily grab the PID of the container/jail the console server runs under.

5

Having some fun here:

(none of this appears to be directly Ten64 related but thought it worth documenting)