I do like the idea of being able to use uci-commands to configure the VM, however I wonder why libvirt is not employed underneath? OpenWRT seems to provide libvirt support: [OpenWrt Wiki] OpenWrt in QEMU (somewhere in the document mentioned).
Especially the huge amount of bash code here: muvirt/files/muvirt.init · master · traversetech / muvirt-feed · GitLab highlights to me that libvirt would be an excellent replacement for this. When looking even deeper into uVirt, there is even more bash code that seems just to circumvent libvirt.
My idea: One could use uVirt to provide the LUCI and UCI handling, while also being able to use /etc/config/virt for setting it up, while the creation of the appropriate KVM/QEMU commands could then be done by libvirt? @mcbridematt any thoughts about this?
The main reason it hasn’t happened is that we haven’t come across anything too difficult to do without libvirt. Things like snapshots and live migrations would be where I would draw the line and have libvirt do the job but I haven’t seen a need to implement those yet.
As far as I am aware there has not been a port of libvirt to OpenWrt yet (the wiki page you linked concerns running OpenWrt as a guest). It’s likely not going to be very difficult to do so, but is another consideration.
Another factor is if having libvirt will get in the way of any future advanced feature development, such as automating some DPAA2 passthrough/VFIO use cases (like DPDMUX). I see μVirt as a testing ground for such ideas, and having to hack “through” another API would make implementing these things harder. The k3os automated setup is one example where driving QEMU directly made it easier to implement.
Indeed some of the shell/bash code is quite messy. Most new development has been in Lua, which is more “verbose” than shell but is more robust and has access to better libraries (like penlight and the libraries used by LuCI). The init script could be a candidate for rewriting in Lua.
Fair points. While I can truly follow the reasoning behind DPAA2 passthrough, I do not so much for k3os. If I see it correctly, that could also be handled with libvirt. While I stop to follow with the DPDMUX: imagine I could express with the libvirt syntax a bridge device to clutter together all the components of interest. And in the beginning it just uses a normal Linux bridge. And at one point in time, DPDMUX is ready and internally libvirt just uses it transparently as soon as libvirt is updated.
When the crowdsupply campaign was still running, you mentioned once to me (by email) that your intention is to upstream as many aspects of Ten64 as possible. I truly liked the idea, as it would make the platform simpler to maintain. While at the same time, I imagine the more is upstream the more the platform becomes attractive. As of this, I wonder if libvirt could be a nice internal replacement for uVirt, as the maintenance of the platform in the long run would become cheaper as long as the patches get upstreamed?
While I looked into libvirt in the past (when I was just a student - long time ago: [libvirt] Rudimentary (basic) s390x architecture functions for libvirt), I remember that it merely just employs Linux and the typical Linux facilities. I wonder if it would be truly hard to get it running on OpenWRT, as OpenWRT is yet another Linux distribution.
That still is my attitude. For now most of this work is on the lower levels (such as U-Boot, kernel and OpenWrt) which will give the biggest benefit to us and the community.
Having the DTS upstream doesn’t immediately change anything for users (since we consider the DTS part of the firmware), but various other projects (such as U-Boot) expect it to be there.
U-Boot is the next challenge and I expect it will take several attempts to get basic Ten64 support accepted (not counting some drivers that have to be added or fixed).
There are also some issues in U-Boot (specific to DPAA2/fsl-lsch3 generation hardware) we may need to “force” a resolution on at the same time.
Due to the ongoing migration(s) to “driver model” it is very time consuming to catch up with changes to U-Boot. Having the Ten64 board in the tree means a lot of these migrations will be done by the “robots” every time there are changes.
Plus there is the Microchip/SMSC sensor drivers and the OpenWrt arm64 target.
(There have been a few other patches arising out of Ten64 development that have been submitted to U-Boot or OpenWrt already)
There are also relevant patches written by NXP or others that aren’t yet in the kernel (like the VFIO patches identified in the restool thread). Some of these are for “political” reasons (the kernel developers worldview disagrees with other users of the hardware), lack of (perceived) upstream interest or other priorities. We will have to start advocating for those to be resolved.
Apologies if I gave a different impression but hopefully this gives a good view of how we intend to interact with upstream projects at this time. Once this is done it will give more room to do “value-add” work