Vyos Build - My Repo

Hey all,

I have managed to get a reliable build of Vyos (finally :slight_smile: ) using this repo for people who are interested https://github.com/grahamhayes/vyos-build/

I did not run it in docker. The steps I did are:

Build kernel from the vyos-build/packages/linux-kernel at ten64-build · grahamhayes/vyos-build · GitHub folder.

Build sensors package (not 100% this is working properly tbh) using the out-of-tree.sh file, and copying the output up one folder.

Get the telegraf arm64 deb and put it in the packages folder.

Build vyos-utils from the GitHub - vyos/vyos-utils repo, and copy the deb to the packages folder.

then just sudo make arm64, and copy the qcow to recovery and use the write-image-disk command in recovery to write to the disk.

Vyos then just booted for me, and seems to be working. Haven’t done any speedtests yet to see what perf is like so ymmv.

1 Like

Out of interest, what was involved in making this work?

A lot of testing, cursing, and trial and error tbh. I started from @mcbridematt 's https://github.com/mcbridematt/vyos-arm64-builder repo, and then moved to trying to keep the patches on top of the vyos-bulld repo current branch.

The main areas of work where in the kernel config (which I am sure is terrible, but I haven’t had time to go clean it up), and finding arm64 versions of packages.

I did have to build it on an arm64 host as well - I ended up using Oracle Cloud for that, with free tier and the arm accelerator program for credits.

1 Like

It is hosting a full IPv6 BGP view pretty well, and I haven’t noticed any major latency / speed issues using a 1gig internet connection, but multi gig speeds are still tbd.

Hi all,
I was asked about this by a customer, so I went and updated my build:

The mistake I made last time was not to ‘pin’ each package version, so the builder broke horribly after recent changes in VyOS-current.
I have now ‘pinned’ everything so if you clone this, it should produce the “same” image I just built.
I also wrote a “local” Jenkinsfile runner rather than doing each package by hand.

There was also a recent change to the image generation in vyos-build which meant we can’t build qcow2 images at the moment. I had to fight my way through a bunch of live-boot and GRUB issues as a result. But this also means the install and update process will work exactly the same as it does on x86.

Bonus: XDP works!

iperf3 Gbit/s throughput iperf -P1 iperf -P4 iperf -P6 iperf -P8
OpenWrt 1.66 6.28 9.11 7.42
VyOS normal 1.93 5.37 8.52 9.41
VyOS XDP 5.72 9.41 9.4 9.41
XDP improvement 2.96x 1.75x 1.10x 1.00x

I reckon the ‘single’ thread speed is capped by my iperf client so I’ll go setup MoonGen for some proper tests.
As the VyOS documentation notes, the XDP path doesn’t work with firewall and NAT. I’m still looking at ways to speed that up.

If you want a copy of the VyOS ISO, DM me. I’ve been reluctant to publish them given the VyOS project depends on subscriptions, but I’ll reach out to them about it.

I would like to offer a suggestion regarding the testing methodology. While using iperf(3) is valuable, it is worth noting that it may lead to misleading results. In order to achieve more accurate measurements, I recommend considering the use of TRex test tool. Furthermore, including session counts in the test would provide a more comprehensive evaluation.
I believe implementing these adjustments will enhance the accuracy and effectiveness of the testing process.
Congratulations on your great work, and thank you for considering my input.

ik keep running into the issue of missing owamp dependencies; the build command to compile those dependencies isn’t working, so i can’t really build them manually as well

does someone know a workaround?

^ @mcbridematt version

Sorry, I’ve been too busy to update the VyOS builder. It usually takes a couple of days to rebase all the tree changes and prepare a new set of patches.
I am looking to revisit it when VPP on VyOS progresses more.

If you are stuck with a PowerDNS repository error, it should be easy to fix: Invalid signatures (EXPKEYSIG) for PowerDNS Release Signing Key · Issue #5 · mcbridematt/vyos-arm64-builder · GitHub

There was also another arm64 builder script posted recently, I understand it generates an EFI iso, but not sure if it has the right kernel options to work on a Ten64.

Thanks for the quick response @mcbridematt! No need to apologise, already appreciate the effort you put in this.

I was able to fix the sign issues, however, the current issue is with the OWAMP dependencies. The script isn’t able to build the corresponding projects correctly and I’ve tried different things like manually installing the OWAMP dependencies, but it keeps failing. Any idea whether those dependencies are really needed and if not, how I can remove these from the project (along with all references)? Perhaps that could do the trick for the time being.

I have updated the VyOS builder:

A couple of patches (like changes to GRUB) have been sent and merged into VyOS already. The remaining changes are mostly hacks around kernel configurations and command line.