Proxmox/PiMox installer for Ten64

Ten64 Forum
21

Should I also be using a particular firmware?

22

Good question! Iā€™m not aware of anything too different between the two firmware series that would affect VFIO.

I know the IOMMU initial configuration is done by U-Boot, so there could be something in there.
Itā€™s worth trying the newest v0.9 firmware (v0.9.1) just to see if it makes a difference.

Another thing worth trying is to make vfio-pci bind to the card immediately at boot.
You can add vfio_pci.ids=14c3:7906 to the kernel command line (in /etc/default/grub ā†’ update-grub) to make the vfio-pci driver bind to it on probe.

dmesg | grep -E '(vfio|0001:03)'
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.1.76-traverse-lts-6-1+109-00021-g9889cb2828ba root=UUID=1bc72a0c-9d13-4a2d-86c3-a929bd2b637f ro earlycon arm-smmu.disable_bypass=0 net.ifnames=0 vfio-pci.disable_idle_d3=1 vfio_pci.ids=14c3:7906
[    1.980544] pci 0001:03:00.0: [14c3:7906] type 00 class 0x028000
[    1.980652] pci 0001:03:00.0: reg 0x10: [mem 0x2840000000-0x28400fffff 64bit pref]
[    1.980727] pci 0001:03:00.0: reg 0x18: [mem 0x2840100000-0x2840107fff 64bit]
[    1.980799] pci 0001:03:00.0: reg 0x20: [mem 0x2840108000-0x2840108fff 64bit pref]
[    1.981175] pci 0001:03:00.0: PME# supported from D0 D3hot D3cold
[    1.981411] pci 0001:03:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0001:00:00.0 (capable of 8.000 Gb/s with 5.0 GT/s PCIe x2 link)
[    1.982190] pci 0001:03:00.0: BAR 0: assigned [mem 0x2840100000-0x28401fffff 64bit pref]
[    1.982243] pci 0001:03:00.0: BAR 2: assigned [mem 0x2840000000-0x2840007fff 64bit]
[    1.982295] pci 0001:03:00.0: BAR 4: assigned [mem 0x2840200000-0x2840200fff 64bit pref]
[    4.463752] vfio-pci 0001:03:00.0: Adding to iommu group 1
[    4.469584] vfio_pci: add [14c3:7906[ffffffff:ffffffff]] class 0x000000/00000000

If neither of those suggestions help, it would be good if you can upload the full contents of dmesg.

23

I just loaded the new firmware and vfio is working :star_struck:

The IOMMU groups are numbered differnetly in this one.

Just got to get the OpenWRT vm guest configured.

Iā€™ve got the dmesg output before the firmware upgrade, if you want a copy. Just tell me the best way to share it.

1 Like
24

It seems this is now broken as the person who ran the mirror used in the script has disabled that repo and moved their Proxmox to (what looks like a Docker Container)

NOTE!

The project has received some donations, but these donations (please refer to the donation list in SUPPORT.md) are not sufficient to cover our expenses on warehouse servers, compilation servers, and other related costs. Therefore, we will stop the distribution of pveportā€™s deb files and ISO images to free up more space for PXVIRT.

The original Proxmox-Port repository will be cancelled. If you want to get updates, please visit docs.pxvirt.lierfang.com to get the latest documentation information!

So Yeah, a bit upsetting. :frowning: seems to be all completely down/removed atm.

25

It doesnā€™t look as bad, he (jiangcuo) has forked Proxmox to add his own features. In my personal opinion that is the right thing to do, especially when the parent project has itā€™s own commercial sponsor (and therefore is sensitive to other parties using the project trademarks)

There appears to be some new repositories here for the PxVirt fork:

The docker container appears to be the build environment and it looks like you can use it to build all the *.deb packages yourself. (I might give it a go sometimeā€¦)

From my personal experience (e.g VyOS), ā€œfollowingā€ any Debian based project outside itā€™s main infrastructure is quite a lot of work. I can sympathize with him for pulling his public repo down.

26

I re-pointed my Promox install to that new repository and have no issues.

Though I see version 9 of Promox is now up, so upgrading to the next Debian 13 / Trixie will be the next major change when Jiangcuo publishes that version.