How much bandwidth can be shaped successfully with SQM (cake) on this platform?
Hi @dtaht we did look into SQM recently while troubleshooting a bufferbloat issue on LTE but haven’t done any comprehensive throughput benchmarks.
Is there a benchmark using flent or iperf you can recommend?
Ideally with multiple flows, unfortunately the softirq system on Linux does cause performance issues for ‘single-thread’ situations.(See Vyos Build - My Repo - #5 by mcbridematt for an example)
sample script: Flaws and features in the Flent network testing tool - http://blog.cerowrt.org/ - try the sydney server
My guess is cake will run out of a53 cpu at about 500Mbit.
My hope would be with fq_codel and bql on the ethernet ports you would do well at a gbit unshaped.
LTE is really problematic, however, try: GitHub - lynxthecat/cake-autorate: Sets CAKE bandwidth in dependence upon traffic load and latency
We can do a bit more than that!
These are in a lab with server and client connected via 10G SFPs.
The router is running OpenWrt main/master on kernel 6.1 using the recently merged armsr target.
Sadly, [affordable] residential FTTH plans in Australia only do 50Mbps upload (1G down) so I don’t think a live test will be very interesting. I’ll keep this test in mind if I visit someone with a big link.
rrul: TCP upload sum : 1956.52 N/A N/A Mbits/s 1400 rrrul_be: TCP upload sum : 1613.04 N/A N/A Mbits/s 1400 tcp_nup: TCP upload sum : 1376.97 N/A N/A Mbits/s 1400
Full result set here:
Here is what the CPU cores look like during the tests. You can see saturation on core 1 here:
If there were an SQM implementation that hooks onto XDP rather than tc then I think there would be a significant uplift in performance.
Results without SQM for comparison:
rrul: TCP upload sum : 2728.44 N/A N/A Mbits/s 1400
I can email you the flent result files, if you would like to see them.
yea, I prefer the flent files, please send. Didyou have ack fltering on the upload enabled?
I used the default settings on luci-app-sqm with bandwidth set to 10Gbit in both directions, e.g:
config queue 'eth8' option enabled '1' option interface 'eth8' option download '10000000' option upload '10000000' option qdisc 'cake' option script 'piece_of_cake.qos' option linklayer 'none' option debug_logging '0' option verbosity '5'
which gives rise to these options:
tc -s qdisc show dev eth8 qdisc cake 8011: root refcnt 129 bandwidth 10Gbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 memory used: 0b of 15140Kb capacity estimate: 10Gbit min/max network layer size: 65535 / 0 min/max overhead-adjusted size: 65535 / 0
Is there anything else I should try?
That also reminds me, OpenWrt changes the memory limit on fq_codel for non-x86 devices to 4MBytes.
I think Arm64 devices should be excluded from this.Would that cause any significant effects?