How well does SQM (cake) work on this?

Ten64 Forum
1

How much bandwidth can be shaped successfully with SQM (cake) on this platform?

2

Hi @dtaht we did look into SQM recently while troubleshooting a bufferbloat issue on LTE but haven’t done any comprehensive throughput benchmarks.

Is there a benchmark using flent or iperf you can recommend?

Ideally with multiple flows, unfortunately the softirq system on Linux does cause performance issues for ‘single-thread’ situations.(See Vyos Build - My Repo - #5 by mcbridematt for an example)

3

sample script: Flaws and features in the Flent network testing tool - http://blog.cerowrt.org/ - try the sydney server

My guess is cake will run out of a53 cpu at about 500Mbit.

My hope would be with fq_codel and bql on the ethernet ports you would do well at a gbit unshaped.

LTE is really problematic, however, try: GitHub - lynxthecat/cake-autorate: Sets CAKE bandwidth in dependence upon traffic load and latency

1 Like
4

We can do a bit more than that!

netdata-sqm-on
netdata-sqm-on1904×955 134 KB

These are in a lab with server and client connected via 10G SFPs.

The router is running OpenWrt main/master on kernel 6.1 using the recently merged armsr target.

Sadly, [affordable] residential FTTH plans in Australia only do 50Mbps upload (1G down) :frowning: so I don’t think a live test will be very interesting. I’ll keep this test in mind if I visit someone with a big link.

rrul:
 TCP upload sum                    :      1956.52          N/A          N/A Mbits/s        1400
rrrul_be:
 TCP upload sum                    :      1613.04          N/A          N/A Mbits/s        1400
tcp_nup:
 TCP upload sum                   :      1376.97          N/A          N/A Mbits/s        1400

Full result set here:

Here is what the CPU cores look like during the tests. You can see saturation on core 1 here:

sqm-cpu-saturation
sqm-cpu-saturation1726×778 61.7 KB

If there were an SQM implementation that hooks onto XDP rather than tc then I think there would be a significant uplift in performance.

Results without SQM for comparison:

netdata-sqm-off-cropped
netdata-sqm-off-cropped1550×886 111 KB

rrul:
 TCP upload sum                    :      2728.44          N/A          N/A Mbits/s        1400

I can email you the flent result files, if you would like to see them.

5

yea, I prefer the flent files, please send. Didyou have ack fltering on the upload enabled?

6

Done

I used the default settings on luci-app-sqm with bandwidth set to 10Gbit in both directions, e.g:

config queue 'eth8'
        option enabled '1'
        option interface 'eth8'
        option download '10000000'
        option upload '10000000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'none'
        option debug_logging '0'
        option verbosity '5'

which gives rise to these options:

tc -s qdisc show dev eth8
qdisc cake 8011: root refcnt 129 bandwidth 10Gbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 0b of 15140Kb
 capacity estimate: 10Gbit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0

Is there anything else I should try?

That also reminds me, OpenWrt changes the memory limit on fq_codel for non-x86 devices to 4MBytes.
I think Arm64 devices should be excluded from this.Would that cause any significant effects?