genet MTU for upstream - #7617
Conversation
|
39dfaf2 contains a brutal approach to make MTU 9000 work (without any offloading). Performs quite ok, but needs more testing |
bcmgenet_hfb_init() runs INIT_LIST_HEAD() on priv->rxnfc_list, which drops every rule off the list, and bcmgenet_open() calls it on each ifup. Every rule the user configured is silently lost: # ethtool -N eth0 flow-type ether dst $MAC action 0 Added rule with ID 0 # ethtool -n eth0 | grep -c Filter: 1 # ip link set eth0 down && ip link set eth0 up # ethtool -n eth0 | grep -c Filter: 0 Initialise the lists once at probe and restore the rules on open, as bcmgenet_resume() already does. Fixes: 3e37095 ("net: bcmgenet: add support for ethtool rxnfc flows") Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
bcmgenet_netif_stop() disables the Tx queues first and only stops Tx NAPI several steps later. A completion already in flight calls netif_tx_wake_queue() in between, so a queue runs again while bcmgenet_dma_teardown() and bcmgenet_fini_dma() free the rings, and a transmit entering that window touches freed control blocks. The close path is not affected because dev_deactivate_many() stops the qdisc before ndo_stop() runs. bcmgenet_suspend() and the MTU change added later in this series leave the qdisc running, so both can hit it. Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
bcmgenet_netif_stop() already takes stop_phy. Give the start side the same choice so a caller that left the PHY running can bring the datapath back without tripping the phy_start() state check. No functional change. Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
ENET_MAX_MTU_SIZE holds a frame length, not an MTU. Both users program it into hardware that wants a frame length, so the name will mislead once the MTU is no longer fixed at ETH_DATA_LEN. Name the receive offset too, which was open coded as 66. No functional change. Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
The driver never sets dev->max_mtu, so the MTU is stuck at ETH_DATA_LEN. Raising it alone does not work: the RBUF and TBUF packet ready thresholds cut a frame off at their 2048 byte reset default. Receive then finds no end of packet marker and drops the frame as fragmented, and transmit takes the frame but never puts it on the wire. Program both thresholds from the configured MTU and size the DMA buffers to hold exactly what the threshold permits. The registers are 8 bit in units of 16 bytes and want a multiple of the 256 byte burst size, so 0xf0 is the largest usable value. That leaves an MTU of 3820 once the alignment bytes, the Ethernet header and a VLAN tag are taken off. Changing the MTU only has to resize the buffers and rewrite those registers, so the PHY keeps running and the link stays up. Link: raspberrypi#5561 Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
39dfaf2 to
c702d70
Compare
RBUF_PKT_RDY_THLD is where the status block gets written, not a limit on the frame. With the status blocks off jumbo frames seem to come through. Costs both checksum offloads and all but one transmit queue. Signed-off-by: Nicolai Buchwitz <nb@tipi-net.de>
c702d70 to
66f6684
Compare
|
I'm still waiting on documentation from Broadcom to read the official word on how jumbo frames with offload was meant to work (if it was). Seeing as it was the offloading headers that seemed to cause issues, I did wonder if disabling offloading would allow it to work with bigger buffers. I only had a very quick read through the patches, but wonder if we can "dynamically" disable offload when the mtu is increased above the magic threshold. Possibly not based on the comment of losing the queues as well. I had considered VLAN headers, but didn't know the answer off the top of my head, and wasn't in a position to set up VLANs to test. Thanks for taking care of it. I'm not fussed over |
|
@ffainelli and @Ryceancurry if you can spare some time, your thoughts on this would be really appreciated (as always). Thanks! |
|
https://lore.kernel.org/netdev/20260406-devel-autonomous-eee-v1-1-b335e7143711@tipi-net.de/t/
https://datasheets.raspberrypi.com/cm4/cm4-datasheet.pdf#page=7
https://www.broadcom.com/products/ethernet-connectivity/phy-and-poe/copper/gigabit/bcm54210
https://magazine.raspberrypi.com/articles/raspberry-pi-4-in-detail
https://www.broadcom.com/products/ethernet-connectivity/phy-and-poe/copper/gigabit/bcm54213
While we do understand the importance of taking one step at a time, it would be great to test whether |
3 participants
Based on @6by9's #7614, with the goal of upstreaming the MTU support.
I've tested the original patch on CM4 and discovered some issues. So I've created a slightly different patch (series) which I intend to send to netdev. It also contains some fixes Sashiko would have flagged any way...
TBUF_PKT_RDY_THLD (TBUF + 0x10)is never programmed and it stays at0x80. At MTU3824TX iperf3 is stuck at 0.00 Mbit/s while ping works and the link is up.tx_pkts rises, buttx_good_pktsdoesn't. Kudos to @wtschueller who discovered this Jumbo frame support on Pi4 ethernet (Genet) #5561UMAC_MAX_FRAME_LENgets the MTU value, but it's a frame length and counts the FCS. Frames from3824up result inrx_length_errors(at least in my testing), so the real limit seems to be MTU3806.RX_BUF_LENGTH 10240costs no throughput (936/941 at MTU 1500, same as unpatched) but is above KMALLOC_MAX_CACHE_SIZE on arm64, thus it cant hurt to derive it from the MTU instead.0xf0seems to be the real limit:0xfbreceives fine but resulted in TX hard-hung on my setup.Happy to add @6by9 as Co-developed-by since it's based on your findings. But this requires a Signed-off, which I wouldn't add without consent.