Ten parcels, one envelope: Linux UDP GSO and Felix's MTU cap

Felix speaks QUIC, and QUIC does a lot of its work per packet. Bigger packets mean fewer of them, so raising the MTU looked like free throughput. On my Mac it was. On Linux, the same setting stopped delivery dead, and it never came back.

The cause was a kernel feature working exactly as designed and a fallback that listened for the wrong error. Felix now caps any MTU override at 6,550 bytes on Linux and bounds path MTU discovery at 4,096 by default.

What we found

Linux has UDP generic segmentation offload (GSO). The sender hands the kernel one big buffer plus a segment size, and the kernel cuts it into packets. One sendmsg call can carry many QUIC packets, which saves a lot of syscalls.

The catch is that the kernel still treats that buffer as one UDP datagram until it splits it. A UDP datagram's payload over IPv4 tops out at 65,507 bytes: the 65,535 the IP length field allows, less 20 bytes of IP header and 8 of UDP. So the limit applies to the whole batch, not to each packet.

quinn, the QUIC library Felix uses, batches up to 10 segments per send. Each segment is the current path MTU. At an MTU of 4,096 a batch is 40,960 bytes and fits. At 16,336, which is what we had set for loopback, a batch is 163,360 bytes. The kernel refuses every one with EMSGSIZE.

That alone would only be an error. What made it a stall is the fallback. quinn-udp does know GSO can fail. Its send path turns segmentation off when sendmsg returns EIO or EINVAL, because, as its source comment puts it, Linux "offers no easy way for us to detect this short of an EIO or sometimes EINVAL" (quinn-udp/src/unix.rs). EMSGSIZE is neither, so GSO stays on, and the next batch fails the same way.

Why it matters

A batch the kernel refuses is a batch quinn has already counted as sent. Its loss detector then sees those packets never get acknowledged, the congestion window collapses, and the connection grinds to a halt. Subscribers stop getting records, and the sender keeps trying the same doomed batch size.

The threshold is 65,507 / 10, so just over 6,550. Anyone running Felix on a jumbo-frame network would hit it. That is exactly the network you pay for when you want throughput, and path MTU discovery would happily climb past 6,550 there. Before the fix, every perf session set FELIX_MTU_UPPER_BOUND=4096 by hand to avoid it. Nobody else would have known to.

How we found it

Slowly, and by being wrong several times. The symptom was Linux runs that sometimes ran fine and sometimes stalled completely. The same configuration was fine on macOS over hundreds of runs.

My first guess was that jumbo datagrams just don't work on Linux. But an initial MTU of 16,336 without other changes ran at 507-671 MB/s with 23 lost packets out of 76,282. Then I tried a larger initial congestion window and turning off ACK-frequency tuning. Neither changed anything.

Next I blamed socket buffers. I checked /proc/net/snmp before and after a failing run: RcvbufErrors=0, SndbufErrors=0, InErrors=0. The kernel had dropped nothing. Meanwhile quinn reported 155,037 lost packets out of 226,986, about 68%. So I wrote it up as spurious loss, quinn's loss detector misjudging packets the kernel had delivered. That was wrong too.

The variable that actually split working from failing runs was min_mtu. For loopback, we had pinned min_mtu to the same size as the initial MTU, so a black-hole verdict couldn't collapse the connection to 1,200 bytes. With that pin, 16,336 and 8,192 failed, while 4,096 and 2,048 ran at 788 and 575 MB/s. For a while I thought the pin caused the loss.

It didn't. Reading quinn's source showed that min_mtu only feeds black-hole detection. The real story was the GSO arithmetic above, and the pin's role was to take away the escape hatch. Unpinned, quinn sees heavy loss, decides the path is a black hole, and drops to 1,200. Ten segments of 1,200 fit easily, so the connection recovers. With the floor pinned high, there was nowhere to fall.

With that model, the earlier oddities all made sense. RcvbufErrors was zero because the datagrams were refused at the syscall and never reached a buffer. The gap between quinn's 226,986 sent and the kernel's roughly 64,500 was the refused batches. The 68% loss was real, not spurious. And macOS was immune because it has no GSO. Its counters showed one syscall per datagram, so there was never a batch to refuse.

The fix

The fix is a ceiling and a default, in felix-transport's config. An override of FELIX_INITIAL_MTU or FELIX_MTU_UPPER_BOUND is clamped to 6,550 on Linux, and a warning says so. Discovery is bounded at 4,096 by default:

const fn mtu_discovery_upper_bound_for(macos: bool) -> u16 {
    if macos { 16384 } else { 4096 }
}

const fn mtu_override_ceiling_for(macos: bool) -> u16 {
    if macos { 65527 } else { 6550 }
}

fn clamp_mtu_override(name: &str, value: u64, ceiling: u16) -> u16 {
    let clamped = value.clamp(1200, u64::from(ceiling)) as u16;
    if u64::from(clamped) != value {
        tracing::warn!(name, requested = value, used = clamped,
            "MTU override out of range; clamped");
    }
    clamped
}

The ceiling was first 6,553, which is 65,535 / 10. A later review caught that this forgot the headers: 10 x 6,553 is 65,530, over 65,507. So 6,551-6,553 could still stall. It is now 6,550, and 10 x 6,550 is 65,500 (#754).

Both functions take the platform as an argument instead of reading cfg! inside. That way a test on a Mac can still check the Linux values. The value that matters is the Linux one, and a test that only checks the platform you're editing on is worth very little.

The default is 4,096 rather than 6,550 for two reasons. quinn's batch size of 10 is a private constant, so Felix can't read it through quinn's API. If it ever rises to 15, 6,550 breaks immediately, while 4,096 still fits. And I couldn't derive the cap from max_gso_segments() instead: that is 64 on Linux, and 65,535 / 64 is 1,023, below QUIC's 1,200 floor.

The nice surprise is that 4,096 was also the fastest thing we measured on Linux loopback. With the floor pinned at 4,096, a run did 1,166 MB/s at fanout 1 and 182 MB/s per subscriber at fanout 10. An explicit MTU of 10,240, unpinned, did 786 and 112. The same 4,096 without the pin did 513, because pinning also freezes discovery, so no probes and no black-hole verdicts ever run.

On real hardware

Loopback is one thing, so we also tried it on Azure VMs with Mellanox (mlx5) NICs. Raising the NIC MTU from 1,500 to 3,900 needed no code change. quinn discovered the larger path, and the 4,096 default bound already covered it. I checked the path first with full-size, don't-fragment pings from every load generator.

With four listeners, throughput went from 2,156 MB/s to 3,995 MB/s, about +85%. Per broker core it went from 300 to 537 MB/s, about +79%. With one listener it roughly doubled, from 831 to 1,724 MB/s (#721).

My per-byte model had predicted +20-35%. The real gain was much bigger because most of the cost was per packet, not per byte. QUIC decrypts, parses and ACKs in user space for every packet, and the kernel's receive path is per packet too. At 3,900 there were 2.67 times fewer packets, and the kernel receive cost per gigabyte fell about 45%.

Two caveats. Azure supports an MTU above 1,500 only inside a VNet or same-region peering, so this was VNet-internal traffic. Anything crossing the internet is still at 1,500. And the gain is specific to UDP and QUIC. TCP hands the kernel a byte stream, and TCP segmentation offload and GRO cut and coalesce it in large units even at 1,500 (kernel docs, Segmentation Offloads). A QUIC stack doesn't get that for free, so the MTU matters much more.

What it costs or where it stops

On Linux, Felix gives up packets larger than 6,550 bytes, and by default larger than 4,096. On a 9,000-byte jumbo network, discovery stops well short of the wire's limit. On loopback that measured as a win, not a loss, but I haven't measured a 9,000-byte NIC.

There's also a gap in what we know. The chain above needs the errno to be something other than EIO or EINVAL. The sustained failure implies that, but I never confirmed it with strace. EMSGSIZE is what the kernel should return for an oversized datagram, and it's what the code comments say, but I haven't seen it in a trace. A one-run check would be strace -f -e trace=sendmsg,sendmmsg -c on the failing config.

The cleaner fix would be to turn GSO off on paths with big datagrams and keep the full MTU, which is what macOS does already. That needs a tuned Linux host to measure before it ships.

What we learned

An offload changes the unit the kernel limits. Every per-packet limit I knew about was fine at 16,336. The limit that bit was on the batch, which only exists because GSO exists.

A fallback that keys on specific errnos can turn a recoverable error into a permanent stall. quinn-udp's check is reasonable for the failure it was written for, a NIC without GSO. It just wasn't written for this one, and an unhandled error that repeats forever looks exactly like congestion.

The platform you develop on can hide the bug. macOS has no GSO, so every run on my laptop was fine. A safety mechanism, black-hole detection, was also quietly saving Linux runs until we pinned it away.

What we'll do differently

Any transport setting gets measured on Linux before it becomes a default, and tests check the Linux branch even when they run on a Mac. When a counter says the kernel dropped nothing but the library says it lost 68%, I'll look at the syscall return values first, not the loss detector.

Trying it

Run docker compose up from the repo root (it uses compose.yaml). The MTU knobs are FELIX_INITIAL_MTU and FELIX_MTU_UPPER_BOUND. On Linux, anything above 6,550 is clamped, with a warning in the log.

Before raising either, check what your path actually carries. On Linux, ping -M do -s 3872 <peer> sends a 3,900-byte packet (3,872 bytes of payload plus 28 of headers) with fragmentation forbidden. If it fails, your path MTU is smaller. The environment variable reference has the details.

docker compose up -d