Chute Devs

The Last 700 Megabits

Hello everyone.

An Apple TV 4K on a wired gigabit network, running a direct Speedtest through Chute, downloaded at 285–352 Mbps. The same test with the VPN off reached 933–935. Upload was unaffected. The gap was not encryption, and it was not the proxy: it was the tunnel’s packet path, doing one packet’s worth of work at a time.

Earlier this year we wrote about doubling TUN throughput on the Mac. That work was about a single flow and the receive window. This one is about many connections and the write path, and it ends a good deal closer to the wire.

Where the Core Went

A direct download through the tunnel has two directions, and they cost different things. The receive side hands packets to the user-space TCP/IP stack. The send side takes the stack’s output, writes it to the tunnel, and waits to be told the data was taken.

On the Apple TV, the tunnel extension was using about 1.7 cores during the test. The stack’s queue was busy roughly 750 ms out of every second, and about 400 ms of that was the output path: every packet the stack produced became its own write to the tunnel, one system call each, under a lock that the read loop also takes.

The pump made it worse. It is stop-and-wait: it hands the stack one read and waits for all of that data to be acknowledged before it asks for the next. With an 8 KiB direct send buffer, a 32 KiB read took four rounds, and each round waited behind a queue that was already three quarters busy.

That is why a single flow was fine. One flow at about 600 Mbps was as fast before the change as after it; it never filled the queue. Speedtest opens many connections at once, and the serial output path was the thing they all shared.

Two Changes, Neither Enough Alone

The first change was to let the stack coalesce its output. Instead of writing each packet as it is produced, the output path gathers what the stack has ready and writes it in one call, flushing at 512 KiB. macOS had done this for a while; iOS and tvOS had not.

The second was to give the direct path a larger send buffer: 64 KiB instead of 8 KiB. The direct receive window stays at 4 KiB, with no scaling and no SACK. That is deliberate: the receive window is the one behind the upload collapse in July, and upload was never the problem here.

Neither change is enough on its own. Measured on tvOS, with the changes limited to tvOS and the runs interleaved:

Configuration Download
Original 285–352 Mbps
Output coalescing only 501–660 Mbps (713 with the profiler)
Send buffer only 326–384 Mbps
Both 887–931 Mbps

The all-in runs, with the read-side change described below, came in at 931/923/910 and, interleaved, 908/920/887/884. Upload stayed at 111–120 Mbps, and the extension’s memory footprint stayed flat at 7.5 MiB. Nothing here trades memory for speed.

Every comparison is the same Apple TV on the same wired network, with Chute connected and disconnected. The only thing that changed between the low and high numbers is the code under test.

The Read Side

The same one-at-a-time habit existed in the other direction. The app handed the stack one packet per dispatch onto its queue. It now hands over the whole read in one block. UDP and ICMP still take their per-packet path inside the engine, because their handling is not a straight hand-off; TCP does not.

On the Apple TV this is worth about 35 Mbps on top of the output work — 887–890 before, 908–931 after — and it takes work off a queue that every flow shares.

What Did Not Work

We tried moving the output off the stack’s queue onto a serial queue of its own. It cut about a third of the stack’s work, which looked like a win, and made no measurable difference over three interleaved rounds. The contention was not the whole story; the number of calls was. We kept the measurement and dropped the change.

The Question We Left Open

The coalesced write has a byte cap — 512 KiB — and no packet cap. That is fine on the systems we tested, but the system’s vector write may clamp the number of packets it accepts per call on some OS versions, and Apple does not document it. The tunnel API treats any positive return as success, so a tail that did not fit could be dropped without anyone noticing.

We have not reproduced it, and we have not captured a packet to prove it. For this release we left the behaviour as it is and wrote down the likely fix — cap each batch at 64 packets and let the next call take the rest. It is the kind of thing that deserves a packet capture, not a guess, and it is on the list.

What It Means

The update that ships this week downloads at about three times the old rate on an Apple TV 4K: a direct Speedtest now sits within a few percent of the line, instead of at a third of it. The same values are used on iOS by decision; we have not measured them on an iPhone yet, and we will say so when we have.

The lesson is not “batch everything”. It is that a tunnel has two packet paths, and the one that looks like plumbing can own the profile. The receive path had been optimised; the send path had not, because a single flow never complained.

Thanks.

Chute Devs