Hello everyone.
A few months ago we wrote about doubling TUN throughput. That was a story about the network stack — TCP windows, batched reads, connection-level caching. Today’s story is about something much smaller and, frankly, more embarrassing: a single number, sitting in the Shadowsocks adapter, that was quietly costing us most of the protocol’s performance.
The headline: Shadowsocks download throughput on a local test rig went from 42 MB/s to about 152 MB/s — a 3.6× improvement — and upload from 19 MB/s to about 165 MB/s, an 8.5× jump. And the cipher didn’t get any faster. We didn’t touch the encryption at all.
The Setup
When you relay traffic through Shadowsocks, the engine doesn’t encrypt your entire stream as one blob. It slices the plaintext into chunks, prepends a short length-and-tag header to each, and encrypts them independently. This is the AEAD wire format — it lets the receiver authenticate and decrypt piece by piece, so a dropped chunk doesn’t poison the rest of the stream, and memory use stays bounded no matter how large the transfer.
Chunk size is a trade-off. Smaller chunks mean finer-grained authentication and lower per-chunk latency, but more overhead — more headers, more encrypt calls, more bookkeeping. Larger chunks mean fewer of all of those, but each chunk is a bigger unit of work and memory.
For a long time, the adapter’s chunk size was set to 2 KB.
Why That Number Was Wrong
2 KB sounds reasonable for a conservative default. But think about what happens to a single 128 KB block of data coming down from a server — a normal-sized read from a socket on a fast link. With 2 KB chunks, that one read gets sliced into 64 separate chunks before it ever touches the cipher.
Each of those 64 chunks is, in practice, an exercise in copying bytes around:
- Slice the plaintext range out of the buffer.
- Copy it into a new buffer.
- Encrypt into another buffer.
- Append the header.
- Append the ciphertext to the outgoing stream.
That is several memory allocations and several memcpy calls per chunk, per 128 KB of real data. On a gigabit link the engine was doing this thousands of times a second. And the cost was not in the encryption — AES-GCM and ChaCha20-Poly1305 on modern Apple Silicon run at multiple gigabytes per second. The cost was in the memory allocator. The cipher was sitting idle, waiting for the runtime to hand it the next small buffer.
In other words: we were not CPU-bound on crypto. We were allocator-bound on bookkeeping. That is a performance pathology that doesn’t show up in a flame graph as “encryption is slow” — it shows up as a forest of tiny allocations that never needed to exist.
The Fix
The fix was to make the chunks as large as the protocol safely allows. Shadowsocks AEAD caps each chunk’s payload at a fixed maximum — a 14-bit length field, which means 16,383 bytes. That is the hard ceiling: one byte more and the format’s length assertion trips. So the obvious, correct value is exactly that maximum: ~16 KB per chunk, the largest the wire format will permit.
With the chunk size raised from 2 KB to ~16 KB, that same 128 KB read is now sliced into 8 chunks instead of 64. The encryption count drops by a factor of eight. The per-chunk allocation and copy count drops by a factor of eight. The cipher, previously starved between allocations, now gets a healthy-sized block to chew on every time.
The Numbers
We measured against a reference Shadowsocks server over loopback, averaged across runs:
| Direction | Before | After | Improvement |
|---|---|---|---|
| Download | 42.6 MB/s | ~152 MB/s | 3.6× |
| Upload | 19.3 MB/s | ~165 MB/s | 8.5× |
Relative to a direct, no-proxy connection on the same rig, Shadowsocks overhead dropped from “6 to 16 times slower than raw” down to roughly “1.7 to 1.9 times slower.” That remaining gap is the genuine, irreducible cost of the protocol — the encryption itself, the length headers, the per-chunk authentication tag. There is no free lunch there. But the artificial overhead, the part that came from slicing data too finely, is gone.
The Lesson
The interesting part of this story is not the number — it is the diagnosis.
When a protocol is slow, the instinct is to reach for a faster cipher, or to optimize the hot function in the profiler. Both are reasonable. But neither would have helped here, because the hot function was not “encrypt” — it was “allocate a small buffer, copy bytes into it, throw it away, do it again.” The encryption itself was a rounding error on the cost. The bottleneck was invisible bookkeeping that the profiler attributed to a dozen innocuous-looking memory operations rather than to one obvious offender.
There is a general principle here, and it applies beyond Shadowsocks: the unit of work you hand to an API matters as much as the speed of the API. A cipher that can do 3 GB/s will still crawl if you feed it 2 KB at a time, because the overhead around each call dominates the call itself. The same is true of socket reads, of rule lookups, of any operation with per-call cost. Batch generously, up to whatever ceiling the format or correctness imposes, and let the fast path actually run.
This is also why the change had to be validated carefully. Enlarging the chunk affects how data is framed on the wire, and Shadowsocks correctness depends on the receiver’s chunk reassembly being robust to partial chunks and arbitrary sizes. Before shipping, we ran the full adapter and integration suites across every cipher (aes-256-gcm, chacha20, aes-128-gcm, aes-256-cfb), every obfs variant, SS2022, and SSR — all green. Larger chunks are only safe if the read side handles partial state correctly, and confirming that is the part you cannot skip.
What It Means for You
If you use Shadowsocks, Shadowsocks 2022, or ShadowsocksR in Chute, your connections are now substantially faster for bulk transfers — downloads and uploads alike — with no configuration change on your end. The improvement is most visible on high-bandwidth links where the old per-chunk overhead was the dominant cost; on slow or lossy links the protocol latency dominates and the gain is smaller, but it’s never a regression.
And if you have ever stared at a profiler wondering why a fast cipher was producing slow throughput: look at how big the pieces are that you’re feeding it. Sometimes the bottleneck isn’t the math — it’s all the tiny allocations in between.
Thanks.
Chute Devs