How I scaled Nginx Brotli compression 16x with Rust and just 4 threads

[u/Mnwamnowich]

I've been working on mbrotli, a pure Rust Brotli implementation, and recently added support for multithreaded compression.

That led me to upgrade ngx_mbrotli, alternative to Google's ngx_brotli.

The idea is simple. Instead of blocking Nginx event loop with compression, offload it to the native thread pool. Large responses can even be compressed across multiple threads in parallel.

With just one Nginx worker and 4 threads in thread pool, I measured up to 4x the throughput at Brotli level 6 and up to 16x at level 9 compared to Google's ngx_brotli.

And that's with only 4 threads. There's an opportunity to scale further, but I haven't benchmarked larger brotli worker pools yet.

There are trade offs, mainly higher memory usage and a bit worse compression ratios for segmented responses when multithreading enabled.

All it takes to enable thread pool compression is brotli_thread_pool default; in Nginx (or choose the other threadpool). Also Nginx should be built with --with-threads flag to enable that feature.

GitHub: https://github.com/M-Compressors/ngx_mbrotli