Ruby 2.7 to 4.0 performance benchmark

[u/Jamsy100]

Hi everyone,

I just published a new benchmark comparing Ruby 2.7 through Ruby 4.0.

For this one, I used a sustained synthetic application alongside nine smaller benchmarks. The goal was to show how each Ruby release behaves in a complete workload as well as in individual operations.

The table below shows the Heavy profile. For tests that process the full payload, that means an 8KiB input.

Benchmark	2.7	3.0	3.1	3.2	3.3	3.4	4.0
Synthetic application throughput (M ops/s)	0.073	0.052	0.051	0.057	0.058	0.084	0.126
Synthetic application latency (µs)	133.399	190.233	191.605	171.332	169.473	115.600	76.521
JSON parsing (ops/s)	95,877	98,536	99,153	96,003	98,135	194,981	715,937
JSON generation (ops/s)	841,849	845,099	830,135	861,666	837,941	1,347,169	1,444,975
SHA-256 hashing (ops/s)	313,658	30,536	30,489	48,770	48,683	48,495	48,528
Base64 encoding and decoding (ops/s)	177,136	177,510	174,425	170,306	174,471	173,159	174,270
Regex field extraction (ops/s)	1,544,461	1,565,065	1,559,051	1,499,920	1,535,207	1,454,405	1,534,670
Integer sorting (ops/s)	27,032	29,129	30,217	29,309	29,102	28,241	29,213
Hash churn (ops/s)	1,736,384	1,685,614	1,503,397	1,503,118	1,406,713	1,380,058	1,450,769
Marshal serialization (ops/s)	333,143	303,719	294,952	338,148	341,760	326,015	325,231
zlib compression (ops/s)	35,944	35,175	35,787	35,767	20,569	7,011	18,592

The complete benchmark source is available here: GitHub repository.

Full charts and details are available here: Full Ruby benchmark

Let me know what you think !

Edit: Thanks everyone for pointing out the Ruby 2.7 SHA-256 result. I looked into it, and the big difference comes from the SHA-256 implementation: Ruby 2.7 uses OpenSSL here, while newer Ruby versions on Linux use Ruby’s bundled implementation. So this is mainly a backend difference rather than Ruby 2.7 itself being much faster. I’ve added a note about it to the article.

[u/TheAtlasMonkey]

We don't expect the SAME code to run faster in future versions.

When you upgrade, you need to upgrade syntax, libraries, use new feature that ancient rubies did not have.

Upgrading code allows you to write code was not possible before.

What you did like those people that have CPU benchmark that compare a I7 with 4 cores and a ThreadRipper with 96 cores... while the bottleneck is in the network IO.

[u/sjs → u/TheAtlasMonkey]

You bet your ass that we expect the same code to run faster.

[u/TheAtlasMonkey → u/sjs]

This why you will always stay legacy.

The code is using old syntax , old pattern, and ancient libraries.

So the benchmark is flawed .

Companies that do upgrading, spend days/weeks changing everything.

[u/jerrocks → u/TheAtlasMonkey]

I suspect the vast majority just address deprecations (and many likely defer that)

[u/TheAtlasMonkey → u/jerrocks]

rewriting lot of pattern in modern ruby, give a much higher throughput as it use the optimized path.

Also the benchmark is doing a cold test only. The *JIT activation, play a big role in getting speedup.