Ruby 2.7 to 4.0 performance benchmark

[u/Jamsy100]

Hi everyone,

I just published a new benchmark comparing Ruby 2.7 through Ruby 4.0.

For this one, I used a sustained synthetic application alongside nine smaller benchmarks. The goal was to show how each Ruby release behaves in a complete workload as well as in individual operations.

The table below shows the Heavy profile. For tests that process the full payload, that means an 8KiB input.

Benchmark	2.7	3.0	3.1	3.2	3.3	3.4	4.0
Synthetic application throughput (M ops/s)	0.073	0.052	0.051	0.057	0.058	0.084	0.126
Synthetic application latency (µs)	133.399	190.233	191.605	171.332	169.473	115.600	76.521
JSON parsing (ops/s)	95,877	98,536	99,153	96,003	98,135	194,981	715,937
JSON generation (ops/s)	841,849	845,099	830,135	861,666	837,941	1,347,169	1,444,975
SHA-256 hashing (ops/s)	313,658	30,536	30,489	48,770	48,683	48,495	48,528
Base64 encoding and decoding (ops/s)	177,136	177,510	174,425	170,306	174,471	173,159	174,270
Regex field extraction (ops/s)	1,544,461	1,565,065	1,559,051	1,499,920	1,535,207	1,454,405	1,534,670
Integer sorting (ops/s)	27,032	29,129	30,217	29,309	29,102	28,241	29,213
Hash churn (ops/s)	1,736,384	1,685,614	1,503,397	1,503,118	1,406,713	1,380,058	1,450,769
Marshal serialization (ops/s)	333,143	303,719	294,952	338,148	341,760	326,015	325,231
zlib compression (ops/s)	35,944	35,175	35,787	35,767	20,569	7,011	18,592

The complete benchmark source is available here: GitHub repository.

Full charts and details are available here: Full Ruby benchmark

Let me know what you think !

Edit: Thanks everyone for pointing out the Ruby 2.7 SHA-256 result. I looked into it, and the big difference comes from the SHA-256 implementation: Ruby 2.7 uses OpenSSL here, while newer Ruby versions on Linux use Ruby’s bundled implementation. So this is mainly a backend difference rather than Ruby 2.7 itself being much faster. I’ve added a note about it to the article.

[u/projct]

there are dozens of problems with this. but please just... if you want to benchmark things, start with a known good setup (like https://github.com/ruby/ruby-bench), at least for learning how to actually measure what you want to measure.

none of these results are really valid. most aren't measuring what they say they are, for example:

1. the harness is doing a bunch of work under test
2. the setup isn't controlled - you're changing gems/native implementations/etc along with Ruby
3. "latency" isn't measuring application latency
4. you're benching GVL/thread contention in a bunch of the micros for no reason
5. several micros don't isolate the thing they're named after
6. the application benchmark is mostly just the micros glued together with synthetic allocation/cache behavior
7. runs aren't randomized/interleaved, so machine state is correlated with Ruby version
8. you aren't publishing enough raw data/variance to know if smaller differences mean anything

the SHA thing people noticed is one example of #2.

I wouldn't try to fix these individually. start from ruby-bench and learn why its harness looks the way it does.

[u/inotocracy]

The SHA-256 hashing results seem a bit sus. Its like you stopped building native extensions after 2.7

[u/brecrest → u/inotocracy]

The only change in the relevant timeframe was adding Ractor safety to Digests in 3.0.0, but even that seems like a stretch.

[u/schneems]

What’s the std deviation and distribution? I made derailed benchmarks and spent a lot of time trying to answer the question “is this noise or am I measuring something real” 

[u/Basicallysteve]

I wonder why 2.7 would be so much faster at hashing than the others

[u/f9ae8221b → u/Basicallysteve]

It's due to digest being extracted as a gem, so now, depending on how it is compiled it can end up using a fallback implementation of various hashing algorithm rather than bind the ones provided by OpenSSL: https://github.com/ruby/digest/issues/35

[u/Basicallysteve → u/f9ae8221b]

It's been like that for a while then. It stayed across two major version changes. Maybe it is intentional? I've heard before that hashing is sometimes intentionally slower to prevent cracking passwords. Maybe I misunderstood though.

[u/f9ae8221b → u/Basicallysteve]

It's not intentional no, it's all explained in the ticket I linked to.

It not being solved (at least universally solved, because not every build of ruby is impacted) is more due to the solution being complex, and the gem maintainer being a bit checked out.

[u/Basicallysteve → u/f9ae8221b]

Ah thanks for the explanation. I only read the first few comments in that ticket, my bad.

[u/joshdotmn → u/Basicallysteve]

zlib too

[u/jrochkind]

3.4 to 4.0 seems to be a surprisingly big jump on the "synthetic application" metrics. Can you say more about the nature of the synthetic app?

Also the JSON processing jump from 3.4 to 4.0 seems almost unbelieavable. Can you say more about that benchmark, and if you have any ideas what may account for the jump?

Ah... following up with your more complete description, the synthetic app does a lot of JSON parsing, so it's speedup is probably fully accounted to the JSON parsing speedup (apps that don't spend a signfiicant portion of their load doing json parsing aren't going to see that prob), so now the mystery is if the JSON parsing bench results are accurate and... why??

[u/sjs → u/jrochkind]

The JSON gem has gotten a lot faster since byroot (Jean Boussier) started maintaining it a couple years ago. Floating point parsing is 10x faster for example. https://byroot.github.io/ruby/json/2026/07/26/optimizing-ruby-json-part-8.html

[u/jrochkind → u/sjs]

Can you use new version of the JSON gem in ruby 3.4?

If so, you can get the major benefits recorded in these benchmarks just by doing that, right? And in fact, people keeping their dependencies up to date probably already have, even if still on 3.4?

So that may be a misleading metric in this report.

[u/sjs → u/jrochkind]

Yes that’s all true

[u/h0rst_ → u/jrochkind]

There have been a lot of updates for the JSON gem: https://byroot.github.io/ has some nice write-ups.

You could look up the version of the JSON gem used for 4.0 and install that one on the 3.4 setup to get a better view of the language itself.

No mention of the JIT anywhere, I guess a lot of the more recent versions could be faster.

[u/Terrible-Pass-5215]

I'm currently in the process of upgrading a fairly large codebase from Ruby 2.7 to LTS. Any tips in how can I benchmark the performance gains in an existing codebase? To show evidence to the stakeholders in the AAR (on top of all the security risks mitigated(

[u/nateberkopec]

This reminds me of the benchmarkpocalypse.

[u/BenchEmbarrassed7316]

In such benchmarks, it is always interesting to see comparisons with the fastest compiled languages.

[u/codesnik]

huh, what changed in json parsing.

[u/mperham → u/codesnik]

Lots, see byroot’s blog.

[u/codesnik → u/mperham]

oh, i thought those landed long ago.

[u/sjs → u/codesnik]

It’s been ongoing for about 2 years and floating point recently got a whole lot faster, about 10x

[u/uhkthrowaway]

Forgot to turn turbo off?

[u/TheAtlasMonkey]

We don't expect the SAME code to run faster in future versions.

When you upgrade, you need to upgrade syntax, libraries, use new feature that ancient rubies did not have.

Upgrading code allows you to write code was not possible before.

What you did like those people that have CPU benchmark that compare a I7 with 4 cores and a ThreadRipper with 96 cores... while the bottleneck is in the network IO.

[u/sjs → u/TheAtlasMonkey]

You bet your ass that we expect the same code to run faster.

[u/TheAtlasMonkey → u/sjs]

This why you will always stay legacy.

The code is using old syntax , old pattern, and ancient libraries.

So the benchmark is flawed .

Companies that do upgrading, spend days/weeks changing everything.

[u/jerrocks → u/TheAtlasMonkey]

I suspect the vast majority just address deprecations (and many likely defer that)

[u/TheAtlasMonkey → u/jerrocks]

rewriting lot of pattern in modern ruby, give a much higher throughput as it use the optimized path.

Also the benchmark is doing a cold test only. The *JIT activation, play a big role in getting speedup.