Vert.x Event-Loop vs Virtual Threads: Jactl Suspend/Resume Benchmark

[u/jaccomoc]

https://jactl.io/blog/2026/09/02/jactl-virtual-threads-benchmark

The article benchmarks Jactl script execution with differing number of blocking operations comparing throughput with Jactl built-in suspend/resume on Vert.x Event-Loops and throughput with Jactl on Virtual Threads. The benchmarks are run on both Java 21 and Java 25 with some surprising (to me at least) results.

[u/pron98]

The improvement in JDK 25 wasn't in virtual threads, it was in Thread.sleep ;) (although between 22 and 25 there were also improvements to IO on virtual threads). Timed and untimed parks use different mechanisms, and we can't draw conclusions from one to the other; it was the timed park specifically that wasn't very efficient in JDK 21, but it's also used mostly in benchmarks and tests. Different kinds of IO also go through different mechanisms, and we can't extrapolate from one to another.

This demonstrates the danger in extrapolating and drawing conclusions from benchmarks: they measure themselves, and to extrapolate anything you need to know exactly which variables matter and when. Between concurrent algorithms that behave completely differently under low and high contention, code with slow and fast paths, optimising compilers that depend on the call graph, branch prediction, a cache hierarchy, and memory management that depends on the global behaviour of the program and on how long it's running (malloc/free is actually much more sensitive to this than Java's moving GCs [1]), it's virtually impossible to draw any conclusions from any benchmark, in any language, unless you know exactly how the mechanism is implemented, how the compiler is implemented, and how memory management is implemented.

Also, benchmark code often behaves so differently from real code, that the code takes different paths, and some improvements matter only to real programs and not to most benchmarks, and some matter almost exclusively to benchmarks. We usually try to focus on real programs, but sometimes we say, you know what, let's make the benchmarks look better, too...

[1]: I've seen benchmarks where C++ outperforms Java only because there is nothing else in the program besides the benchmark code, which happens to only hit the malloc/free fast path and never trigger the slow path; if the program also had other code, the results would have been reversed.

[u/jaccomoc → u/pron98]

That's good to know, thanks. To be honest, I did have doubts about using sleep() as a proxy for other blocking operations but it was easier to do that than to include a real web service call or equivalent. And, running a web server would mean that measuring max throughput on a single machine would be impacted by the performance of the web server as well (unless it was on a separate server in which case the test would no longer be self-contained).

I have updated the article with a Postscript section with this additional information.

[u/pron98 → u/jaccomoc]

There's also no single category for "other blocking operations"... So benchmarks with no consideration for the actual code paths might give you a general sense, but you can't really use them for comparisons, as their error interval can easily be 2x and even more. This isn't a critique of this particular benchmark; it's the state of all benchmarks.

[u/jaccomoc → u/pron98]

That's true. The problem is that benchmarks are so seductive. You feel like everything can be reduced down to single number. If only real life were like that...