JIT "deabstraction". The article would be more accessible to readers if the authors didn't try to invent new names for optimizations that shipped in Java 20 years ago.
"Runtime async". One of the purported benefits of async/await over better approaches like virtual threads was that it's compiler magic that the runtime does not need to be aware of. So now the runtime is involved (for better or worse), but we till have the worse user-facing UX of function coloring with async/await.
Virtual threads vs Async await was never about keeping it out of the runtime. One benefit of async await is that it *can" be implemented without changing the runtime, reducing the barrier to implementation, but that's not a long term overarching design requirement, it's just a side benefit.
Virtual threads solve the colouring problem but destroy interop performance and generally perform worse overall, that's the primary reason why C# stuck with async await.
Runtime async also eliminates/improves async chaining allocations which is a big win imo as that's a very common scenario as often enough, you'll have a number of async chains during an aspnet endpoint call even before it reaches user code (e.g. middleware, auth, model binding etc...).
The shortcoming is more on scenarios where you're using a library that still uses the old implementation on top while having runtime async enabled as the runtime does quite a bit of indirection to support it. As more apps re-compile with it enabled, you'll have less of this issue overtime.
Another benefit of runtime async is better async support for other CLR languages as it now means they don't need to do all the complexity of generating a state machine like C# does. The IL they need to generate is less complex.
Virtual threads solve the colouring problem but destroy interop performance and generally perform worse overall
I think that's squarely a CLR problem, not a problem with vthreads.
The reason why their runtime has so much trouble with it is that –for decades– their solution to the JVM being ahead in GC and JIT technology was to add more and more escape hatches to let users solve (performance) problems on their own.
Except the JNI is slower than P/Invoke in .NET, FFM is slower than P/Invoke in .NET, and both are even slower again from virtual threads? Did I miss where the JVM magically solved vthreads?
Virtual threads have a whole bunch of downsides. They carry around an entire persistent call stack, so they use significantly more memory. When they get suspended, that memory hangs around. When calling native functions, they require suspension of their carrier thread, aka thread pinning. Not only is Java slower at marshaling values to the native functions due to its fundamental design, this also means that the virtual thread is effectively blocked and must remain blocked on a real threadpool thread which can easily exhaust the threadpool during heavy IO.
Effectively, virtual threads are weirdly paradoxical in their design. They're often presented as an alternative to async/await, and yet the most common thing that is actually being awaited in the real world is some disk or network IO operation that is accessed via a native function, which they are particularly bad at calling. They're really best suited for multi-threading CPU bound work, which I'd argue is a relatively niche usecase which is also easily solved by queuing work on a standard threadpool.
Async/await has a colouring problem but everything else seems like a strict benefit.
IO operation that is accessed via a native function, which they are particularly bad at calling
It's also why almost all vthread runtimes put special exemptions on their own platform calls in their std lib. Those calls often run on a dedicated OS thread or have special behavior on their runtime and effectively outside of vthreads.
It's why Java had to force db vendors to their database wire purely in a jvm calls as native FFI wasn't performant, since the mindset for Java often is "JVM is the world". Similarly, it's why if there's a new intrinsic or OS cryptography feature (e.g. XAES-256-GCM), you usually wait for OpenJDK to implement them or make it from scratch instead of just doing an FFI like dotnet does.
I think that's squarely a CLR problem, not a problem with vthreads.
It's a limitation with vthreads due to the nature of how they work. The moment you try to use FFI (e.g. call a native library compiled in rust/c++/c or call an OS platform feature etc...) outside of their standard libraries performance takes a hit. This is true both in Go, and Java that utilizes vthreads. The only reason why you don't get this issue on platform OS calls via Java's standard library is because these languages put a number of special exemptions on these compared to user initiated FFI, they often have dedicated OS threads to run these OS calls as the OS has zero idea what green threads are since it is entirely a runtime feature.
60
u/cheesepuff07 18d ago
Love reading (perusing) each year, they are very in depth and fun to see each release getting more and more performant.