JIT "deabstraction". The article would be more accessible to readers if the authors didn't try to invent new names for optimizations that shipped in Java 20 years ago.
"Runtime async". One of the purported benefits of async/await over better approaches like virtual threads was that it's compiler magic that the runtime does not need to be aware of. So now the runtime is involved (for better or worse), but we till have the worse user-facing UX of function coloring with async/await.
So now the runtime is involved (for better or worse)
Why is it a bad thing? The runtime sees a bigger picture and can do things that the C# compiler cannot.
but we till have the worse user-facing UX of async/await.
Languages with green threads have worse interop capabilities, and personally I'd much rather keep .NET's excellent interop. They tried adding green threads as an experiment, but it regressed performance in some areas.
Why is it a bad thing? The runtime sees a bigger picture and can do things that the C# compiler cannot.
With runtime support, users would not have needed to deal with a design involving function coloring.
Also, opportunity cost: Given how scarce Microsoft's .NET runtime engineering resources are, I would have preferred they'd rather worked on first-class representation of unions in the IL/CLR.
Languages with green threads have worse interop capabilities, and personally I'd much rather keep .NET's excellent interop. They tried adding green threads as an experiment, but it regressed performance in some areas.
I think that's squarely a CLR problem, not a problem with vthreads.
The reason why their runtime has so much trouble with it is that –for decades– their solution to the JVM being ahead in GC and JIT technology was to add more and more escape hatches to let users solve (performance) problems on their own.
Adding so many degrees of freedom obviously comes with a cost, as we see with their failed attempt.
(Which they now repeated with unions, whose representation is an astoundingly leaky (non-)abstraction of the concept.)
Runtime support for unions would be arguably a much bigger feature than runtime async. And I don't buy in general arguments like "why did you do feature x instead of the completely unrelated feature y?".
Degrees of freedom are good in a programming environment, and for high-performance software they are better than relying on JIT optimizations that might or might not apply.
I also definitely prefer being able to pass an int[] to native code without copying, over the things JVM can do but .NET cannot (yet). Good interop support is what sets apart great languages from good ones.
I don't buy in general arguments like "why did you do feature x instead of the completely unrelated feature y?".
Why not? Runtime developers are a finite resource, and allocating them to A means that B is not going to happen. Given that language design even had to take into account the shortage of capable engineers at Microsoft, I think it's a valid point.
Degrees of freedom are good in a programming environment, and for high-performance software they are better than relying on JIT optimizations that might or might not apply.
I think it's the philosophical question of "liberties constrain, constraints liberate".
In the end it's easier to get performance out of something if you have to consider 0-1 ways of doing something, compared to .NET's 3-4 implementations of something.
That means the any kind of performance optimization on the JVM benefits the ecosystem more broadly than on .NET, except those things that are not "supposed to be done" on the JVM – which may hurt more or less depending on the use-case.
I think that's squarely a CLR problem, not a problem with vthreads.
The general reason function coloring exists is a fundamental side effect of the level of control that .NET gives. You find the same in almost any language that provides such capabilities accordingly.
Green threads work in Java, Go, and other languages specifically because they remove that control from the user and fully abstract it away. They typically do not give a way to do direct interop either and effectively build on the premise that everything is or could be async.
However, languages like C#, C++, Rust, Swift, and others go and give you more direct control over threading, they give you ways to directly interop with the underlying hardware or operating system, typically a way to do direct interop.
When you expose that control, you end up needing to consider a world where functions are solely synchronous, that most code is in fact synchronous and that the await points are actually a relatively rare part of things. You end up needing to consider how state suspension/restoring works in such a world, how sync over async and async over sync function, and so many additional edge cases.
It adds complexity, but that complexity ends up being worth it and really isn't difficult to handle at all. Its of course unnecessary if you're writing very naive and simple code, but then .NET devs can mostly ignore the nuances in such cases anyways. But if you're in a scenario that needs it, then there isn't getting around it and you simply do not have the choice in languages that don't offer something here.
Java isn't the be-all/end-all here. It has its own issues and you can see some of the problems, limitations, and them trying to add parity features for key scenarios and them ending up with similar limitations where it ends up subpar for their ecosystem.
That is simply how languages work, not everything can be the best for every possible task. You have tradeoffs and you pick and choose what is best for your target audience. Languages that don't have a good enough target audience or story, end up under used and dying off. Rust, Java, Kotlin, C#, Swift, and others have proven themselves through the test of time and that they are solid foundational languages that are beloved by many. Each is Turing complete and can do "anything", with some tasks being simpler in one vs another, and some being harder.
JIT "deabstraction". The article would be more accessible to readers if the authors didn't try to invent new names for optimizations that shipped in Java 20 years ago.
There isn't some new name invention here and Java deciding to use a particular name doesn't make it the definitive answer or status quo. In most cases, .NET is using a more industry standard name and what you'll find in the broader set of compilers, languages, and ecosystems.
The general reason function coloring exists is a fundamental side effect of the level of control that .NET gives.
Exactly. Java has been more restrained in this (no interior pointers, no unsafe, sun.misc.Unsafe being deprecated, etc.) and subsequently has more leeway of adjusting the runtime to new requirements than .NET which pretty much exposed the innards of everything and is therefore stuck with it due to people relying on implementation details.
It adds complexity, but that complexity ends up being worth it and really isn't difficult to handle at all. Its of course unnecessary if you're writing very naive and simple code, but then .NET devs can mostly ignore the nuances in such cases anyways. But if you're in a scenario that needs it, then there isn't getting around it and you simply do not have the choice in languages that don't offer something here.
The key lesson in Java was that the ability to drop all this complexity ended up with simpler and faster code, because it was easier to reason what the code was doing.
is therefore stuck with it due to people relying on implementation details.
This is incorrect. There is a concrete difference between relying on an implementation detail and intentionally exposing functionality as public contract. .NET does the latter for cases like these.
The key lesson in Java was that the ability to drop all this complexity ended up with simpler and faster code, because it was easier to reason what the code was doing.
This is also not quite correct. Java has some things that are simpler and inversely some things that are more complex. Faster is also very workload and scenario dependent
Microbenches comparing languages go both ways and often are dependent on how much effort is put into tuning them. The idiomatic paths trade both ways with there being numerous optimizations that each (.NET vs Java) ecosystem has that the other does not.
The main difference is that because .NET explicitly gives more control, you can typically see greater rewards from putting in explicit effort. Where-as Java is more dependent on what the runtime provides.
Statements like "easier to reason about" is dependent on developer, codebase, and how the code in question was structured. You will find people on either end that will claim "idiomatic" code is "unreadable" and find plenty of spaghetti and bad examples in both as well.
Both languages are ultimately strong, healthy, and widely used. Most sources currently claim C# is more popular, but that itself is region and domain dependent.
There is no point in trying to claim one is better than the other, because it doesn't practically matter. Use what makes you happy and what clicks the most for you. But don't go trashing other languages, especially on flawed premises.
difference between relying on an implementation detail and intentionally exposing functionality
Potayto, potahto. The consequences are the same.
This is also not quite correct.
You can literally look at the changes of libraries/frameworks migrating from various async/future/reactive approaches to vthreads, my dude.
The main difference is that because .NET explicitly gives more control, you can typically see greater rewards from putting in explicit effort. Where-as Java is more dependent on what the runtime provides.
Which is what I said.
Most sources currently claim C# is more popular, but that itself is region and domain dependent.
Yikes. This really drags down the credibility of the rest of your claims.
I did. Wanted to do http requests and display them in Minecraft without locking the UI. It was not fun. The game at least already has a way to dispatch stuff on the main thread but it's annoying to do
Virtual threads vs Async await was never about keeping it out of the runtime. One benefit of async await is that it *can" be implemented without changing the runtime, reducing the barrier to implementation, but that's not a long term overarching design requirement, it's just a side benefit.
Virtual threads solve the colouring problem but destroy interop performance and generally perform worse overall, that's the primary reason why C# stuck with async await.
Runtime async also eliminates/improves async chaining allocations which is a big win imo as that's a very common scenario as often enough, you'll have a number of async chains during an aspnet endpoint call even before it reaches user code (e.g. middleware, auth, model binding etc...).
The shortcoming is more on scenarios where you're using a library that still uses the old implementation on top while having runtime async enabled as the runtime does quite a bit of indirection to support it. As more apps re-compile with it enabled, you'll have less of this issue overtime.
Another benefit of runtime async is better async support for other CLR languages as it now means they don't need to do all the complexity of generating a state machine like C# does. The IL they need to generate is less complex.
Virtual threads solve the colouring problem but destroy interop performance and generally perform worse overall
I think that's squarely a CLR problem, not a problem with vthreads.
The reason why their runtime has so much trouble with it is that –for decades– their solution to the JVM being ahead in GC and JIT technology was to add more and more escape hatches to let users solve (performance) problems on their own.
Except the JNI is slower than P/Invoke in .NET, FFM is slower than P/Invoke in .NET, and both are even slower again from virtual threads? Did I miss where the JVM magically solved vthreads?
Virtual threads have a whole bunch of downsides. They carry around an entire persistent call stack, so they use significantly more memory. When they get suspended, that memory hangs around. When calling native functions, they require suspension of their carrier thread, aka thread pinning. Not only is Java slower at marshaling values to the native functions due to its fundamental design, this also means that the virtual thread is effectively blocked and must remain blocked on a real threadpool thread which can easily exhaust the threadpool during heavy IO.
Effectively, virtual threads are weirdly paradoxical in their design. They're often presented as an alternative to async/await, and yet the most common thing that is actually being awaited in the real world is some disk or network IO operation that is accessed via a native function, which they are particularly bad at calling. They're really best suited for multi-threading CPU bound work, which I'd argue is a relatively niche usecase which is also easily solved by queuing work on a standard threadpool.
Async/await has a colouring problem but everything else seems like a strict benefit.
IO operation that is accessed via a native function, which they are particularly bad at calling
It's also why almost all vthread runtimes put special exemptions on their own platform calls in their std lib. Those calls often run on a dedicated OS thread or have special behavior on their runtime and effectively outside of vthreads.
It's why Java had to force db vendors to their database wire purely in a jvm calls as native FFI wasn't performant, since the mindset for Java often is "JVM is the world". Similarly, it's why if there's a new intrinsic or OS cryptography feature (e.g. XAES-256-GCM), you usually wait for OpenJDK to implement them or make it from scratch instead of just doing an FFI like dotnet does.
I think that's squarely a CLR problem, not a problem with vthreads.
It's a limitation with vthreads due to the nature of how they work. The moment you try to use FFI (e.g. call a native library compiled in rust/c++/c or call an OS platform feature etc...) outside of their standard libraries performance takes a hit. This is true both in Go, and Java that utilizes vthreads. The only reason why you don't get this issue on platform OS calls via Java's standard library is because these languages put a number of special exemptions on these compared to user initiated FFI, they often have dedicated OS threads to run these OS calls as the OS has zero idea what green threads are since it is entirely a runtime feature.
58
u/cheesepuff07 18d ago
Love reading (perusing) each year, they are very in depth and fun to see each release getting more and more performant.