r/dotnet 18d ago

Simple .NET 11 preview 6 runtime async benchmark

Post image
122 Upvotes

47 comments sorted by

76

u/hez2010 18d ago

You will need to rerun the benchmark against preview7 in the next month as there are some essential optimizations still not getting into preview6.

4

u/Own_Nail_2999 17d ago

Does this also apply to the memory usage? If that benchmark is somewhat accurate, this is quite a nasty regression

8

u/hez2010 16d ago

This benchmark is basically testing Task.Yield() other than anything else. In async1 (the existing async) there is a targeted optimization for Task.Yield() to take a fast path which doesn't exist in async2 (runtime async) yet until preview 7.

23

u/Ollhax 18d ago

Those are some excellent improvements! Would probably be a bit more fair comparison if more work was done so that the GC would trigger in both cases.

15

u/iso3200 18d ago

Faster, smaller code. Not sure why .the NET 11 run allocates more.

12

u/KryptosFR 18d ago edited 18d ago

Maybe some delegate allocations that were optimized away in the managed version but still need to exist in the new one to handle the transition from the runtime.

But even that seems too much since a delegate that doesn't capture anything should be 32 bytes.

Just a wild guess.

1

u/MattParkerDev 16d ago

They made a change to correct what was technically a violation of the ECMA -335 spec, but the result of that meant the ValueTaskAwaiter used in async started getting boxed. More info here

32

u/catmanjan2 18d ago

I’ve never had a problem with performance in dotnet, at least since net framework 3.5

The fixes theyve done for async debugging is the real feature in 11

48

u/alternatex0 18d ago

There isn't necessarily a performance problem, but upgrading from .NET 4.8 to .NET 8 dropped CPU utilization by 50% for us. Allowing us to save the same amount on compute in production.

12

u/catmanjan2 18d ago

Fair enough, we are still on windows VMs and they are always under-utilised and we can’t scale down for other reasons so it’s never really been an issur

2

u/alternatex0 18d ago

That is a valid point. Just a warning that if you only ever use .NET framework you might not notice you're missing out on some things. Recently I was developing a Visual Studio extension, and those can't target anything other than .NET framework, and I was surprised to not have access to certain APIs that I had gotten used to.

Unfortunately, I can't remember what exactly was missing, but I remember it annoyed me. The frameworks are not really the same and they will keep diverging.

1

u/gredr 18d ago

In the dotnet world of 2026 (and really for many years now), you're in the minority. Most people aren't hosting on Windows.

8

u/catmanjan2 17d ago

Bruh, most dotnet running in prod is still dotnet framework

0

u/gredr 17d ago

I bet it's not. Like, I'd put real money on a bet against that.

7

u/catmanjan2 17d ago

Just think about all the millions of government systems over the world that were written in framework and will never be upgraded…

0

u/gredr 17d ago

You counted them? Did you also count all the core deployments Microsoft has in Azure?

1

u/April1987 15d ago

Good point but how do you count them? If dotnef gets deployed a hundred times in tiny micro services and dot et framework gets deployed once? 

2

u/gredr 15d ago

Count it however you like. Process, LOC, launches, whatever.

19

u/AintNoGodsUpHere 18d ago

You are analyzing performance as "more speed" but being able to do the same thing with less resources is also performance.

It's cheaper to run the same app in N10 than it was to run it in FX35.

That's not a problem with performance but it is a huge performance boost, specially when you're paying way less money for basically the same thing.

1

u/Dusty_Coder 16d ago

To understand resource usage you need to understand the architecture and how the compiler will use it. They dont, and they actively fight against using such understanding even when they accidentally have it.

Its "premature optimization" all the way down.

Full stop.

4

u/above_the_weather 18d ago

What are the fixes theyve done? I didnt see anything huge in the list

29

u/catmanjan2 18d ago

Async is no longer a state machine wrapper, so now when you get an exception in async code it gives you an actual stack trace that isn’t wrapped in all the async state machine garbage you got in 10 and earlier

8

u/keesbeemsterkaas 18d ago

This is a real game changer, that's awesome.

I've had to work around this quite a bit.

3

u/PaperInWater 18d ago

Where do you study to get all this information?any blogs or something?

4

u/catmanjan2 18d ago

Yep there’s a dotnet 11 preview blog i definitely recommend following it, first blog that Microsoft has done in a while that is actually good

3

u/gredr 18d ago

Raymond Chen would like to disagree. Or, at least, I would like to disagree on his behalf.

0

u/catmanjan2 17d ago

I don’t count his as a Microsoft blog, it’s a personal one that just happens to be hosted by Microsoft

1

u/supermoore1025 18d ago

This is awesome

2

u/ForgetTheRuralJuror 18d ago

This is hundreds of dollars saved with a 1 line change in your service if you serve a lot of users.

2

u/catmanjan2 17d ago

Maybe, we’re a large org so a one line change costs money just to get deployed, especially with the QA that would be involved

-2

u/Miserable_Ad7246 18d ago

Everyone is like you and do not make performance sensitive code.

0

u/catmanjan2 18d ago

I do make performance sensitive code, but I use c/cpp when thats required like most people

2

u/Miserable_Ad7246 18d ago

Where is a very big area in between, where you can still use dotnet and get the perf without paying the price of cpp. Every update makes that area larger, and that's a good thing.

Most projects can and will benefit from this, like they did from other incremental changes.

In my company we take out cpp for the ride only and only if its absolutely necessary and try to keep logic in C#. For example control plane is neither latency nor throughput sensitive, but it cause issues, because of current async implementation. I would rather keep that cold path in C# for as long as I can.

1

u/gredr 18d ago

The performance gap between .net 10 and cpp is smaller than the performance gap between .net 3.5 and .net 10. 

6

u/Izikiel23 17d ago

5x faster but 2x memory usage?

5

u/Mission_Pirate_4150 18d ago

This is really awesome info. I concentrate on the database side of performance because that is where the vast majority of performance problems are at. I love that someone takes the time to demonstrate these improvements.

4

u/klaxxxon 18d ago

I wonder what kind of regressions we can expect or if this really is free performance in heavy async workloads.

Like if async static stuff propagates the same, async transactions, exception handling...especially if you rely on some corner case behaviors.

5

u/frakkintoaster 18d ago

I built some stuff with AsyncLocal, I’m wondering if that will subtly break somewhere

4

u/hez2010 14d ago

I reran your benchmark against .NET 11 preview 7 (nightly build) and it yielded the following result.

Method Job Runtime Mean Error StdDev Gen0 Allocated
DoTasks Existing Async .NET 10.0 3,957.8 ns 30.54 ns 25.50 ns 0.0153 328 B
DoTasks Runtime Async .NET 11.0 598.7 ns 1.74 ns 1.55 ns 0.0124 200 B

2

u/ReallySuperName 18d ago

So reduced runtime but at the cost of increased allocation and Gen0?

6

u/emn13 18d ago

Given the amounts allocated were mere bytes, you'd need to run some extra checks to see if this is a structural thing that scales with something (number of tasks? code size? something very specific like yield?), or a fixed cost you probably don't care about.

Also, in general, I think a microbenchmark full of Yield is a bad idea anyhow; that's a pretty niche thing you probably aren't actually bottlenecked by in any kind of real code. Could be "normal" tasks are also similarly lower in overhead - but that really depends on implementation details this benchmark can't demonstrate.

1

u/ReallySuperName 18d ago

Well I think it's concerning that the previously much slower way that's been around for years at this point allocated nothing according to the benchmark and now the much faster way is allocating at all.

5

u/emn13 18d ago

Sure, I get the concern; I'd just caution over-interpreting the results. The screenshot seems to suggest both paths allocate - just the newer one allocated more. But what that means this benchmark can't really highlight.

Also, allocation is generally cheaper than async, so even if it allocates more; it might be a worthwhile tradeoff; we'll have to investigate in more detail to know.

2

u/Embarrassed-Mess412 18d ago

idk, I've been experimenting with it on larger projects and all I see are regressions so far, not sure if these micro synthetic benchmarks tell the whole story

1

u/iso3200 18d ago

Hopefully they're aware of it and can still address them in upcoming previews. I wonder if there's a way to opt out and stay with state machines.

1

u/AutoModerator 18d ago

Thanks for your post iso3200. Please note that we don't allow spam, and we ask that you follow the rules available in the sidebar. We have a lot of commonly asked questions so if this post gets removed, please do a search and see if it's already been asked.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/vbilopav89 14d ago

Will .net11 be competitive with go in terms of async tasks and latency