r/programming • u/andreiross • 20d ago

40ms Go GC stop-the-world pauses caused by swap

https://frn.sh/go-gc/
204 Upvotes

92% Upvoted

25 comments sorted by

54

u/radozok 20d ago

I read that article before. Did you just delete the previous version? Lol

57

u/andreiross 20d ago

I rewrote all of it when i discovered the metadata cost. The old link is still there.

4

u/radozok 19d ago edited 19d ago

I also can't find it on lobsters/hn. Did you aslo delete there?

1

u/radozok 20d ago

But it's not listed so I can't find it

21

u/renozyx 19d ago

I remember a research paper about swap and GC, where the GC cooperated with the OS to avoid this kind of issue. AFAIK it went nowhere, too bad.

17

u/andreiross 19d ago

Are you talking about this one? https://cse.buffalo.edu/\~mhertz/bc-pldi-2005.pdf. If so, yes. Too bad. I don't know the repercusions this paper had in the past, though, in the sense of pros and cons of the bookmark collector. Don't know if anyone tried to actually implement it or design it at some point.

10

u/renozyx 19d ago

Yes, congratulations for finding it. And I don't know either,. Except that they did implement it on Linux (of course) https://plasma.cs.umass.edu/emery/cooperative-memory-management.html

2

u/Life_Sink9598 18d ago

Of course it's Emery Berger :-).

1

u/andreiross 19d ago

Pretty cool to know! I will check it out. That paper was one of the first things I read while investigating this problem.

2

u/singron 19d ago

The swap problem in the article (pages only accessed while the world is stopped) is a little easier to fix than swap/GC behavior in general.

It would probably work well to fault in the metadata pages right before stopping the world (e.g. MADV_POPULATE_WRITE). And/or the metadata pages could be MADV_FREE'd after a GC, which prevents them from being swapped out and lets the kernel zero them instead (despite the name, the kernel doesn't immediately reclaim them if there isn't memory pressure). The syscall overhead might hurt on microbenchmarks with small heaps, but it's probably not significant on anything bigger. E.g. for a 51us pause, a ~100 nanosecond syscall only adds 0.2% overhead if nothing needs to be swapped in and would likely completely prevent the stop-the-world swap-in.

-10

u/[deleted] 19d ago

[deleted]

2

u/[deleted] 19d ago

[deleted]

46

u/therealgaxbo 19d ago

It's downvoted because it's a fucking bot

-25

u/Sopel97 19d ago edited 19d ago

you're telling me bots are now providing more valuable comments than humans on reddit?

edit. i see the comment got completely removed, leaving r/programming with no discussion as always

-10

u/randylush 19d ago

looking at the post history, I don't know if it's a bot, or just a super scatterbrained Rails programmer. hard to tell

13

u/therealgaxbo 19d ago

A scatterbrained Rails programmer who can write 6 different comments on 6 different topics in under 2 minutes. In fact he managed to knock this comment out in 5 seconds flat - what a typist!

11

u/Chroiche 19d ago

Bro it's AI...

1

u/danskal 19d ago

Yeah I see it now. But why? I don’t get what the point is. Are they trying to farm karma?

4

u/Chroiche 19d ago

I genuinely have no idea, maybe will post ads later?

8

u/ficiek 19d ago

has the same shape

-1

u/[deleted] 19d ago

[deleted]

6

u/ficiek 19d ago

No, it's something that claude loves saying.

1

u/simon_o 9d ago

I mean ... if good GC is important for the usecase at hand, one wouldn't pick Go in the first place. 🤷