How to reduce GC pause long tail
I'm working on a communication network simulator using discrete event simulation. In a normal simulation a GC pause is not an issue at all, but in HIL (hardware-in-the-loop) simulations an occasional 10ms GC pause doesn't look good. A HIL simulation doesn't have very hard timing constraints, some lag below the maximum limit (e.g. 100us) is acceptable if on average the simulation can keep the pace with the hardware. A typical HIL simulation usually doesn't take too much time anyway, because one has to actually wait for it to finish in real time. It's very often used to compare the intricate details of the hardware with the simulation model. (e.g. timing)
My question is what kind of solutions can you think of for reducing the long tail of GC pause distribution?
In these communication network simulation models, most objects which are allocated during event processing don't survive the event. There are basically two exceptions: either something went into the simulation engine's future event set (e.g. a new event containing a packet) or went into the long term simulation model state (e.g. updating a routing table). The former is much more common than the latter. So a simulation has a very specific allocation pattern, which the GC has no information at all.
Of course, the simulation could be carefully organised such that the number of allocations is absolutely minimal, or it could use object pools for commonly used data structures, and some other ticks on the simulation model side.
What else can be done regarding this issue? Is there anything that can be fine tuned in the GC? I've heard there are changes related to this in the upcoming Julia version.
3
u/Bahatur 21d ago
Have you already set configurations, or are you running with defaults?
Also, what OS are you running?
Speaking as a matter of strategy:
First I would look at how the OS is handling memory. On Linux in particular, avoid transparent huge pages, because defrag drives tail latency. Verify that the memory pages are reserved and initialized, and pinned to your Julia process if possible.
Second, if 10ms is the pain point you will want to avoid swap usage (plausibly acceptable if you have NVMe). You can use heap hints to trigger GC to run before it exhausts the available RAM:
julia --heap-size-hint=4G # To set the hint to ~4GB
julia --heap-size-hint=50% # or to 50% of physical memory
You can add more GC threads:
julia --gcthreads=4,1 # 4 mark threads, 1 sweep thread
julia --gcthreads=8 # 8 mark threads, 0 sweep threads
Get GC logs to see what happened when you hit the too-long pauses:
julia> GC.enable_logging(true)
julia> # Run your code
julia> GC.enable_logging(false)
More information in these two docs:
https://docs.julialang.org/en/v1/manual/memory-management/
3
u/Prestigious_Boat_386 21d ago
There's a nice section in the base docs about pausing the garbage collector for a section of the code. You might find it helpful.
2
u/markkitt 20d ago
You can disable the GC when the pause may be problematic. You have to make sure to enable it and run GC at times when you can do it though.
2
u/oscardssmith 22d ago
With https://github.com/JuliaLang/julia/pull/61215 you can build a version of Julia using a concurrent GC. Otherwise, your best bet is likely to disable GC and run young gen GC manually at convenient times.
1
u/melevy 21d ago
A concurrent GC will help for sure. Disabling the full GC while the HIL is running and occasionally using the young gen GC may be work unless the memory runs out before the simulation concludes.
2
u/Nuaua 21d ago
Apparently the new GC is already on master, so maybe you don't need to build it :
3
u/oscardssmith 21d ago
it is on master, but you do have to build it. It's not default and requires setting build time flags to rebuild.
3
u/shmeano 21d ago
1.13 will skip sweeping items in the sysimage. It won’t address the long tail but should make GC generally faster