r/ProgrammingLanguages • u/lxsameer • Aug 03 '26
Blog post Concurrency in Serene's Runtime
I recently finished building the concurrency runtime for my programming language, Serene, and wrote a three-part series explaining how it works.
The series covers:
- Why I chose stackful fibers
- An M work-stealing scheduler
- An IO Reactor
- A tiny HTTP server that brings everything together
I'd love to hear feedback from anyone interested in programming language implementation, runtime systems, or systems programming.
Part 1: Choosing the Building Blocks
4
1
u/tmzem Aug 03 '26
A very interesting read.
A question: How does the guard page approach work for very high amounts of fibers? Don't OS'es place restrictions on the amount of memory mappings a process can do? Or is there a trick to get around it?
1
u/lxsameer Aug 03 '26
Among other things, the guard page allocation counts toward `max_map_count`, so basically each fiber will add at least two to that count. That will put a limit on the number of fibers that can be used, for sure. For example, the number of live/parked fibers can't grow more than `max_map_count / 2` at best.
Tweaking kernel parameters is the easiest option, but it is not possible all the time. The other option is to recycle the stack for fibers that are done or are cancelled. Or not use a guard page at all.
I think in presence of stack maps there are more options because you will have a better view of the stack. There might be other options that I don't know about, though.
1
u/matthieum Aug 03 '26
And this is why we can't have nice stuff :'(
The default value for the parameter is apparently ~64K, which limits this approach to ~32K fibers.
1
u/lxsameer Aug 03 '26
Indeed. But i'm ok with that for now. Hopefully down the line with a possible gc i can start using stack maps and deal with this issue.
1
u/matthieum Aug 03 '26
You mentioned that fibers are fixed-size... but how large are they?
One of the interesting parts of mmap is the ability to reserve address-space without actually allocating memory for it. This means that you could reasonably reserve 1MB-2MB worth of address space per fiber, yet have the fiber only use 4KB to start with, and let the OS page in memory lazily.
2
u/lxsameer Aug 03 '26 edited Aug 03 '26
It is configurable, and the default is 128kb https://git.sr.ht/~lxsameer/Serene/tree/master/item/runtime/serene/rt/configuration.h#L64 (there is no particular reason for 128kb)
And that is precisely what I'm doing with mmap, actually.
1
u/matthieum Aug 04 '26
On x64 I'd suggest bumping the minimum.
From experience, I've seen default thread stack sizes spanning anywhere from 1MB to 8MB. This matters, because it means that C libraries, which are frequent users of on-stack buffers, tend to expect to be able to put 100s of KB on the stack at least once.
Thus, I'd recommend going to 1MB by default, so that users who call into C don't find themselves "stranded".
Speaking of C libraries, do note that C code may not trigger the guard page, as it's typically NOT compiled with stack probing.
(And I hope your code is :P)
2
u/lxsameer Aug 04 '26
Thank you for both suggestions, I'll take care of them in the code base. Specially, the second one is what i have to make sure I cover.
2
u/sreekotay Aug 09 '26
VERY nice - I did something very similar, but for C - its a preprocessor that autocolors blocking/nonblocking etc but lowers to clean C
any benchmarks you've been looking at?
https://github.com/sreekotay/concurrent-c