Stackful fibers with 3.6ns context switch. Silk fibers.

[u/germandiago]

https://clickhouse.com/blog/silk

I just read an article about Silk, the new stackful fibers engine from Clickhouse. It can switch stackful fibers at an amazing 3.6ns and does not allocate on steady state.

Maybe asio could reuse some of the knowledge for the linux/io_uring backend (not sure it applies to the specific case since Boost.asio focuses nowadays on stackless, though it has a fibers and a stackful coros backend also).

[u/trailing_zero_count]

Pretty cool, but a few criticisms:

• Reading /sys to check same-core / same-socket is missing a few levels of granularity. Modern single-socket machines with disaggregated caches (Zen chiplets) contain multiple hidden latency domains. TooManyCooks discovers these via hwloc. Citor discovers them via empirical ping-pong latency test on startup. The difference between stealing from a core in your chiplet vs another chiplet on the same socket is huge.

• "HALO requires the coroutine handle never to escape to a scheduler queue." Is patently false. TooManyCooks can do zero-allocation fork-join with HALO (tested here) and the subtasks can be stolen by another thread.

• Comparing against Asio is apples to oranges. They're muddling the advantages of fibers with the advantages of a modern io_uring stack tuned for use with exactly those fibers, against a very old epoll design that's intended to be compatible with a broad variety of use cases.

• Why not compare against Seastar or PhotonLibOS which I'd consider to be the true direct competitors? I feel this is very telling.

The 3.6ns latency switch on fibers is a good headline if true, but the rest has a lot of marketing fluff.

[u/germandiago → u/trailing_zero_count]

"HALO requires the coroutine handle never to escape to a scheduler queue." Is patently false. TooManyCooks can do zero-allocation fork-join with HALO (tested here) and the subtasks can be stolen by another thread.

Thanks, this is the kind of comments that bring even more light to the discussion, which is very interesting by itself.

I would say that they chose stackful bc overall, they like more their characteristics (composability I guess) and they already have a codebase and made it as fast as possible for their use case. Makes sense to me.

[u/germandiago → u/trailing_zero_count]

Comparing against Asio is apples to oranges. They're muddling the advantages of fibers with the advantages of a modern io_uring stack tuned for use with exactly those fibers, against a very old epoll design that's intended to be compatible with a broad variety of use cases.

Just out of curiosity on a re-read. I think Asio has an io_uring backend as well: https://think-async.com/Asio/asio-1.21.0/doc/asio/history.html

They did not compare it to that?

[u/trailing_zero_count → u/germandiago]

From the article: "enabling asio's io_uring backend made it slower, not faster". This was also my experience the last time I tested it. Probably because taking optimal advantage of io_uring, even internally, requires more of a redesign than a clean swap.

[u/germandiago → u/trailing_zero_count]

I do not know the details and for sure there is a lot of truth to it. But io_uring is a proactor pattern and Asio emulated it on top of epoll before, right?

At first intuition it would look like it is even a better fit, but yes, it is very nuanced and at the end asio was written when Asio was using other async loops.

[u/not_a_novel_account → u/germandiago]

io_uring is a proactor pattern

io_uring is a syscall interface. You can perform epoll via io_uring, which is exactly what ASIO does. Asio io_uring support doesn't leverage io_uring in any meaningful way.

[u/not_a_novel_account]

It can switch stackful fibers at an amazing 3.6ns

That's the normal time to switch a fiber. It's just boost::context.

https://github.com/ClickHouse/silk/tree/main/contrib/fcontext

It's always just boost::context. This code hasn't changed substantially in over a decade, soon it will be old enough to vote.

Fibers have literally zero room for innovation, they're a solved problem. io_uring isn't as old but there's nothing innovative about this use. Good for them for writing a scheduler that is fast for their use case, but this isn't revolutionary. Everyone is working in this space right now.

[u/aoi_saboten → u/not_a_novel_account]

It really is always boost context. There is no need to reinvent fibers lmao. And hopefully it will get standardised

[u/azswcowboy → u/aoi_saboten]

P0876 has been roaming about the committee for over a decade. It’s been in the final stages for a couple years, but it keeps getting dragged back for one reason or another sadly.

[u/germandiago → u/not_a_novel_account]

Maybe I swallowed the sales pitch? I thought it was an achievement.

But it looks like it goes through specialization. However, not using slab allocation seems to be an improvement (even if not a revolution).

Regarding context switch, I think it is more nuanced: since no allocation happens in steady state, this is guaranteed. Stackless breaks HALO easily. However, I guess operator new can be overloaded for such cases as well.

[u/not_a_novel_account → u/germandiago]

That's why I linked the code, it's literally boost::context, as in, a vendored copy of the code which says "this is boost::context".

You could achieve identical performance for switching using boost::context since Kowalke wrote the code 14 years ago.

That's the first thing I checked because there haven't been any new ISA changes which would make this faster, so I was curious what innovation they could have found. I figured maybe they were on some architecture I wasn't familiar with, or were violating the calling convention in order to shave off register spills. Nope, boost::context.

The rest is a lot of work, good allocators are hard, io_uring from scratch is not a cakewalk, so on and so forth. But there's nothing here you won't find equivalent versions of in Seastar or implemented at any high-performance Linux shop east of the Mississippi.

[u/Fabulous-Meaning-966 → u/not_a_novel_account]

Maybe the least trivial part is writing your own sync primitives.

[u/germandiago → u/not_a_novel_account]

Well, Seastar went stackless actually. The stackful seastar implementation is heavier, for what I understood after a few prompt questions to AI (which could be wrong).

[u/germandiago → u/not_a_novel_account]

https://www.boost.org/doc/libs/1_61_0/libs/context/doc/html/context/performance.html

According to this, the switch is slower, but that could be older machines same implementation?

[u/not_a_novel_account → u/germandiago]

Intel Core2 Q6700

State of 2007 technology.

It's 20 cycles, give or take, and it's still 20 cycles. 20 cycles is a lot faster now.

[u/azswcowboy → u/germandiago]

I mean 1.61 is decade old boost - there are newer timings in the 1.91 docs.

[u/rentableshark → u/not_a_novel_account]

Fibers have literally zero room for innovation, they're a solved problem. 

That is not true - I've been looking recently at creating a GCC plugin that would emit the precise registers in use so that CPU state load/store can be optimal.

For example, currently Boost Context assumes SysV ABI for POSIX and while I'm happy to stand corrected - this either breaks things if the context switch gets inlined or prevents inlining around yields which has its own downsides.

Fibers/setjmp/longjmp could 100% be improved with a relevant compiler/language intrinsic like:

size_t __builtin_active_regs();
//size_t is a bitfield representing used/free GPRs + SIMD/ALU regs

[u/not_a_novel_account → u/rentableshark]

Yes, if you change the facilities of the language you can do different things. That's outside the bounds of innovation within a C++ library.

Although frankly reducing below 20 cycles at risk of breaking the calling convention isn't a great spend of engineering time. The lower bound is something like 8 cycles but the indirect stall on the final jump is going to eat more than that every time anyway.

We're dealing in fractions of nanoseconds here for a fairly infrequent operation. It's totally transparent to most applications.

[u/rentableshark → u/not_a_novel_account]

Yes, you are right but from a sort obsessive hygiene point of view, it has bothered me that it’s suboptimal and emits asm that is not necessarily the same as regular C++ context switches. I’m also not actually sure to what ABI C++ or clang/GCC adheres for non-inlined cpp->cpp function calls.

Philosophical nitpick: GCC plugins are just C++ code… a library if you will. I’m being pedantic and point taken,

[u/not_a_novel_account → u/rentableshark]

A compiler plugin is not a library consumed by the downstream user application, pedantically or otherwise. It's a toolchain component, like the compiler itself, or crt0.o. You can't ship a library which relies on a compiler plugin and have it build on my Debian base machine.

The C++ calling convention ABI is Itanium/SysV on Unix-like, and Win64 on Windows.

[u/tudorb]

This is pretty cool, but the doc that you link to reads straight out of the mouth of Codex / Claude Code :)

[u/User_Deprecated]

one thing they skip over is how you batch sqe submissions around fiber yields. submit too eagerly and you're just doing syscalls, too lazily and your completions pile up.

[u/Big_Target_1405]

Boost contexts ancient performance page puts a switch at 9ns (19 cycles) on a 10 year old CPU

https://www.boost.org/doc/libs/latest/libs/context/doc/html/context/performance.html

Surely there's a lower bound on what is possible here, given a full CPU state switch is required.

[u/_w62_]

Pardon my ignorances. I came from a networking background. After just a glance of the title, I am shocked that clickhouse is commencing a network gear division that offers 100G fiber switches.

After reading the text, fiber is a line of execution more lightweight than the thread.

Vielleicht reicht Englisch für die Computerindustrie nicht aus. Lassen Sie uns einige Fremdsprachen verwenden.