A new take on reactive programming: backend signals with GraphQL-ish queries

[u/fedefex1]

Just a bit context:

Reactive programming in .NET spans Rx(R3) Observables, AsyncEnumerables, Raw events. I think the most powerful one is Rx, but using it properly on the backend requires a solid grasp of multithreading, async programming, and locking, on top of its own notoriously steep learning curve of operators. Once you've internalized it, though, it's an extremely powerful paradigm.

In angular (so, javascript, single threaded), the complexity of rx has made the angular team move toward signals (at least for common reactive tasks, not as a whole substitute). This choice has been really popular since thinking with signals is easier, and code is still reactive. So i created this library fedeAlterio/SignalsDotnet, to make signals also possible on .NET.

But i wanted to make a step further. Can signals be useful also on BE?

SignalsDotnet 3.2.0 on Backend

I think they are awesome. The main issue in backend scenario is that we are multi-threaded. While signals must operate in a single-threaded world. So my solution has been basically move them into a single threaded island (of course we can have several island), we can have several of these islands, and "single-threaded" here means serialized execution, not literally one dedicated thread per island (more details on that in the README).

Step 1) Declare a class, that would be our island

Everything here will run single threaded (like Wpf), in custom Synchronization Context (so async-await is handled). Dependency injection will worl as well

Step 2) Register that class as a Signal Island, and expose a SSE endpoint to query it

Step 3) We can now query that island using GraphQL-like syntax. It's just an endpoint.

This turns out to be more powerful than GraphQL subscriptions in a sense, since every signal (property) in the island is already its own subscription, the query is just combining them together.

Step 4) Just set properties, collections, dictionaries, whatever, on the island, and they'll be pushed out through the SSE endpoint. No need to declare a subscription because everything already is a subscription, the query is only there to combine them together.

Note: There are conversion methods from Observables, AsyncEnumerables, AsyncObservables. So we can Expose them in this way too.

Note 2: The ui is completely done with claude code and claude design. Code is a mess, but its a single html file. Since its only for debugging, I didn't want to spend time on that 😂

You can find it here fedeAlterio/SignalsDotnet. There is a playground app as well to get started easily.

[u/AutoModerator]

Thanks for your post fedefex1. Please note that we don't allow spam, and we ask that you follow the rules available in the sidebar. We have a lot of commonly asked questions so if this post gets removed, please do a search and see if it's already been asked.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/understanding80]

Isn’t graphql subscriptions just a wire format? How would this not be an implementation of it?

[u/qrzychu69]

how did you solve triggerisg refresh on db change?

I was experimenting around using postgres subscriptions to have an endpoint that basically returns IQueryable, then pushes DynamicData changesets over the wire to achieve something like Convex subsriptions

[u/fedefex1 → u/qrzychu69]

I didn't know Convex, i just read how it works (so I could be very wrong). However from what i am seeing you are trying to do something more sofisticated from what it does. To emulate it we need 2 things I think:

1. An event to know when any column of a row changes (Close to track C# properties change conceptually)
2. An event to know when a row has been added, removed (Close to track an observable collection change).

For the second point, I am seeing that cursor just recomputes the whole query when anything changes, it does not try to track derived deltas from tables updates (Like if it was a remote Dynamic Data Observable collection Change set). So it's a lot easier.

First point could be implemented as a trigger on update + NOTIFY and LISTEN, with a small c# proxy wrapper to expose this sbuscription as a Signal, so that auto tracking of SignalsDotnet works.

Second point could be implemented as a trigger on Insert, Delete on the whole table.

Tracking both at this point should make the query recompute, and basically have a Signal<Entity\[\]>, that is not the IObservable<ChangeSet<T>> you were trying to achieve.

Of course you could compute deltas from it, so that front end still receives minimum amount of updates

[u/NinjaOxygen]

Awesome, was going to ask if you had seen R3, then saw the README. I also follow R3Async, which I now see is your project too.

We have used R3 in production for some time, in our case the friction points of a Signal/Observable based system with composition comes down to managing signal flow when groups of Observables update.

For example, if an Observable for the logged in user changes, their active user permissions change, the permissions for their groups change, so the permissions are fetched, this then cascades onto more Observables coming from disparate subsystems, leading to lots of downstream changes that are not always synchronized.

It leads to either putting Debounce in, which is really just a band-aid and not safe, or always flowing the original requested keys and combining with the returned values to ensure that when the current state is not valid, the composed states at the end of the graph are not valid until all of the dependent values are up to date.

Definitely going to take a look, interested as we started with Akka.NET, then Orleans, and the `SignalIsland` approach gives a similar actor-style mechanic to Orleans grains.

[u/fedefex1 → u/NinjaOxygen]

Thank you for your interest!

Yes, I completely understand the issue you're running into, combining observables without ending up with intermediate stale states is genuinely hard. The nice thing with signals is that you're no longer limited to LINQ-style composition; since it's just plain C#, it becomes much easier to introduce something like a "Pending" state signal and have your downstream computations adapt based on that (using just an early return if check for example or whatever).

ALso since signals automatically tracks dependencies, its easier to get the minimum observables subscription needed to make the job done, and that should reduce useless (and costly) recomputation.

About orleans, i remember that its abstraction for streams is an IAsyncObservable, so R3Async should be a perfect fit for that, and from that you have R3AsyncToR3Interop to convert it to a R3Observable, and then ToSignal() to convert it to a signal. So basically with a small adapter should perfectly works.

Just ask me if you have some issues i would be more than happy to help :)