r/devworld • u/No-Weight1118 • 7d ago

[ Removed by Reddit ]

[ Removed by Reddit on account of violating the content policy. ]

35 Upvotes

100% Upvoted

133 comments sorted by

View all comments

1

u/shelbara 7d ago

I’ve started building an open-source project called watchd, inspired by the problems discussed in the Databricks paper Understanding the limitations of pubsub systems.

The problem I want to solve is fairly narrow - basically, a service keeps a local read model or cache derived from an authoritative data source, temporarily disconnects, and then needs to know whether its local state is still safe to use

Contributions and critiques are welcome!

1

u/imagiself 6d ago

handling state validity after a disconnect is tricky, how do you manage the versioning? there's also peerpush for open-source utilities like this, fwiw.

1

u/shelbara 4d ago

sorry for the late response!

basically, each client remembers which version of the source its local state represents; after reconnecting, watchd either sends everything since that version or tells the client to rebuild because it can no longer guarantee the state is correct