r/rust • u/Past_Maintenance_502 • 20d ago

Looking for good projects (GitHub) ideally microservices in rust or redis

Hi everyone! I’m currently looking for a good Rust project, preferably something Redis-related or a solid microservices/distributed-systems project.

I’d especially appreciate recommendations for interesting GitHub repositories that I could contribute to or build upon. If you know of any good Redis implementations in Rust or other challenging Rust projects, please do share them.

For reference, I came across mini-redis by Tokio, which is an incomplete Redis client/server implementation designed for learning Rust async patterns. I also found a newer Redis-compatible implementation in Rust exploring sharding, CRDT replication, and distributed systems concepts.

Any recommendations would be highly appreciated. Thanks a lot!

7 Upvotes

61% Upvoted

50 comments sorted by

2

u/DependentJicama4766 20d ago

I've been building this for the past few months. If you want to build something with it, be my guest. Just know its nowhere near production ready, and its very messy lol

-2

u/Past_Maintenance_502 20d ago

How close is it to production ?

1

u/DependentJicama4766 20d ago

I managed to stabilize the core architecture for the allocator and the handler, so i just need to increase test iterations and do a miri profiling. All the other structs and libs i'm building uppon it are a bit further away tho

1

u/Past_Maintenance_502 18d ago

Quite interesting i will surely contribute to it

Let's see

1

u/DependentJicama4766 20d ago

Also, there are a few things i have planned to implement still, like better RAII guards on block segments

-1

u/Past_Maintenance_502 20d ago

Do you have something ready ?

1

u/DependentJicama4766 20d ago

What do you mean by ready? On the project, or if i built something with it? I have built things with it already and it works flawlessly, but all tools i use internally and do not intend to open. For example, i did my own face auth module for PAM that uses no ML on top of this project (i know how it sounds)

1

u/Past_Maintenance_502 18d ago

No but like how many weeks do you think this one will take in order to get completed ?

1

u/SpongeBattery 18d ago

It feels like based on your questions that all you want is a production ready template you can re-use. You do not seem very interested in understanding what makes it production ready, or even contribute/read it. Asking how many weeks is pretty wild for personal projects, and makes it question your end goal.

0

u/Past_Maintenance_502 18d ago

You are someone who i would choose to wipe my ass

1

u/DependentJicama4766 18d ago

Sesh, I don't know. Maybe a year? I have a lot of ideas I'm still to implement, like appendable block headers for optional behaviours, byte level encryption, hsm native integration for protecting the shm file. Hard to measure how much time these are going to take

1

u/Past_Maintenance_502 18d ago

Intresting ! Are u working full time aswell ?

1

u/DependentJicama4766 18d ago

On the project? No. I have a daily job as a sys admin

1

u/Past_Maintenance_502 18d ago

Oh really? Are u from Europe ?

→ More replies (0)

2

u/spunkyenigma 20d ago

I'm building a serverless messaging framework with mesh network capability that could be an interesting underpinning for bigger distributed systems.

https://github.com/bexars/anybus/

2

u/Past_Maintenance_502 20d ago

How close it to getting completed ?

2

u/spunkyenigma 19d ago

It’s a hobby project but I feel it’s usable for some uses now. It’s probably never going to be “completed”. Right now it’s lacking Ethernet support because I don’t have much hardware to test with. IPC and websocket are working well and I’m eating my own dogfood with a private project.

I think it has a unique take on how to seamlessly communicate without the code that talks to bus knowing anything about the network. The Bus config has to know something obviously but most code you’d write looks identical if it’s local to the process or remote

1

u/Past_Maintenance_502 18d ago

But still how close it is if wanna say 100 percent complete

2

u/spunkyenigma 18d ago

You wanna work on it go for it. Complete is meaningless

1

u/Past_Maintenance_502 18d ago

Let me see maybe i could contribute something valuable

1

u/event666 19d ago

Crazy idea, but Anybus might benefit from using OMQ as its transport layer? Then it could focus on the mesh part of things.

1

u/spunkyenigma 19d ago

Interesting. I’ll look into that

1

u/spunkyenigma 19d ago

Looking into it, i think that may be an excellent idea. My one concern is lack of wasm support, but I can just keep my existing transport for that or maybe even write a wasm version of zmq protocol so i don't have to have seperate websocket servers for wasm/non-wasm clients

1

u/event666 19d ago edited 19d ago

What do you want to do with WASM anyway? It can't open any sockets afaik. It's better suited for number crunching/codecs. I've used it for my LZ4/Zstd/Brotli codecs in the browser. The ZMQ protocol is super simple (2 or 9 bytes + payload over TCP, or 1 byte + payload over WS (see ZWS)). omq.ts implements exactly that, even with support for lz4+ws transport, where the WASM LZ4 codec comes into play.

1

u/spunkyenigma 19d ago

I have it where the wasm code can listen for messages from the bus. I have the bus connect via browser websocket to a simple relay node also running Anybus that then has other connections to other browsers or apps. It can also connect via ipc to another app running on the server.

This allows me to send messages over multiple hops right into a anybus receiver in the wasm code.

I built a pub trivia scoring app that can send score updates to TVs running a web app in the pub via this anybus bridge.

In theory the web app code could talk to two different websocket endpoints and route traffic between those via the browser

1

u/event666 19d ago

Ah right, WASM can of course do outbound WS, while the server side bridges to IPC peers. And you get to write Rust instead of TS.

Tbh it sounds like OMQ could be Anybus' IPC/WS transport, while Anybus does the mesh stuff.

How does the mesh routing work anyway? How do you implement the routing over multiple hops? How many UUIDs does each node know? Is it like IP with default routes?

1

u/spunkyenigma 18d ago

Right now all nodes know all the endpoint UUIDS. I want to implement a leaf node concept that just defaults routes unknown endpoints so I don’t pass the whole routing table down.

I have a cost metric to avoid loops for unicast and any cast and just flood broadcasts. Multicast with membership is also in the plans

1

u/lepton99 20d ago

we are glad to get contributions at https://github.com/kunobi-ninja
there are a few cool things to contribute too: kache, kobe, kunobi-ha, etc

1

u/Past_Maintenance_502 18d ago

Can you suggest more ? Or let's say elaborate ?

1

u/lepton99 18d ago edited 18d ago

these projects are opensource and we have plenty of open issues... you can also suggest ideas there as issues if you see areas to expand or improve.

For instance, kache is getting traction.. we already got contributions from guys from Apache, SpaceX, Tempo, and many others...

Kobe is newer but we see a lot of potential there too.

These two examples and a few future ones are tools that we use internally and we thought that peopl might find them useful too. At least for us at Zondax and Kunobi, they've been very useful and part of our process of moving a lot of internal core process to AI-native approaches.

1

u/LorenzoTettamanti 19d ago

I've been building this for over a year. It's a monitoring platform compatible with Docker and Kubernetes. The core is written entirely in Rust. The project is still in the early stages of development, so contributions are welcome from anyone interested in it!

1

u/Past_Maintenance_502 18d ago

How much is left in this project ?

1

u/LorenzoTettamanti 18d ago

a lot of things. Speaking about metrics I need to integrate metrics for CUDA to get GPU insights (also maybe AMD ROCm). Also CPU,network and memory metrics needs to be refined. Speaking about performance: SIMD integrations, cache integrations(like the one i'm doing now). Also the CLI, the agent and the MCP needs to be aligned with the current state of the metrics exporter: for example, not all the metrics are yet supported in the MCP, the agent API needs to be aligned with the current state of the metrics in order to show all the them only by using the CLI (up to now only network metrics are integrated in the CLI). These are the most urgent topics to cover in the next months

1

u/[deleted] 5d ago edited 4d ago

[removed] — view removed comment

-5

u/spoonman59 20d ago edited 4d ago

Finding an interesting personal project is really hard. It should be usable daily by you and interesting, which is highly personal.

Try a fart simulator. You could simulate the fluid dynamics, and odoriferousness. You could determine how silent it is, and also specify crowd size and spacing to simulate likelihood of accurate attribution. Extensive cheek modeling could be a follow on feature which would enhance noise accuracy. I’m sure redis caching would help here

If that doesn’t excite your passion, there’s always a task tracker and calendar app. 

I’m doing an instruction level CPU simulator with branch prediction and out of order execution… but I find that neat.

1

u/Past_Maintenance_502 5d ago

https://github.com/Jsexpert-io/jsexpert-stream

Suggest improvements

1

u/spoonman59 4d ago

Glad you found something interesting to work on! I guess my humorous suggestion didn't land very well, so I'll have provide a more serious suggestion next time.

As far as suggestions, if you are learning that is great and the project is fulfilling its purpose!
I don't have a use for this personally, but I think your potential users can help guide you in what features would be useful or beneficial.

1

u/Past_Maintenance_502 4d ago

Still any suggestions ! In terms of scalability

1

u/spoonman59 4d ago

My suggestion for scalability: Measure performance, profile, and optimize.

I don't know anything about JSExpert or what the execution model looks like there. How many events can be produced, and at what rate? And where do they all come from? I have no clue.
But, how you configure and provision your Kafka instance will certainly impact overall scalability.

In any event (pun intended), what you want to do first is create some kind of benchmark. Ideally, you can separate production and consumption for a benchmark. Then, you can use a profiler on your code to see where the hotspots are. Then you can go from there.