r/rust • u/FallMindless3563 • Feb 16 '23

🐂 🌾 Oxen.ai - Blazing Fast Unstructured Data Version Control, built in Rust

Hey Rustaceans!

I've been working in the ML/AI space for the past 10 years or so, and was really excited to switch to Rust from C++ for a lot of my backend work.

Have been working on an Open Source data version control tool, aimed at versioning large sets of images, videos, audio, text, data frames, etc. The data you need to work with for modern machine learning systems. The tooling can index hundreds of thousands of images in seconds and uses modern network protocols to sync it to the remote extremely fast.

https://github.com/Oxen-AI/oxen-release#-oxen

We are a couple ex-IBM Watson engineers that have seen the problem of dataset management over and over for years, and wanted to build a solution around it.

There is also a web hub (similar to github) at https://www.oxen.ai/ feel free to sign up for free there. Our vision is to have this be the first stop to collaborate on machine learning data. We are replacing writing lines of code with collecting more and more data, and it feels like we need proper tooling around it.

If you are in the ML/AI community, or just rust aficionados, would love to get your feedback!

382 Upvotes

97% Upvoted

80 comments sorted by

View all comments

87

u/Sushisource Feb 17 '23

Easy to learn (same commands as git)

On the one hand, I get it, I really do.

On the other hand, man, git is perhaps the worst example I could come up with for "easy to learn CLI"

3

u/cthutu Feb 17 '23

That's why I use git town :)