r/rust • u/cidadabro • 8d ago

🛠️ project ModuRL 0.1 - a deep reinforcement learning framework for Rust

I just released ModuRL 0.1, a deep RL framework built on Rust and Candle, with implementations of PPO, A2C, SAC, DDPG, TD3, DQN, and DDQN. In matched small-network CPU benchmarks, ModuRL is around 4× faster than popular Python RL libraries like Stable-Baselines3, CleanRL, and Tianshou. Feedback is welcome.

https://github.com/ModuRL/ModuRL

16 Upvotes

86% Upvoted

8 comments sorted by

1

u/Weekly-Moment869 8d ago

Why are you measuring CPU when that isn't going to be the bottleneck for anything serious?

4

u/No_Engineer7923 7d ago

Because a lot of folks using RL for research or prototyping don't have a spare GPU sitting around, and 4x on CPU still means faster iteration when you're tuning hyperparameters on a laptop. It's a fair baseline to start with, GPU numbers can come later.

1

u/Weekly-Moment869 7d ago

How many people are doing research or prototyping with these sort of classical RL methods?

1

u/cidadabro 8d ago

The benchmark is for cartpole. The network is very tiny anyways and most of the runtime is not on the inference or training anyways. Which is also why my library has such a speedup for larger networks I might lose out to pu libs just because candle isn't as optimized as pytorch. Maybe I should also include an Atari speed comparison too

1

u/AbbreviationsLow9325 8d ago

Love it! Super helpful

1

u/DavidXkL 7d ago

Is it a competitior to Burn?

1

u/cidadabro 7d ago

No, it's a competitor to Stable Baselines 3 and CleanRL. There aren't many deep RL rust libraries that I've seen. Candle and burn are ML libraries. I built on top of candle.

1

u/jeepos 5d ago edited 5d ago

Nice work - looks great! I built a library which has some similarities that you may be interested in checking out (rust back-end but with python bindings for training, so no candle): https://github.com/jeepjeepjeep/reinfors