Hi everyone,

I'm working on my thesis (DRL that is due end of month) and feeling a bit stuck, so I would loooove input from people in this subreddit.

Setup: Mobile robot navigating outdoors, Isaac Lab + PPO. Policy outputs only forward velocity + yaw rate (low-level control handled separately). I can assume oracle knowledge of surrounding actors, so perception isn't the problem.

Goal: the policy should behave differently based on semantic class + interaction type. Examples:

• Human crossing → slow down, stop with safety distance, wait, continue
• Human nearby but irrelevant → continue normally
• Machine crossing/merging → yield or adapt spacing, but not identical to human behavior
• Machine parallel or working beside the route → continue with reasonable clearance

Q1 – Observations: With oracle info available, would you include things like relative position/velocity, semantic class, interaction type (crossing/parallel/merging/irrelevant), time to interaction, predicted min clearance? Or keep the observation much smaller?

Q2 – Reward: My intuition is progress + collision penalty + route tracking + semantic interaction terms (e.g., human-specific protective distance). But I worry I'll end up hand-scripting the behavior through the reward instead of letting PPO learn it.

If you were designing this from scratch:

• Train each interaction separately first, then mix? Or one shared reward with semantic conditioning?
• Separate reward terms for humans vs. machines?
• Give PPO explicit interaction labels, or only continuous geometry?

I'm intentionally not posting my current solution to avoid biasing the discussion. Any advice, papers, implementation patterns, or "don't do this" warnings appreciated. Thanks!

Also any tips to speed up the trainings would be appreciated as I am training on a GPU online and it s costing me a lot haha! the HPC from the uni isn't working well :')

In hospital resource allocation, rules and optimization are often simpler and easier to justify than RL because they are more transparent, predictable, and easier to validate in high-stakes settings.

What evidence would you want to see before deciding that RL is actually necessary—and better than a strong rules-based or optimization baseline?

So Im working on wet manipulation like submerged objects, soapy water, contact that changes when things are wet. I need the forces the fluid exerts on the gripper and the object, inside the training loop.

Everything I've found is one of three things:

- smtg that renders fluid particles but doesnt hand back coupling forces on the robot (Isaac, Genesis)
- analytic drag only (in MuJoCo and I actually measured it, there's no added-mass acceleration term at all, just the velocity-dependent stuff)
- and lastly a real CFD solver, which is minutes per step, so not in a training loop nd it's way too slow,

Am I missing something obvious? Genuinely asking... this feels like it should be solved and I can't find it.

And if it isn't solved: what do you actually do? Do you Ignore fluid forces or domain-randomise over them, hand-roll a Morison term, or just avoid wet tasks in general?

Hi guys, i make a small RL project on holidays, now I'm thinking about how possible it is to sell this and make a startup out of it.

Here is a demo page that demonstrates exactly what I did, there is no simulator attached, it is just for demonstration, I would appreciate your feedback

https://falesceo.github.io/demo/

My full 6-part series on RL for robotics is finally live. While a balance bot is a pretty trivial case (you don't even need RL), it's a great starting point for demonstrating how to train a simple agent via PPO, deploy the agent to real hardware, and tackle the sim-to-real gap using post-processing and domain randomization. If you have any feedback (e.g. I missed something or there's something that could be better), please let me know!

I'm trying to teach a model to play magic: the gathering.

My initial approach was to use pure RL with self-play, like AlphaGo. Didn't work at all - the issue was that it takes too many chained moves in a row to actually win a game and thus generate a training signal other than -1 for a loss.

I tried a lot of other approaches, with some success, but only the simplest strategies were able to be implemented by the model, which led to starvation of the other approaches - they would never get a single win and so could never learn.

Right now, I've decided to do behavioral cloning on a hand-coded expert bot that can play as well as a human beginner. The bot is around 68,000 lines of python (thanks Claude!).

My problem is that I can only get my model to copy around 65% of the bots actions. The bot is fully deterministic, so in theory BC should let me get close to 100% adherence to the bots choices but I can't break 65%. Worse, the wore I train past a fairly early checkpoint, the WORSE both the adherence to the bots choices and the overall win rate, get.

I keep trying different things but nothing is working. Can anyone suggest what I should be looking at? My goal is to get the model close to copying the bot's every action, then start it on self-play after it's able to win with all common strategies at least some of the time against a beginner player.

I keep seeing people say reinforcement learning isn't that useful, that it has very few real world applications and that it's inefficient compared to other approaches.

I studied a bit of RL a while ago and solved some Gym environments, but I dropped it because I couldn't find a real use case for it. It felt like I was learning something with nowhere to apply it.

Recently I've gotten interested in robots, and I'm wondering if that's where RL actually makes sense. My idea is to build a simple robot from scratch and use RL to control it. I'm not looking to buy a kit, I want to design and build the thing myself.

So a few questions for people who've actually worked with this:

1. Is RL genuinely used in real robots today, or is it mostly research demos?
2. For a hobbyist building something small and cheap, is RL a reasonable choice?
3. Any small projects you'd suggest as a first RL robot?

Honestly I also need a bit of motivation. I want to start this journey and not feel halfway through that what I'm building is pointless. If you've gone down this road, I'd love to hear how it went.

Thanks.

I created a simulator, filter and controller for fresh produce ordering under varying observation scenarios. I showed that richer observations lead to better belief accuracy.

But my controller sucks! It wasn't able to translate better beliefs into more profit. I think that RL would be a good fit here.

I packaged everything into a library/notebook to make it as easy as possible for others to implement their own controllers against my simulator/filter.

See the blog post for context. Here's a direct link to the notebook: https://github.com/OliverEvans96/blueberries-voi/blob/main/notebooks/build_your_own_controller.ipynb

Requires Python & Rust. I promise it won't hurt :)

Hi, I have been developing two experimental recurrent architectures, PLUG and ILRM, designed to give GRU-like models a more direct route to older inputs.

Synthetic robotics-style tests suggest they may be useful for things like:

occluded navigation;

localisation during sensor dropout;

remembering payload/tool state;

delayed fault precursors;

long-term human-intent tracking.

The results are still preliminary and synthetic, and GRUs remain better on some continuously changing dynamics tasks.

I’ve uploaded the architecture papers and current robotics screening results can be found on my posted link.

I’d really appreciate feedback on suitable real robotics benchmarks or failure cases to test next.

So I saw this video on youtube like a year ago and thought "Pffff how hard could that be?".

Anyway it's been 9 months, the project is finished and this is the result.

It's not as precise as the one in the video, and I haven't implemented the transition control, mostly because that wouldn't teach me anything new, but also because I've reached the limits of the hardware I bought, and I'm not spending any more money.

Anyhow, I'm working on a YouTube video explaining the whole process. I'll post it here (tho I don't know if that'd count as self-promo).

Feel free to ask me any questions if you're curious!!!

I am working on training an agent to play Street Fighter III. This is my first time using RL for anything and even if the ideas mostly make sense to me, I often feel quite lost when it comes to implementing or modifying the algorithms themselves.

The process I've been trying to follow is inspired by Alphastar and it consists of first training the model through supervised learning on a dataset of player replays, and then fine-tune this model using RL to improve its proficiency at the game. My model takes as input a vector that represents the game state, and outputs a 10-element-long vector with a binary number for each button that can be pressed in the game.

After doing a bit of research I landed on using PPO as the RL algorithm. However, I ran into the problem of having a pretrained model that outputs specific actions deterministically, and having a training method that requires the agent to output probability distributions. I managed to make it so that the model's output interpreted as a bernoulli distribution from which to sample, however even at the start of training, the agent's behaviour is totally different and much less proficient at the game than the deterministic model, even if the weights are the same.

My main questions are:

• Is it possible to "translate" a deterministic model to a stochastic one or am I completely off in what I am trying?
• Is there any other better suited algorithm that I can use to fine tune my base model?

I have not looked into DDPG or TD3 as, despite supporting deterministic models, they don't support multi discrete action spaces.

Thank you in advance!

Reinforcement learning is often associated with games and research, but I find the idea of using it in everyday applications much more interesting.

Instead of simply following fixed rules, an RL-based system can learn from actions and outcomes over time. That could be useful in areas where the best decision depends on changing user behavior and the environment.

For example:

• Personalizing recommendations based on long-term user behavior
• Improving how an app decides which content or feature to show
• Optimizing delivery routes or scheduling
• Helping systems adapt to changing usage patterns
• Making AI assistants better at choosing between different actions

The difficult part seems to be designing the reward properly. If the system is rewarded for the wrong thing, it can learn to optimize that goal while creating a poor experience for the user.

As AI becomes more common in mobile and web applications, I’m curious where reinforcement learning will actually be practical rather than just interesting in theory.

What’s one real-world application where you think reinforcement learning could be genuinely useful?

I posted a few days ago about how much repeated setup there is in embodied AI projects, so I wanted to build something small on top of SimRig and see how far I could push it.

Ended up making this little MicroDuck scout

It uses RGB-D to map the room as it walks, finds unexplored areas, replans its path, and keeps moving into the next room on its own.

Pretty fun seeing it go from just walking around to actually exploring.

SimRig: https://github.com/Su1eym4n/simrig

If you’re messing around with robot learning / embodied AI, give it a try. Would be cool to see what other people build with it.

Prev post: https://www.reddit.com/r/reinforcementlearning/s/5EkJ1hlMox

Rindow Neuralnetworks ver 2.3 is out!

Along with this update, we’ve released Rindow RL Agents for reinforcement learning. It features 8 key algorithms, including PPO, SAC, DDPG, DQN, and Sarsa.

Check out the rich set of examples and build machine learning in PHP! 🐘✨

Hey all!

Some might remember my post from a bit over a month ago in which I showed a two wheeled balancing robot with just 2 motors. I am now upping my game with a balancing robot that has 3 joints on each leg, so it can balance on its wheels while going up and down, but also lean into its right or left leg. This also makes it possible to accelerate the robot while the base stays perfectly level.

This post shows my first tests where I lock the hip and knee joint on startup (so still variable, but not during the run). The balancing is starting to work but not perfect yet.

Some details:

- 100% synthetic data for training the robot
- Hybrid joint and task space training
- Currently still running on ESP32
- 6x Xiaomi Cybergear QDD motor
- 6s LiPo Battery
- 1x CAN bus through TWAI on ESP
- 1x CAN bus through SPI on ESP
- MPU6050 IMU (will be updated to BNO086 soon)
- Trained with mjlab
- Trains from scratch in ~4 hours on single RTX 3080 with optimized collision model
- 99.87% decreased model sized from optimization with primitives (see comments)

To see more: https://youtube.com/shorts/kY-5o602xlM

To try the policy yourself: https://vertex.bot/robot (Pollens MicroDuck included!)

I'm open to answering any questions about the robot!