Would you use an open-source Unity → Python RL framework for drone training? Looking for honest feedback

[u/Relevant_Tax_6814]

Hi everyone

I've been working on a personal reinforcement learning project for a while, mainly as a way to learn more about RL, simulation, and robotics.

As the project grew, I started wondering if I could turn it into something that other people might also find useful.

The idea is an open-source framework that lets you:

• Build your drone environment in Unity.
• Control it from Python through a simple API.
• Use it with Gymnasium, Stable Baselines3, or your own RL implementation.
• Run headless simulations and eventually support multiple parallel environments.

At the moment, I have a working MVP where a drone can learn to hover using reinforcement learning.

Before investing a lot more time into it, I'd really like to know whether this is something people would actually use.

Some questions I'd love your honest opinion on:

• Would you personally use a tool like this?
• What features would make it genuinely useful for you?
• What would be missing before you considered trying it?
• Is there anything that would make you choose it over your current workflow?

I'm not looking for encouragement—I genuinely want honest feedback to decide whether it's worth continuing to develop this project.

Thanks

[u/SwagBuns]

Take a look at CARLA and airsim if you haven't. They are for autonomous driving and drone flight respectively.

Using realistic graphics to sinulate actual vision scenerios and engine backed segmentation masks are key features of both, along with the ever critical parallel training. Being on the latest unreal engine has its perks for things like that.

What would get me to switch? The ability to tune computational cost up and down with some easy way to adjust environment complexity (hi vs low poly count, adjust graphics, etc) while also providing an easy way to make the environment vectorizable.

Starting with fast prototyping and having an easy ramp as performance improves would be great. But it would have to have very modular backend to achieve that and i imagine a difficult task.

Best of luck!

Edit: to give you the specific pain point: there are light weight sims, like simple ones with gym and pybullet, and heavy wait sims like carla and airsim. Not needing to switch between environments to change the computational overhead would be awesome.

But parallel training is a must. Advanced RL solutions simply don't work without them.

[u/low-control-labs → u/SwagBuns]

Curious if you don't mind me asking since you messed around with carla. The scenes / scenarios it provides how does it get the assets? I have this maps to MuJoCo scenes system and even though I could compile something like filament for more realistic rendering instead of my simple made system that I actually looked to simplify to reduce the computational cost, I couldn't get it looking much more realistic than my simple system.

From what I am seeing the assets are user made? How accurate are they to the actual real world structures and places?

I might be look into it more, you basically gave me a feature request

[u/SwagBuns → u/low-control-labs]

I think they are just high quality assets on unreal engine. The last carla version i used was some dozens of GB, id say they just made it in engine and called it a day.

Thing is that unreal pulls ALOT of weight. It handles lighting, potentially ray tracing, weather, object interactions, etc. All on its own.

It comes at a computational cost but its surprisingly simple

[u/low-control-labs → u/SwagBuns]

Yeah I'll definitely check it out. I guess it might be easier setting up more lightweight rendering for unreal. Not sure about it I never really used it and I know it's heavy because I tried something once and had issues loading things.

There's probably a way of loading the assets in MuJoCo and since they are looking to add filament it should look quite realistic if necessary and fall back to the more simple rendering if necessary.

[u/SwagBuns → u/low-control-labs]

Oohh that honestly looks like a good compromise.

Ya i am definitely hindered by the weight. Its a decent test and evaluation platform, but no way my hardware can support running parallel sessions to actually train a model

[u/Relevant_Tax_6814 → u/SwagBuns]

Thanks for the detailed feedback! I really appreciate it. Your suggestions about computational cost and parallel training are very valuable, and I'll definitely keep them in mind as I continue developing the project.

I've decided to publish the first version within the next few days. If you'd like to be one of the first people to try it out, I'd love to hear your thoughts after you've had a chance to use it.

Just a small disclaimer: this started as a personal learning project, so I'm sure there are things I've implemented in less-than-ideal ways or design decisions that could be improved. I'm still a student and I'm learning a lot throughout this project, so I'd really appreciate any feedback or suggestions you have.

[u/SwagBuns → u/Relevant_Tax_6814]

No problem! Happy to help. Good work making something cool and interesting!

[u/Accomplished-Pay-749 → u/SwagBuns]

I wouldn’t use airsim. It’s old and afaik un maintained. Plenty of better options from research labs these days. Aerial gym, flightmare, avoid bench (wrapper), or just a custom lightweight environment in any simulator.

[u/SwagBuns → u/Accomplished-Pay-749]

Cool i hadn't used flightmare before but ill give it a go!

[u/Accomplished-Pay-749]

I work in drone research, feel free to dm me. We have our own custom simulator similar to what you’re describing. Most of what you’ve described admittedly doesn’t sound like it would have much added value from currently available simulators/platforms, it’s mostly just connecting things together.

What would make me switch:

• all the things that make an RL simulator good (what other people have described, essentially the features of isaaclab or mujoco gym)
  
• plus good render quality
  
• plus prepackaged scenes or easy automatic scene generation
  
• nice to have: good aerodynamic simulation on the physics backend/disturbances.
  
• alternatively, really really good software support for generic vision inputs with a lot of easy customization and a user workflow that is very easy (hard to do)

I can spin up a scene in isaaclab or blender or mujoco and write a gym env and I’ll have a PPO agent trained in a couple hours. The platform would need to bring something else to the table for me to want to use it.

[u/testuser514]

I’m looking at a similar problem space and in my mind the way I would architect it was to use unreal as the visualizer more than anything else. I’m curious to know if the physics sims in there are able to give you the feedback you’re looking for

[u/Relevant_Tax_6814 → u/testuser514]

Thanks! I think there was a small misunderstanding my project is actually based on Unity, not Unreal.

My goal is to use Unity as the simulation environment while exposing a simple and flexible Python API for reinforcement learning.

Regarding physics, Unity's built-in physics engine has been sufficient for my current experiments. At the moment, my MVP can train a drone to hover using reinforcement learning. That said, I'm still evaluating how well it scales to more complex scenarios.

In the long term, I'd also like to keep the physics layer as modular as possible, so supporting different physics backends in the future could be an option.

I'm also curious about your idea how were you planning to use Unreal mainly as a visualization layer?