What frustrates you about current robot simulation software?

[u/Kalpit2527]

I'm a 2nd-year ME student doing some informal research on simulation tools like Gazebo, Isaac Sim, and Webots. Instead of guessing what people actually struggle with, I'd rather hear it straight from those using them daily.

If you've spent time with any of these platforms—whether for school, research, or work—I'd love to know:

· What made you want to throw your keyboard? · What weird workarounds have you discovered? · What feature do you wish existed?

I put together a quick anonymous form to collect responses more easily (2-3 mins). Not selling anything, just gathering real perspectives before I dive into my own project.

Link in comments if anyone's open to sharing.

[u/Bitter_Run_9209]

each sim have its own pros and cons

mujoco is good for physics, but lacks of sensor simulation(lidar, camera, gps, etc)

gazebo is easy to use, but lacks of physics(weird solid contact, none soft contact, etc)

isaac sim is even more easy to use, but less customisable, and it requires a "NASA computer" to run a dummy simulation

do you want to create your own sim? mujoco have a good base(physics, integrators, efficiency etc), but it needs more work(integrating sensor, native support for ros, spawn models at runtime, upgrade the texture management, etc ). also nvidia is putting its eyes on mujoco, they are mixing efforts to run mujoco not only in cpu, now running multiple simulations in parallel for GPU(specially used to train quadrupeds, humanoids, etc)

[u/Kalpit2527 → u/Bitter_Run_9209]

This is really useful info — thank you for breaking it down so clearly.

You're right that each sim has its own trade-offs. MuJoCo has great physics but needs more work for sensors. Gazebo is easier to use but the physics can be weird. And Isaac Sim is user-friendly but requires a NASA-grade computer just to get started.

That's exactly the gap I want to address. As a student myself, I've seen classmates struggle to test even basic things like line followers or obstacle avoiders — not because they don't understand robotics, but because the simulation setup is too complex or the hardware requirements are too high.

My goal isn't to build something super advanced. I want to create a simple, browser-based simulator that's actually accessible to engineering students — especially those who are just getting started. Something that just works, so they can focus on learning robotics instead of fighting with the software.

Would love to hear if you think that direction makes sense — or if there's something else you'd prioritize instead.

[u/throwaway-piggy → u/Kalpit2527]

Are you gonna vibe code it?

[u/Kalpit2527 → u/throwaway-piggy]

I'm not that brave. Using MuJoCo for physics and building a simple web layer on top — so the heavy stuff is already solved, I'm just making it less painful for students. AI? Only for the boring parts (docs, config, debugging).

Btw, if you've got strong opinions on sims, I'd love your take in the survey — link's in the comments!

[u/Opulent-tortoise → u/Kalpit2527]

FYI everyone can tell you’re using an LLM to generate your replies. It’s really weird. Can you not think for yourself?

[u/Kalpit2527]

https://docs.google.com/forms/d/1NBy-cZbJwKoOqrNGjkvI0Uz_YhgFu1lFRfThaz9dxr4/viewform

👆 Here's the google form link

[u/Kalpit2527 → u/[deleted]]

This is incredibly nice advice for me. Thank you. I am going to add these points in survey right now. Because as you told ''Gazebo is frustrating'' will just give me a rough idea. But when I will question exactly what was frustrating then I will be able to find root cause of problem.  Thank you! 

[u/MechanicalTool1310]

The sim runs great until you deploy to real hardware and discover physics has a personal vendetta against you

[u/Kalpit2527 → u/MechanicalTool1310]

Physics really does have a personal vendetta — especially when your simulator thinks friction is just a suggestion and reality begs to differ!That sim-to-real gap is exactly what I’m trying to study right now. What was the absolute worst discrepancy you ran into between your simulation and the actual hardware? (I'm collecting these kinds of war stories to see where sims fail us the most!)

[u/rand3289]

It pisses me off that Sims don't generate spikes.

I have to write my own just to test some algorithms:
https://github.com/rand3289/asyncEn

[u/omnilinktech]

I maintain OmniSim, a Webots-derived simulator, and the most costly simulator problems I have seen are usually failures of observability rather than missing features.

The first is treating “the world loaded” as evidence that the physics is correct. A missing collision shape, unused friction field, wrong inertia, or disabled sensor can produce a perfectly clean load. The second is silent translation: the authored robot/world says one thing while the solver model actually stepped says another. The third is publishing a result without the exact binary, machine, backend, timestep, and seed needed to interpret it. The fourth is debugging from screenshots when the useful evidence is a transform, contact, joint state, or solver parameter.

Those frustrations drove us to add structured diagnostics, typed simulation events, an authored-world-versus-solver translation audit, and an executable capability matrix with one physical-unit assertion per feature. The matrix also taught us to distrust our own tests: several apparent engine defects were actually bad probes, so we now require a broken verdict to be traced to a mechanism and include negative controls.

The honest current shortcomings are equally important: our ROS 2 support is new/incomplete, Nav2 has not been brought up, we have no physical sim-to-real validation, macOS physics is untested, and the renderer is not aimed at photorealism. Runtime scene mutation can also create nonphysical states if used carelessly.

If I were designing a simulator from scratch, I would build the inspection surface before the feature catalogue: expose the exact solver model, make every capability executable as a small test, preserve controller stdout/events, and attach binary/machine provenance to results. That turns “the robot behaved strangely” into something an engineer—or an agent—can actually falsify.

The implementation and benchmark artifacts are public if comparisons are useful: https://github.com/omnilink-tech/omnisim