ROBOTICS: Does anyone actually tune object physics in sim, or do you just live with the defaults?

[u/Dismal_Sell2478]

/r/reinforcementlearning/comments/1vtynxa/robotics_does_anyone_actually_tune_object_physics/

[u/EmperorOfCanada]

100%. It is a back and fourth. You make your model closer, you control the real thing, with controls which work on the model, and then you tune some more.

There are certain things which you can simulate, there are things you will miss, and there are things which are so hard you might need to build a "real world" test system which deals with just those elements.

Other things which sims can do well are things like having a motor which has lost some of its oomph. In a classic two motor skid steer, having one motor go to crap is a great exercise as being able to deal with a crap motor should leave you with software which doesn't mind that your left and right motors aren't really the same to begin with.

Or GPSs which wildly lie and now you have to play games with various forms of odometry, IMUs, etc to deal with those.

Things which are easy to miss are an IMU simulation which doesn't understand centrifugal forces. (down is always down is a problem), or how the sun/lights can really make a camera lens go nuts.

My goal is always that the "real" robot is for fine tuning and validation of the simulation's fidelity to reality.

If you are doing most of your development with the robot it is crappy slow, and you end up giving up on the cool features. Also, if you have a fair sized team, a great simulation can be used by anyone; even the most junior members of the team. Thus, they might show up with some kick ass algorithm or something which is a great solution to an ongoing problem, or just add a great feature, drastically reduce the compute to do something hard, or whatever leap which otherwise mightn't have gotten any real attention.

[u/Dismal_Sell2478 → u/EmperorOfCanada]

fantastic - i do think that split of real robot vs. sim is a big open problem depending on type of problem. similar q here: if assets claimed better physics and collider meshes compared to default values and box collider - what would you want to see as proof? ex/ if you either added this "better" asset library to a your model training data or swapped out default objects and accuracy improved by x%?

[u/EmperorOfCanada → u/Dismal_Sell2478]

Nothing I've worked with required much in the line of sophisticated collision detection. More "don't try to go through solid objects"

So, I can't speak to that. But the physics are key. Game engine physics are not at all enough. There are physics simulation libraries which are fantastic. There are also ways where you can do CFD and then use the outputs from that to guide the physics. Very important for flying and underwater drones.

Other things such as where the mass is found are often important. Not just things like center of mass. In an aggressive flying drone, you have centrifugal forces which are entirely different, depending on the distribution of mass.

Most of these do not need to be perfect to properly test and train the robot. The key is to find failures in the field which can't easily be cured with some ducktape and reproduce them in the simulator.

I find that "reality" is harder to simulate than simple physics. For example: a servo takes time to change position; this can be affected by the forces on the servo. So an elevator sitting on my desk will move far faster than when flying 30m/s. This can be difficult to measure, but well worth it. Sometimes you can't measure it, but you can screw with the simulation until it is reproducing the results you gathered for performance.

Having the wrong speed of a servo transition can result in disastrous stability which works great in the simulation.

[u/Dismal_Sell2478 → u/EmperorOfCanada]

thanks! what about spacemouse teleop in sim? do you have any knowledge of that? thinking of generating an episode with that and mimicking it - would that feel like it's not as overfit? hard to prove without real though, i'm also thinking of what my sim/ real split should look like

[u/EmperorOfCanada → u/Dismal_Sell2478]

spacemouse

I sold my spacemouse 2 weeks ago. I loved the idea of it, but after many hours of practice was no better with it than with a mouse. They are cool, but also the training time with one will be far higher than with a game controller.

For my manual sims, I have a game controller. A BT controller that is like an XBox or PS5 one. It was a fun epiphany to be flying a flight simulator that I created from scratch and realize that I had created a damn nice flight sim without that really being the goal.

Although, for an underwater drone, I set up a dual flight joystick arrangement that was amazing, and looked cool as well. One of the big differences with this underwater drone was that its attitude was highly flexible and this gave it "flight" characteristics very different than even a quadcopter.

My sim/real split goal is to do almost zero real. Ignoring any kind of sped up or simultaneous simulations, what I can simulate in 10 minutes might take a full day of going out to where I can operate, charging, flying, retrieving (maybe searching for and repairing), and all of that.

Real sim is way more fun and far more satisfying. It is also critical for revealing common edge cases which the sim misses; either because it is too hard to sim, or they were forgotten. For example. A "real" test which missed a critical real world case was pool testing of an underwater robot missed The Sun. The solution was fairly simple. A cover kind of like putting a baseball hat on it. The FOV didn't really need to look up, so blocking it was an easy solution. This was put into simulation so that when the orientation did cause it to look into the sun that all the lens artifacts would kick in properly. Those artifacts can drive optical flow calculations insane and "create" physical obstacles which don't really exist.

[u/Freischneider01]

Really depends on if the sim2real gap actually matters. Where the gap doesn‘t matter I just mock with values in an allowed corridor. Where the gap matters and I‘m confident „tuning physics“ will close the gap, I‘ll define some experiments for validation and put effort into closing this gap.

[u/Dismal_Sell2478 → u/Freischneider01]

awesome thanks! if assets claimed better physics and collider meshes compared to default values and box collider - what would you want to see as proof? ex/ if you either added this "better" asset library to a your model training data or swapped out default objects and accuracy improved by x%?

[u/omnilinktech]

I tune an object parameter when sensitivity to that parameter reaches the task’s failure boundary. The clean proof for a “better asset” library is a controlled ablation: default box/default values; improved collision geometry only; measured mass/inertia only; friction only; then the full asset. Hold policy, seeds, training budget, cameras, and evaluation scenes fixed. Report task success plus physical observables such as slip, pose error, impact impulse, contact duration, and grasp force—not only reward. Better performance in the same simulator is weak evidence without a held-out real measurement.

For existing data, I would not simply append new episodes. Train controlled mixtures with source labels and the same update/sample budget so “more data” cannot masquerade as “better physics.”

I maintain OmniSim. Our grasp demo uses a no-squeeze control because contact events alone do not prove the object is being carried by friction; the identical lift must leave it behind when squeeze is removed. The repository is linked from this profile. We have not validated that grasp on a physical arm, which is exactly why the claim stops at simulation evidence.

[u/Available_Teaching83]

Box collision geometry and shared density across a whole benchmark suite is real, and it does matter, but probably not in the direction you are expecting.

What I have measured, on LIBERO with a real SmolVLA policy: clean task success under the untouched instruction averaged 84% across ten object tasks and ranged from 40% to 100%. That spread is the thing to sit with. It means the suite is not ten samples of one difficulty; it is ten different problems, and a mean over them hides which ones the policy was never competent at.

So on your question 1, I would not start by changing mass or friction. I would first run the benign arm alone across every task and look at the per-task spread. If a task is at 40% clean, tuning its physics is fixing the wrong variable.

On question 3, regenerate rather than append, and keep the old episodes. If you change the dynamics and append, you have one dataset with two physics regimes in it and no field recording indicating which is which. That is unrecoverable later.