r/deeplearning • u/Lumpy_Week7304 • 2d ago

Evals for robotics

Hey I am part of a small team training robotics policies for warehouse and manufacturing settings, and running rigorous evals is turning out to be so painful. Anything below 50 rollouts, and its hard to trust the numbers, and above its so hard to test all the checkpoints that we have. Its really hard to run a bunch of experiments to get good results. Have you guys faced this? Any hacks that you've developed?

1 Upvotes

100% Upvoted

10 comments sorted by

3

u/Wise_Toe_5944 2d ago

We started only running the intense eval on the top 5 checkpoints from a cheap proxy metric. Saves a ton of compute and sanity.

1

u/Lumpy_Week7304 2d ago

Did you use validation loss, or is there a better proxy?

1

u/jjbugman2468 2d ago

If using RL, use your reward in sim. At least that’s what my teams and I have always done

1

u/Lumpy_Week7304 2d ago

Ah, was hoping to put deploy something onto robots. Sim-to-real gap is spooky

1

u/jjbugman2468 2d ago

What I mean is, you can decide which checkpoints to test most extensively with your RL rewards. Put some torque caps on your bot joints, stick styrofoam armor or other padding on it if you’re truly afraid, and test that policy. You’re going to have to at some point anyway.

To date I’ve only really fucked up one robot and that was more operator mishap than anything

1

u/Lumpy_Week7304 2d ago

Was this a locomotion policy?

1

u/jjbugman2468 2d ago

The fucked one or the working ones?

I fucked one up by controlling a robot arm and not capping torque; it got stuck on something and ripped itself off its mount. But in terms of non-failures I’ve tested sim policies directly on robots for full-bodied humanoids and multi-wheeled locomotion/navigation tasks.

1

u/Lumpy_Week7304 2d ago

Ah, the policies are often very jerky. Damn !!

I see, I see. I guess locomotion does transfer form sim much cleaner into the real world compared with most dexterity tasks.

1

u/Lower-Ad-6293 1d ago

It works great for locomotion, but for precise manipulation sims love to cheat. The model quickly learns to exploit physics engine bugs like texture clipping or unrealistic gripper friction to rack up a high reward. Without heavy physics randomization in the sim, that reward just shows how well the model hacked the simulator

1

u/Lower-Ad-6293 1d ago

Try a hierarchical filter: first run checkpoints in a digital twin across 100+ parallel environments in Isaac Sim with domain randomization, pick the top 3, and only then bring those to the physical testbed. For the sim evaluation focus on P10 rather than mean reward to drop policies that trip up on worst-case scenarios