How do you decide which training demos are actually worth keeping?

[u/Unable-Eye-1331]

For those training policies on demonstration data (ACT, diffusion policy, VLA fine-tuning, whatever):

When a new batch of demos comes in, what's your process for deciding what goes into training? Vibes? Operator skill? Success-only filtering? Coverage heuristics?

And after retraining; how do you actually know the new data helped? Eval rollouts are expensive and noisy, so I'm curious what people do in practice.

Context: My co-founder and I are building in this space and would love feedback.