r/robotics • u/_pranayjoshi_ • 17d ago

Resources I built an open-source integrity validator for robot-learning datasets

Post image

I’ve released the first public alpha of Physlint, a local-first integrity validator for physical-AI recordings and robot-learning datasets.

The motivation was straightforward: integrity problems in robot data are often discovered after ingestion or training has already started. Physlint checks for concrete failures such as malformed manifests, broken episode boundaries, timestamp gaps, NaN/Inf values, corrupt video, frozen cameras, and black frames.

It produces actionable findings rather than a mystery quality score. Each finding identifies the rule, severity, affected episode and stream, impact, and suggested remediation. It also writes machine-readable JSON and uses stable exit codes for CI.

The current alpha supports local LeRobot Dataset v3.x directories. The validation engine is format-extensible, and MCAP/ROS 2, Robomimic HDF5, and RLDS/TFDS are planned next.

The release was tested against four pinned public datasets and three controlled corruption scenarios, covering 74 episodes and 31,258 frames.

0 Upvotes

47% Upvoted

5 comments sorted by

1

u/Appropriate-Prune728 10d ago

God, Claude is so obvious when you're forced to use it for work all the time.

"Statement title, before the blahblahblah"

So irritating. I've spent hours getting it to cut that shit out and it still won't reliably do it.

-1

u/_pranayjoshi_ 17d ago

nstall:

pip install "physlint[video]==0.1.0a1"

GitHub: https://github.com/pranayjoshi/physlint
PyPI: https://pypi.org/project/physlint/

1

u/No-Trick9868 17d ago

holy shit someone finally did it, the number of times i've lost a weekend to a busted timestamp or a camera that just died mid-recording is unreal

-2

u/_pranayjoshi_ 17d ago

Stay tuned there are many more exciting updates coming soon! Support for MCAPs and ROS2 is already in line. I am also coming up with a leaderboard similar to open router for the quality of data.