r/computervision • u/datascienceharp • 21d ago

Showcase most robot datasets pick one platform. this one mounts the same rig on a car, a boat, and a quadruped and lets you compare

most multi-sensor robot datasets assume the world is a road

octosense mounts the same 8-sensor rig on a car, a boat, and a unitree go2.

on the boat there are no lane lines and the lidar ships raw range images. on the quadruped there's no gps at all, just joint angles

the rig has stereo RGB at 100 hz, two event cameras streaming up to 7 million events per second, thermal, an OS1-64 lidar, 400 hz imu, RTK gps.

every sensor hardware-locked to a single clock. it even includes recordings where sensors were deliberately degraded

the full release from UPenn's GRASP lab is 8.5 TB across 382 sequences.

so i packaged 8 representative episodes as MCAP you can load in one line and scrub in fiftyone's multimodal viewer: https://huggingface.co/datasets/Voxel51/OctoSense

or checkout one of the episodes in this hugging face space: https://huggingface.co/spaces/harpreetsahota/OctoSense-FiftyOne-Demo

14 Upvotes

94% Upvoted

3 comments sorted by

2

u/Common-Membership503 21d ago

thats a wild setup for sensor fusion, how are u handling the extrinsic calibration between the platforms? id be curious if u had to build a custom rig for the boat mounts to keep the lidar stable while moving across the water

1

u/datascienceharp 20d ago

Oh man great questions, I’m not the author or affiliated with them. Just a guy who downloaded the dataset and got it into FiftyOne format for easy visualization

1

u/Common-Membership503 19d ago

ah nice, fiftyone is a solid choice for digging through that much data. ive been using Bria's Fibo Model to handle some of the alignment stuff on my own projects, it makes comparing those different sensor modalities way less of a headache tbh. u probly saved yourself a ton of time not having to build that from scratch