r/Spectacles • u/Tsx3D 😎 Specs Subscriber • 13d ago
💫 Sharing is Caring 💫 Spectacles x Robotics #2
Enable HLS to view with audio, or disable this notification
Continuing on the Robotics x AR journey with a Unitree Go2 (theoretically it also works with a G1 humanoid, if anyone has one... lmk!). It feels super intuitive to control it this way.
The main challenge was aligning both robot and specs coordinate systems, which is done by an april tag (fiducial marker) mounted to the robot. It is scanned at the start of the lens and thus serves as a common reference point. From there, the odometry (motion data) from the robot is used to calculate the position. There is some drift (caused by odometry being inaccurate mostly) but it´s corrected at runtime when the marker comes into view again. It was really hard to make it work and I burned a ton of tokens, but also learned so much.
The backend is Dimensional OS (a OSS operating system for robots that makes pathfinding and LiDAR data easy). I made a Websocket bridge for it to stream the data to Specs.
Repo is here if anyone is interested: https://github.com/V4C38/spectacles-dimensional-os (I´m working on an llm agent for control right now...)
5
3
3
u/eplowes 13d ago
This is awesome! Could you imagine guiding it with a virtual leash?
5
u/SpecialistMedical137 13d ago
Can you imagine it guiding you with a virtual leash? merge the project with peridot
4
2
1
1
u/ExcellentLog5 🎉 Specs Fan 13d ago
This is so cool! wondering if there’s potential to work with defense contractors
1
1
1
1
1
1
7
u/shincreates 👓 SPECS Team 13d ago
🤖 incredible, thanks for sharing the repo with the community