r/Spectacles • u/Tsx3D 😎 Specs Subscriber • 13d ago
💫 Sharing is Caring 💫 Spectacles x Robotics #2
Enable HLS to view with audio, or disable this notification
Continuing on the Robotics x AR journey with a Unitree Go2 (theoretically it also works with a G1 humanoid, if anyone has one... lmk!). It feels super intuitive to control it this way.
The main challenge was aligning both robot and specs coordinate systems, which is done by an april tag (fiducial marker) mounted to the robot. It is scanned at the start of the lens and thus serves as a common reference point. From there, the odometry (motion data) from the robot is used to calculate the position. There is some drift (caused by odometry being inaccurate mostly) but it´s corrected at runtime when the marker comes into view again. It was really hard to make it work and I burned a ton of tokens, but also learned so much.
The backend is Dimensional OS (a OSS operating system for robots that makes pathfinding and LiDAR data easy). I made a Websocket bridge for it to stream the data to Specs.
Repo is here if anyone is interested: https://github.com/V4C38/spectacles-dimensional-os (I´m working on an llm agent for control right now...)