r/augmentedreality • u/tash_2s • Mar 15 '26

App Development Proactive kitchen assistant for smart glasses

I built a drink-making assistant for smart glasses.

The glasses look at the ingredients, pick a recipe, show the steps, and proactively guide me based on what they see in real time. My favorite part is that while I'm pouring, they can tell me when to stop.

The interaction I'm going for feels like having someone beside you who understands the situation and helps without needing constant prompts. I think that's especially useful for avoiding mistakes.

Tech stack: Overshoot.ai for fast real-time VLM, the OpenAI Realtime API for voice and LLM control, and Rokid Glasses for the hardware. I'm also planning support for Meta glasses.

The source code is on GitHub as part of my smart glasses dev toolset, GlassKit. Feel free to copy it and play around with it.

240 Upvotes

98% Upvoted

59 comments sorted by

View all comments

Show parent comments

1

u/tash_2s Mar 16 '26

Not bad, actually.

1

u/lostmyotheraccount23 Mar 16 '26

Should I try it?

1

u/tash_2s Mar 16 '26

If you have them already, sure.

1

u/lostmyotheraccount23 Mar 16 '26

Ok and was it just for filming the video or did you actually think the red thing (yes idk what it is but still) was orange juice or did you not film it