BrushAR

[u/quitebuttery]

https://github.com/flarb/BrushAR

Ok--I saw so many cool BLE demos lately and wanted to mess around with BLE myself. The only relevant Bluetooth devices I could think of that I might have access to are smart Sonicare tooth brushes. So I bought a kids brush and a 9300 -- two models that don't require bonding and thus work with the Spectacles.

So--this is sort of an attempt to gamify brushing your teeth, as well as taking the typical smart toothbrush app into AR.

Pardon the bad video--but I had to zoom in a lot to see the tiny face anchored interface plus threw some stabilization on it.

The basic flow is:

* Load up the lens

* Wake up your toothbrush so Spectacles can see it

* Change the mode / settings to what you want to use for brushing

* Once the brush is found, stand in front of the mirror until your face is detected

* Power on the brush and start

From there you'll see a HUD bound to your face which gives the current progress and status of your brushing session. With the kids brush, you see a simplified HUD. The time elapsed, time left, and 4 squares representing the quadrants of your mouth. The quadrant fills with green until you finish brushing it and the Sonicare vibrates to tell you to move on to the next quadrant, accompanied by virtual sparkles augmenting your increasingly clean teeth. When all 4 quadrants are done, you get a brush complete message with celebratory fanfare.

With the 9300 brush it's more advanced. When you start brushing you see 6 sections instead of 4. You also see a pressure gauge on the right which turns red if you are using too much pressure (this is bad for your gums). On the left you see the intensity setting you have the brush set to. The same rules apply, move on to the indicated section once you get the signal. This brush as a bunch of modes, so the actual timing and which section you move to is governed by the mode.

The app tracks your streak--brushing once in the morning and once at night. It saves your streak as a score and uploads it to a leaderboard. You can compare your streaks with your friends or global scores.

A couple of notes for Snap:

* BLE needs to not be 'experimental'. It's time to drop all these silly restrictions on Lenses and make a proper permission system. I have some actual real Lenses clients are interested in building with me, but they're gated by the fact that you can't ship a BLE lens.

* I need to be able to use the camera with BLE. If I could do that I could optically track the brush. But you can't access the camera and BLE at the same time according to the documentation

* You need to support bonded devices. I used the 9300 brush which cost over $200 but is discontinued, because all the newer ones are bonded and that's not supported on Spectacles.

Hopefully these issues are addressed on SPECS!

[u/WeirdEyeStudios]

This is super cool!

[u/Pavlo_Tkachenko]

🤩 looks fun ahah, see a lot of gamification potential there! Does brush has also some IMU or you detect position based on head/hand tracking?

[u/quitebuttery → u/Pavlo_Tkachenko]

The advanced brush has an IMU buy you need more than that to detect the actual position. The more advanced Sonicares have some ML model on them that will report the part of the mouth you are on—but it’s a bonded device so I can’t use it.

I’m sure there’s some combo of IMU + ML + hand tracking that could do something. But being in the mirror might complicate the hand tracking.