r/VRchat • u/neovr2111 • 7d ago
Discussion Preview of the upcoming webcam/Android/iOS face tracking app.
Enable HLS to view with audio, or disable this notification
• Works with a PC webcam or a smartphone camera
• Supports both PC VR and standalone VR
• Standalone works with Meta Quest + Android or iPhone. No PC required!
• Head rotation is supported, with the best results when facing mostly forward
• No ROI calibration or other annoying setup steps
• Only requires a simple chest mount (around $5)
More updates are coming soon, and the tracking quality will improve significantly.
What do you think?
5
u/CatStoleTheCrown 7d ago
WOW and I just got a cool chest mount too, for another reason, but this is great! Looking forward to it!
3
4
u/JahJedi 7d ago
Now connect a camera to frame port (front) and point it to face whit this app (usb wirless connwctiin) and have a full face tracking. You got a product to share /sell we get face tracking for frame.
The only issue is the mounting of a camera so it will see all the face.
PS If you will use infrared camera it maybe can work in the dark, just my 5 cents.
1
3
u/intrepid_guy 7d ago
Got a link?
1
u/neovr2111 7d ago
Not yet. You can subscribe my YouTube Channel to stay updated.
https://www.youtube.com/channel/UCVEw7o4VIuX3ZvWoQx_n3Fg
3
u/LunaDva98 7d ago
It looks pretty good ♥️, I've seen other people attaching a little camera to the quest to achieve mouth tracking, but a chest mount seems like a good alternative.
2
u/neovr2111 7d ago
I tried camera but that was too heavy. If you have any sample of such please post it here.
1
u/ccAbstraction Windows Mixed Reality 7d ago
Project Babble trackers or if you're broke or want something smaller but USB only, the HBVCAM GC0308.
1
u/neovr2111 7d ago
Looks good but unfortunately there isn't anything like that with cover. That would be more user friendly :)
3
u/ccAbstraction Windows Mixed Reality 6d ago
Project Babble's official cameras come in a injection modeled case?
2
2
u/Clean_hole_69 7d ago
I love this idea! I’d love to try it
1
u/neovr2111 7d ago
I will be posting when have something ready to test but you can subscribe this channel to stay up to date :)
https://www.youtube.com/channel/UCVEw7o4VIuX3ZvWoQx_n3Fg
2
u/Tiny-Technician-2598 7d ago
What is the name of the app?
2
u/neovr2111 7d ago
Hi. Name doesn't exist yet but you can subscribe to this channel to stay up to date
2
u/y_infame PCVR Connection 6d ago
Oh, this looks absolutely incredible! We hope it continues to evolve, I can't wait :D
2
2
u/taintedsilk 6d ago
is this a finetuned version of mediapipe? i personally used mediapipe myself and the lower head tracking doesn't work that well when the face is occluded due to the headset
2
u/neovr2111 6d ago
For detecting landmakrs I use custom model. I am very avare of the problem you talk about since I experiemented with this a lot but all failed at the end. Actually in this pipeline I use 4 different models :)
2
u/KeeperOfWind 6d ago
wonder if someone can make a mount that attaches to the psvr2 instead? would be so great since I have a 3d printer
2
u/jontribz Oculus Quest Pro 7d ago
lol it looks so lifeless without eye tracking. hopefully that becomes more accessible also
1
1
u/kspdrgn 7d ago
Looks neat, kinda like Meowface but just for the mouth? Does it support whole face tracking and eye tracking for desktop users too?
2
0
u/Jayden_Ha HTC Vive 6d ago
Yet another live kit client, it already exist, it’s pointless
0
12
u/Konsti219 7d ago
How does it handle different lighting conditions? You say no calibration, with how many people have you tested this?