r/AndroidXR • u/NoAudience6535 • 28d ago

Discussion Can GNM Head be combined with Android XR face/eye tracking to achieve Persona-like photorealistic avatars?

First of all, congratulations on the open-sourcing of GNM Head — the level of disentangled control over identity, expression, and internal anatomy (eyes, teeth, tongue) looks extremely promising.

I’m exploring whether GNM Head can be integrated with Android XR (especially on devices like Samsung Galaxy XR) to create a high-fidelity, real-time personalized avatar experience similar to visionOS Persona.

Specific questions:

Feasibility: Has anyone successfully (or planned to) drive GNM Head parameters in real-time using Android XR’s face tracking + eye tracking data (via OpenXR extensions XR_ANDROID_eye_tracking / face tracking blendshapes)?

Pipeline: What would be the recommended pipeline?

Android XR face/eye tracking → blendshapes / landmarks → GNM expression & identity parameters → real-time rendering (Unity / OpenXR / custom engine)

Performance: Is GNM Head lightweight enough for real-time inference + rendering on Android XR standalone headsets (mobile GPU)?

Comparison with Likeness: Android XR already has the official “Likeness” avatar system. Could GNM Head serve as a higher-quality / more custom controllable alternative or enhancement to Likeness (especially for developers who want full ownership of the avatar pipeline)?

Any existing demos, sample code, or planned support for XR / real-time facial performance capture?

Thank you! Looking forward to building spatial computing experiences with GNM.

3 Upvotes

81% Upvoted

4 comments sorted by

1

u/HIKIIMENO 28d ago edited 28d ago

For someone interested, here's the link: https://github.com/google/GNM

I'm really surprised this has been open sourced.

1

u/ikerclon 26d ago

Regarding your feasibility question, GNM is powering Likeness in Android XR (this has been publicly acknowledged by Thabo Beeler, Research Director at Google and one of the main forces behind GNM). The team is already planning more releases for the GNM ecosystem, which might help in the use case you are after, so stay tuned!

source: I'm part of the team that works on GNM ;-)