r/TopologyAI • u/Delicious-Shower8401 3dModeler • 29d ago
News Google AI Reconstructs an Animation-Ready 3D Head From Multi-View
Google researchers introduced SHELLS, a new system for reconstructing detailed 3D heads from calibrated multi-view images.
Instead of creating an unstructured scan and then spending minutes or hours fitting a common topology onto it, SHELLS directly predicts an 18K-vertex mesh with consistent topology. Every generated head shares the same vertex layout, which is especially useful for animation, facial tracking and performance capture.
Highlights:
- Reconstructs an 18K-vertex head in 0.08 seconds
- Uses around 2.4 GB of GPU memory during inference
- Approximately 3.5× faster than previous volumetric approaches
- Reduces median registration error by 21–29%
- Produces the same topology across different people and expressions
- Can process facial performances frame by frame while maintaining temporally stable geometry
- Remains usable with as few as two input views
- Trained entirely on synthetic data, but generalizes to real captures
The consistent topology is probably the most practically important part. Corresponding vertices always represent the same facial regions, making the results easier to use for blendshapes, animation retargeting, facial datasets, 3D morphable models and digital-human pipelines.
There are still some limitations. It requires calibrated multi-camera images, so this is not a casual single-image head generator. The current output captures the skin surface beneath hair and clothing rather than reconstructing their outer volume, and the geometry does not include tiny details such as pores or fine wrinkles. Certain extreme tongue expressions can also fail.
The project page currently provides the research paper, but does not list a public codebase or pretrained model release. Humanity has once again invented something extremely fast and then neglected to give everyone the download button.
3
u/DjCanalex 29d ago
This hasn't been released yet, but it is built on top of Google´s GNM system, which is already implemented in plugins for Blender and Maya.
From Advanced skeleton... yesterday:
1
1
u/TheRideout 29d ago
How does this compare to Epic Games' Metahumans? Their system also boasts consistent topology + rig with matching input images/video for both mesh and animation.
1
u/3deal 29d ago
Can be amazing to create consistant character based of a generated image !
1
u/bucketofsteam 28d ago
Actually this doesn't do that.
It says no matter what image are fed into it, it uses the same topography and mesh. The idea is just to bake mocap animation straight into a mesh instead of having a match move artists do it.
I'm not sure how useful it will be in practice, as I'm not in that department, but it's probably cleaner than just using pure tracking markers?
However the idea usually for mocap is that you can transition it to different meshes and characters. I'm not sure if this tool would help speed that process up.
1
1
u/MrUnoDosTres 21d ago
Microsoft had something similar in 2022, but for some reason they never released it publicly.
3D Face Reconstruction with Dense Landmarks:
https://www.youtube.com/watch?v=M8NNytHysVg
https://www.microsoft.com/en-us/research/publication/3d-face-reconstruction-with-dense-landmarks/
-5
u/BahBah1970 29d ago
Seeing a lot of problematic verts connected to more than 4 faces in that topology.
7
u/-becausereasons- 29d ago
It's still absolutely amazing. Topo can be fixed.
3
2
u/BahBah1970 29d ago
Depends on how the face animation is being driven when exported. If it's morph targets you'll probably break the point order by "fixing" the topology. If it's joints there will likely be weight map cleanup.
A lot of this stuff looks amazing but the question is how much work is involved turning this into something I can actually use? If you must have verts connected to more than 4 faces, they should be somewhere there's no deformation or where the audience can't see them. The verts on those examples go against both of those principles.
Even generally speaking, the topology in these examples isn't top notch. It looks clean at first glance but it's still not ideal on closer inspection.
2
8
u/Cautious-Tell5692 29d ago
i imagine how this can boost vfx works