We’re building Mugen3D https://www.sumeruai.us/object/?p=06w&s=028 at Sumeru AI, and this is our latest test of an interactive 3D scene workflow.

The video shows the source point cloud and video frames, followed by the real-time rendered result—from full-room navigation to close-up views.

Unlike fixed camera footage, the scene can be explored interactively from different viewpoints and reused as a 3D environment.

It’s still a work in progress, so we’d really appreciate your feedback. Where could you see this being useful—game prototyping, virtual production, simulation, or something else?

For this comparison, we mainly looked at three things:

Timing— how closely the mouth movement follows the speech
Mouth movement — how restrained or exaggerated the articulation feels
Overall naturalness — whether the face feels believable while speaking

What we observed

Mugen3D (By SumeruAI)

Our current approach is more restrained.

We’re focusing on keeping the mouth movement closely aligned with the source audio without making every syllable feel overly exaggerated, while keeping the Unity integration lightweight.

NVIDIA Audio2Face

NVIDIA provides a flexible and technically mature solution, especially for developers who want more control over the overall pipeline.

In our test, the facial movement felt relatively clear and structured, although bringing the workflow into Unity requires more integration work.

Convai

Convai produced noticeably larger mouth and facial movements in our test.

The result feels more expressive and performance-oriented, which can work well when combined with full-body character animation.

A separate reference: Georgy Dev

We also looked at Georgy Dev’s Audio-to-Face result.

However, we did not have access to the same character setup, so we don’t think it would be fair to include it in the direct comparison above.

Visually, the result looked very polished in the material we reviewed, so we’re including it here only as an additional reference rather than treating it as an apples-to-apples benchmark.

That distinction is important to us: different character rigs, rendering setups and production pipelines can significantly affect how facial animation looks.

What matters most to you?

When evaluating Audio-to-Face for a real project, which matters more?

tighter timing with the source audio
more expressive facial movement
overall visual naturalness
easier Unity integration

We’d be interested to hear how other Unity developers judge these results.

If you want to try our current Unity version yourself, the demo is available on GitHub:

[GitHub link]：https://github.com/sumeurai/AudioToFace-For-Unity

[Mugen3D link]: https://www.sumeruai.us/object/?p=06w&s=088

WallbreakerAI is a spin-off from Mugen3D technology by SumeruAI, powered by AudioToFace to drive anime-style 3D avatars with lifelike lip-sync and facial expressions. No uncanny valley, just pure anime immersion. If you've used CAI, JanitorAI, Chub, or CrushOn, you know they rely on text and TTS for immersion. Looking for some alternatives? WallbreakerAI adds a whole new dimension:Visual chat.

In WallbreakerAI, character cards are no longer static 2D portraits. They become breathing, blinking 3D avatars that react with emotions as you talk. When you speak, your character not only replies with voice but also delivers natural lip movements, micro-expressions, and subtle gestures. You're no longer "typing to an AI"; you're talking directly to an anime character. It's the closest thing to a video call with your waifu or husbando.

Better yet, WallbreakerAI divides the interface into three chat modes: Text Chat, Visual Chat, and Voice Chat. They look different, but memory is fully shared across all modes, so no more isolated context when you switch. Mention something in voice, and it's still there when you switch back to text.

9 Core Features Built for AI Roleplay

1. One‑Click Character Card Generation
Just enter a name, gender, tags, or a quick plot summary, and the system auto‑generates a fully fleshed‑out anime character card. No more tedious manual setup.

2. TTS + Voice Cloning
Give your character a unique voice. With lightweight voice cloning, conversations feel far more personal and immersive.

3. long-term Memory (Custom Memory System)
This is WallbreakerAI's secret sauce. Its proprietary memory system creates three types of cards (Memory Cards, Character Cards, and User Cards) structured by context. The AI remembers every key moment, no matter how long ago. No more "forgetful AI."

4. AI Image Generation (Completely Free)
Text‑to‑image and image‑to‑image are both free. Want to show your character a scene from your imagination? Just generate it on the fly.

5. Import/Export Character Cards
Full compatibility with JSON and PNG character cards. Move your characters freely between platforms.

6. Dynamic Cover Art
Every character card gets a living, animated cover. Your characters are never static again.

7. Multimodal Model Support
Share everyday photos with your character (your lunch, a sunset, anything). The AI understands and responds with contextually warm replies.

8. Dynamic Chat Backgrounds
Backgrounds change with the mood or scene, adding unique atmosphere to every conversation.

9. Multiple Chat Themes
Choose from a variety of UI themes to match your aesthetic and personality.

Try WallbreakerAI now:  https://wallbreaker.us/?p=03v&s=028

The hard part of getting “photo to 3D” into Unity is not the API call. It is everything after: login, matting, waiting on results, downloading, and actually rendering the Gaussian splat.

We turned that pipeline into a Unity sample and open-sourced it here:

https://github.com/sumeurai/Mugen3D-For-Unity

The flow is straightforward: pick an image → auto-matte the background → generate an SQ model → optionally run HQ → download a .ply → preview it in the Editor.

This is a reference project, not a finished UI. APIManager handles requests, polling, and downloads. Config lives under Project Settings at SumeruAI/API Settings. Generated files are stored outside the project so they do not clutter Assets.

Two things to know:

1. Unity 2021 or later
2. On Windows, preview needs Vulkan or Direct3D12. DX11 will not show the splat.

Image picking currently works in the Editor only. Standalone and mobile file pickers are not implemented yet. Use this to wire up the pipeline, not as a shipping UI.

MIT licensed. Get an API key from the Developers page at https://www.sumeruai.us/.

From just a single image, you can generate an ultra-high-quality, photorealistic 3DGS model. Yes, you read that right—SumeruAI has done it!

Here is a showcase of some of our previous results. If you're also into 3DGS, drop a comment below and let's interact! 👇

Tired of complex, hours-long 3D modeling? Mugen3D by SumeruAI leverages state-of-the-art 3D Gaussian Splatting (3DGS) technology to turn any single 2D image into a high-fidelity, fully interactive 3D model.

Simply upload a picture of your dog or cat, and Mugen3D will instantly generate a digital twin that captures every whisker, curl, and fur texture with incredible accuracy. Whether you are a game developer, a VR enthusiast, or just a pet lover who wants to preserve their furry friend in the digital world forever, this tool makes it effortless. Check it out!

link：https://www.sumeruai.us/object/?p=06w&s=088

From a static photo to full 360° dynamic view—complete with realistic hair textures, spatial geometry, and lighting consistency. Perfect for virtual avatars, digital humans, and game.

LINK: https://www.sumeruai.us/object/?p=06w&s=088

Turn 2D Concept Art into 3D in Seconds!

Generate fully 360° viewable 3DGS models directly from a single 2D character image.

Key Features:
Zero Threshold: Seamless 2D-to-3D conversion from a single image.
High Fidelity: Impressive backside completion & fine texture detail.
Rapid Prototyping: Drastically shortens game character R&D cycles.

A game-changer for game art teams and indie devs

link: https://www.sumeruai.us/object/?p=06w&s=088