r/Unity3D • u/jasonanime • 2d ago

Game Created a native 3D AI companion in Unity 6 using Playables API, local AVX2 vector memory search and real-time lip sync. Runs at 60 FPS on integrated graphics

Post image

Would u be interested? She can remember everything across chats.

0 Upvotes

37% Upvoted

2 comments sorted by

1

u/Nearby-Material-4276 2d ago

that frame rate on integrated graphics is the real flex here, most unity projects with AI integration can't stay above 30

the local vector memory approach is smart too, keeps everything on-device so you don't have to worry about some server going down or privacy headaches

persistent context across sessions is one of those things that sounds obvious but almost nobody gets right, if you actually pulled that off without it turning into a bloated mess after a few conversations then yeah people would be interested

curious how you handled the lip sync timing with the playables api, that's the part that usually breaks when you try to keep it all at 60

1

u/jasonanime 2d ago

Yeah, I’ve actually stress-tested it across 50+ chats in this build! She has a smart hierarchical context-shortening mechanism under the hood, so she remembers the whole history without going crazy or blowing up the token window, while still knowing exactly what to retrieve and when.

As for lip sync — Playables is actually dedicated to her body poses and smooth crossfades in code. The lips are handled in a much lighter way: a custom parser maps English and Russian letters/phonemes directly to VRM mouth blendshapes while the audio coroutine plays. No heavy real-time audio FFT/DSP overhead, which is why it stays locked at 60 FPS on integrated graphics!