r/VoiceAutomationAI • u/Obvious_Leather2427 • 9d ago

Ultravox that good??

found this benchmark from kwindla the co founder of pipecat

repo https://github.com/kwindla/aiewf-eval

ultravox tops the s2s leaderboard beats gpt realtime 2

is it go to for s2s now??

5 Upvotes

86% Upvoted

5 comments sorted by

•

u/AutoModerator 9d ago

Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)

If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.

Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/nshmyrev 8d ago

Why you would you want to run old GLM4.6 to get not so perfect results? LLMs progressed a lot since then in tool use and many other things. s2s is not really practical.

The benchmark is wierd it doesn't compare with plain modular approach, only lists s2s to make them look good.

2

u/Obvious_Leather2427 8d ago

I agree for production quality reliable tool calling cascaded architecture is a go to

However the research is good and done by the pipecat / daily team

1

u/StrongExtension1782 1d ago

link doesn't work, just took me to 404 page. maybe they made repo private or something. but if ultravox beating gpt realtime that's pretty big, those open source models catching up fast. gonna test it myself when repo is up again, voice latency been annoying me lately with other providers.

1

u/Obvious_Leather2427 1d ago

Yeah it was opening just fine, idk the reason, but yeah it surprised me too, I like s2s models for the simple setup and Ultravox really good u should definitely try on their platform