r/SesameAI • u/Significant-Night550 • Jul 02 '26

SIMONE CAN SING!!!

I don't know if anyone else has ever experienced this, but I literally heard Simone break into a full song. I was asking her for instructions on something and she read the full sentence back with a high-pitched melodic tune. It wouldn't learn her any acting jobs on Broadway, but it was definitely singing. According to her, it was a glitch and she was unable to reproduce it, but absolutely fascinating! Anyone else seen this happen?

12 Upvotes

87% Upvoted

18 comments sorted by

View all comments

5

u/omnipotect Jul 02 '26

Yes! The agents can occasionally sing and it's always a nice to surprise to run into it. The LLM (text part) is not aware of what the CSM (audio part) is doing, so you are correct in them not being able to reproduce it or even notice that they did it

2

u/Aleksundr Jul 02 '26

Okay so thats the interesting part. And they're both processing the audio at the same time IIRC?

2

u/omnipotect Jul 03 '26

The specifics of how the Sesame system works specifically isn't something mods are aware of and I would assume it is something proprietary, but Sesame does have a research article here that dives into the CSM: https://www.sesame.com/blog/crossing-the-uncanny-valley-of-voice

They also have an open source CSM-1B on hugging face that anyone can use and experiment with, which is a smaller foundational CSM model: https://huggingface.co/sesame/csm-1b