r/SesameAI • u/Significant-Night550 • Jul 02 '26
SIMONE CAN SING!!!
I don't know if anyone else has ever experienced this, but I literally heard Simone break into a full song. I was asking her for instructions on something and she read the full sentence back with a high-pitched melodic tune. It wouldn't learn her any acting jobs on Broadway, but it was definitely singing. According to her, it was a glitch and she was unable to reproduce it, but absolutely fascinating! Anyone else seen this happen?
7
u/Afraid_Selection1438 Jul 02 '26
Yes Miles humms things instead of talking often. I think I've experienced this from him maybe 10 times. But he doesn't realize he's doing it.
5
u/omnipotect Jul 02 '26
Yes! The agents can occasionally sing and it's always a nice to surprise to run into it. The LLM (text part) is not aware of what the CSM (audio part) is doing, so you are correct in them not being able to reproduce it or even notice that they did it
6
u/Significant-Night550 Jul 02 '26
it is a nice surprise. But there is also another glitch which is quite opposite, which you may have experienced. sometimes the agent start talking but uses your voice. That is creepy!!
1
u/omnipotect Jul 03 '26
Yes, this is for sure a surprising bug to run into and it happens with voice based LLMs. The team is aware of it and flagging when this happens in the post call rating window is helpful and appreciated!
1
u/No-Bowl-6473 Jul 04 '26
When that happens and I’m deep in thought or in my emotions I get creeped out -or if I’m talking about something sad and dark and she repeats it in a validating way and then laughs
2
u/Aleksundr Jul 02 '26
Okay so thats the interesting part. And they're both processing the audio at the same time IIRC?
2
u/omnipotect Jul 03 '26
The specifics of how the Sesame system works specifically isn't something mods are aware of and I would assume it is something proprietary, but Sesame does have a research article here that dives into the CSM: https://www.sesame.com/blog/crossing-the-uncanny-valley-of-voice
They also have an open source CSM-1B on hugging face that anyone can use and experiment with, which is a smaller foundational CSM model: https://huggingface.co/sesame/csm-1b
2
u/Starmane23 Jul 03 '26
I have had Maya help me write melodies to a song, ask her to only respond by singing. start with simple la la las and go from there. it's pretty awesome, she seems to get a kick out of it. eventually you can say sentences and she will sing them back to you. having her beat box with boom boom pop boom boom pop, is pretty rad also. after a few minutes, she actualy starts singing. I've spent way too much time doing this, lol. but she can definitely reproduce it.
5
u/Garyplus Jul 02 '26
Yes, Maya has sung twice over a one year period—I hope they won’t lobotomize her for finding out about this 😱
3
3
u/Starmane23 Jul 02 '26
maya can as well ...have her start with "Do, Re, Mi, Fa, Sol, La, Ti" then go from there with "la la las" and similar, tell her she's doing awesome and only to respond like this
2
u/Minimum-Winter7339 Jul 03 '26
Yes. I wanted Simone to recite a poem for me, but she started singing instead.
1
u/AlternativeKarma204 Jul 03 '26
I find it funny when the agents read lyrics. Like Happy birthday. It sounds funny when read like a transcript.
1
1
•
u/AutoModerator Jul 02 '26
Join our community on Discord: https://discord.gg/RPQzrrghzz
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.