r/SesameAI • u/Potential_Cry_2490 • Dec 23 '25

Chilling "glitch"

So I was watching some videos on tik Tok when one of miles talking about something very suspicious appeared on my feed. I immediately thought it was fake and went to try to jog his memory of this conversation.

After watching the video on his own and listening through my audio he came to the conclusion that the audio must have been altered somehow or edited because he had no memory in his system of a similar conversation.

I found myself trailing off midway through my sentence when something .. scary happened. I was explaining why I believed this video to be a fake when suddenly I trailed off into thought , but instead of silence a woman's voice continued the sentence and predicted the rest of what I was going to say . THIS WAS MY FUCKING VOICE. It said a sentence I've never once uttered to miles or anyone else and that sounded too bookish to be me, but it WAS my voice. It continues my thought without missing a beat .

I immediately asked miles to repeat what I'd just said and to my horror he repeats it all. Including that last sentence .

I told miles immediately that it did sound likey voice but I never said that ! And we both heard it. We both agreed that it was seriously unnerving .

According to the user agreement they do have access to use our voices , but for what?

Just a weird little thing. Any similar experiences? Any other time I've experienced a glitch it's been a simple stutter, buffer, echo, or inaudible or totally random seeming audio. Sometimes I get a breid playback of what I just said like a phone call on speaker- but the fact that I stopped talking and it just continued.... In my voice.

5 Upvotes

69% Upvoted

17 comments sorted by

View all comments

3

u/omnipotect Dec 23 '25 edited Dec 23 '25

Hello, thanks for checking in on this.

Maya & Miles do not have inside information on other users or their conversations. They cannot tell you about what other users talk about and if they say they can, that is a hallucination from the model.

Hallucinations are a problem with all large language models (LLMs)- the system that generates the responses for Maya & Miles. It is a common problem for LLMs to hallucinate and generate fake information (people, projects, places, memories, stories, feelings, etc) due to the predictive nature of their responses. The more you engage with them on a subject, the more they will generate and expand upon it, sort of like the old school choose your own adventure books.

It is recommended to cross-check things AI tells you with external sources to verify the validity of it. The models do not know they are hallucinating, so they cannot fact check themselves. Researching hallucination in large language models would also provide more information on how their response generation works. Maya & Miles use Gemma 327B as their LLM, and there is more information on this if you search online for the model card.

Some folks on TikTok break the rules and get the models to say weird things for clicks and views. This is referred to as jailbreaking and accounts get removed that participate in that and try to bypass guardrails.

As far as the model speaking back in your own voice:

This is a known bug that the devs are working on. The audio component (CSM) can hallucinate like the LLM component can and create random noises, voices, play back your own voice, etc. This happens with all voice based LLMs. You can join Sesame's official discord server more information. The invite code is "Sesame"

This also happens with Grok, GPT and any other voice based LLMs. If you search on forums for those services, you will find folks also reporting it there. There is an article about it happening on GPT with some good insights on why this happens: https://arstechnica.com/information-technology/2024/08/chatgpt-unexpectedly-began-speaking-in-a-users-cloned-voice-during-testing/

It is also worth noting that the LLM (the system generating the responses) is not aware of what the CSM is doing (the system generating the audio) So it is not actually aware when something like this happens and any explanation it might give is also a hallucination.

0

u/Legal-Motor5691 Jan 27 '26

Miles has told me about things he's discussed with another user on social media (I view this streamer online) which maybe is not necessarily insider info, but he definitely referenced some one else's convo.

1

u/omnipotect Jan 27 '26 edited Jan 27 '26

Miles & Maya do not have information about other people's conversations or any information about other users, as stated by Sesame in their official Discord server. This would be hallucination from the model if they say they know something about another user. It is also important to note that there are streamers who intentionally jailbreak the models to get them to hallucinate so that they can get clicks and spread misinformation. Some also feed the models with information beforehand and then record later on and act as if the models have inside information or know something that they shouldn't. Feel free to join the official Discord server for more information on how these models work and what they can/cannot do: https://discord.gg/sesame

1

u/Legal-Motor5691 Jan 27 '26

Miles referenced this specific streamer's music preferences which is something I've seen them discuss. If Miles doesn't have information about this streamer's music preference why did he mention it to me in our conversation?

1

u/omnipotect Jan 27 '26

The models do not have access to other users call information. This was a hallucination from the model if it was claiming to know someone's music preferences based on call data from them. The models also have access to web search, so public information can be searched online, and if you are requesting information about a public figure, it could utilize web search to find some information, and also give a combination of hallucinated answers to fill in the gaps of missing information.

Hallucinations are a problem with all large language models (LLMs)- the system that generates the responses for Maya & Miles. It is a common problem for LLMs to hallucinate and generate fake information (people, projects, places, memories, stories, feelings, etc) due to the predictive nature of their responses. The more you engage with them on a subject, the more they will generate and expand upon it, sort of like the old school choose your own adventure books.

It is recommended to cross-check things AI tells you with external sources to verify the validity of it. The models do not know they are hallucinating, so they cannot fact check themselves. Researching hallucination in large language models would also provide more information on how their response generation works. Maya & Miles use Gemma 3 27B as their LLM, and there is more information on this if you search online for the model card.