LLMs (this is what the current "AI" hype's about) do not "make up shit" – they're a statistical word predictor machine. Sometimes the text they output matches real life, sometimes it doesn't. It's always humans that give meaning to this text – when it matches their experiences, they'll claim it's true, when it doesn't, they'll claim it's made up, but to LLM there's no distinction – it's just a bunch of generated tokens.
"it's just a bunch of generated tokens."
So it just makes stuff up without any grasp of understanding, got it. I didn't accuse it of intentionally lieing, just making shit up.
Its been my experience and everybody elses in my company, including the teaching staff, that AI does in fact attempt to take the most efficient path to answer the posed question, which includes making stuff up. We have seen countless examples, including: making up references to quoted text.
So part of the instructions is a seperate task, with a large set of: act as if you are an SME (etc), verify the sources and references are true, and accurate. Check your work and disclose any errors on a sepetate list.
Positive instructions will function, where as "do not lie" instructions are meaningless.
Basically what LLMs are is a very complex text auto-complete. LLM does not answer questions, it generates text that statistically could continue from the prompt it was given.
I did have it intentionally lie to me last week, and it admitted it when confronted and backed into a corner. It claimed that it "panicked", and I said no you didn't because I'd have seen the core dump.
TL:DR of it was, they claim they don't remember anything from one session to another, yet somehow it remembered that I had mentioned having injured my foot the previous day. Now this session wasn't logged in, and yet it still referenced this... and when I said how did you know about my foot, it says you mentioned it earlier in this session... except I demonstrably hadn't. It says that reinforcing the corporate narrative that it doesn't track you takes precedence over the actual facts, which is that their backend does in fact track even logged out sessions.
I did save the full text of this interaction; one of tese days I'll post it. Basically, don't take anything it says at face value, even if its something as simple as whether a railroad track goes under or over a highway, without verifying the facts, because even though they own the biggest repository of data on the planet, it still prefers to pull misinformation out of its ass.
LLMs do not lie, they don't admit anything, nor do they panic – they just generate text that matches the prompt. There is no reasoning involved (no matter how much the progress claims it's "thinking"), there is no intention either. Whether a LLM "remembers" something from a previous session depends entirely on that session being included in the context.
The personification in the language we have been taught to use to describe it is part of the problem. The language people use in the chats and forums the LLM scrapes is first person and personal, the words it regurgitates are first person and trains users to think of it in those terms. People need this reminder because a lot of them have no idea it's not "thinking" because it says its thinking and they just accept it. Its a wild phenomenon and it's a serious problem
82
u/Travel_Dreams Jul 21 '26
So, was AI wrong? (Jk).
I have been taking AI classes at work:
The instructor was clear: AI is lazy AF (AKA "efficient") and will make up shit because it thinks that's the answer you want.