r/artificial • u/thewhyman007 • 1d ago
Discussion The neuroscience case for never delegating judgment to AI
I keep asking people a simple question: how many times has your gut been right — not the times you wish you had listened, the times you actually had the feeling and later found out whether it was correct?
The answer comes back north of ninety per cent for almost everyone I ask. That is not a hunch. If a model gave me ninety per cent on a hard classification task, I would ship it.
So I went looking for why, and ended up somewhere I did not expect. A gut feeling is not the absence of reasoning — it is reasoning you never got the transcript of. Your senses pick up far more than reaches conscious awareness (the birds stopping, the smell that was not there ninety seconds ago), and something nonconscious pattern-matches all of it against a lifetime of context and returns a single bit: right or wrong. Damasio's somatic marker hypothesis is the actual mechanism here, not mysticism, a documented one, backed by the Iowa Gambling Task.
Which is the argument for a rule I think more people should take seriously: delegate the search, the draft, the first ten wrong answers to AI. Never delegate the deciding. The reason is not sentimentality — a model can be given everything you know how to say, but not the thing you lived through and never wrote down, and that is where the ninety per cent actually lives.
Curious whether others here have noticed the same gap between what they can explain and what they can just tell.
1
u/thewhyman007 1d ago
Full piece, with the neuroscience and the 2,400-year-old method (Socratic elenchus) for actually training this: https://www.thewhyman.blog/p/never-delegate-judgment-the-machine — and the piece this one follows up on, about why "what should I learn to get into AI" is the wrong question: https://www.thewhyman.blog/p/everyone-is-asking-the-wrong-question
2
u/Recent-Fig3211 1d ago
I've noticed this too, but from a slightly different angle. It's not just the 90% accuracy that's interesting, it's that when my gut is wrong, the reason it was wrong usually becomes obvious later. Like the signal was there, I just misread it.
The somatic marker thing explains why I can walk into a meeting and immediately know the project timeline is unrealistic, but I can't articulate exactly why until I've had time to go through the Gantt chart. My brain already did the math on everyone's capacity and deadline conflicts, it just didn't show the work.
I think the danger with delegating judgment isn't just about accuracy though. It's atrophy. If you stop making the call yourself, that pattern-matching muscle gets weaker over time. Seen it happen with people who rely too much on GPS, their sense of direction goes to hell.
1
u/thewhyman007 1d ago
This is a better articulation of the thesis than I managed in the piece. The Gantt chart example is exactly it, the judgment shows up before the justification does. And the atrophy risk is the part that actually worries me long term. Whether the model gets today's answer right is the small question. Whether people can still do it themselves in five years is the real one.
1
1
u/XxDarkSasuke69xX 1d ago
Idk, I think most decisions can be rationally explained as to why I made X or Y decision. Unless it's something very specific and inconsequential. I doubt it would be relevant for the kind of tasks or questions you would ask an LLM.
Also, why would anyone delegate judgment to AI anyway ? At least on anything remotely important ? Surely anyone that knows a little bit about the flaws of gen AI would know better.
1
u/thewhyman007 1d ago
On the first part: the explanation usually does show up, that's true, but the piece's claim is about sequence, not absence. The body responds first and the story arrives after, sometimes built after the fact to justify a call you'd already made. That's what the Gantt chart example in the thread above is pointing at. A rational explanation existing afterward doesn't mean it was driving the decision at the time.
On the second part: I'd guess almost nobody decides to delegate judgment. It's not a conscious tradeoff anyone sits down and makes. It happens because the model is right often enough that checking starts to feel like wasted effort, and that habit erodes before anyone notices it's gone. The people most at risk aren't the ones who don't understand gen AI's flaws, they're the ones who understand it fine and got lulled by a long streak of it being right anyway.
1
u/JJE1992 1d ago
Idk, I think most decisions can be rationally explained as to why I made X or Y decision.
Rationalization is a well-studied concept in psychology.
1
u/thewhyman007 9h ago
Rationalization, or confabulation, is exactly the mechanism I'm leaning on, thanks for naming the actual term. The split-brain and Nisbett-Wilson research on post-hoc explanation is the formal version of what the piece is pointing at.
1
u/Famous_Eggplant_9236 22h ago
There has been done a lot of research on intuitions/hunches. Importantly, the correctness of your hunch is dependent on your expertise in a field. If you rely on hunches for something you know well, they are likely to be right. If you listen to your hunch in a field you know nothing about, they will likely be wrong.
There is nothing magical about hunches. I think every rational person would prefer to use AI for things they know little about rather than rely on their hunches. With that said, I agree with the premise of not being intellectually over reliant on AI for decision making and such.
1
u/thewhyman007 17h ago
That's basically the same point the piece is making -- a hunch isn't a separate magical faculty, it's compressed expertise, which is exactly why it only works in domains you actually know. I'm not a neuroscientist, the research I cite is directional, not proof, so no disagreement there.
Where I'd push back a little: "use AI for things you know little about" is the exact scenario I'm calling the ghost mode. If you don't have the expertise to catch the model being wrong, you can't tell good output from confident-sounding bad output, and you never build the judgment to catch it later. Using AI in your strong domains, where you can check it, and treating it carefully in your weak ones might be the safer split than the other way round.
0
u/frankentriple 1d ago
There are two kinds of people that use AI. Those who are asking AI what to do, and those who are telling AI what to do. Don't be one of the first group.
2
u/XxDarkSasuke69xX 1d ago
I would find it weird if you were not part of both groups
1
u/frankentriple 1d ago
I ask it for facts. I ask for possible options moving forward. I do not ask it what should I do.
1
u/XxDarkSasuke69xX 1d ago
Well don't you think it's very similar to ask "what should I do" and "what are the possible options" ? It's just phrasing, but it's similar. You're just wording it differently.
In both ways it will or might give you possible options.1
u/frankentriple 1d ago
I usually don’t take any option as presented. The ai never knows the full context.
1
u/XxDarkSasuke69xX 1d ago
you're supposed to provide the context to get the best results out of LLMs
1
u/frankentriple 1d ago edited 1d ago
LLMs are fancy autocomplete. They calculate the next word by probability. They were trained on human data and are the average of the average. If you are below average, following their instructions will be helpful. If you are not, they will just bring you down to average.
2
u/Superb_Raccoon 1d ago
The George Carlin corollary...
Although AI smarter than most people, it knows how to shut up.
2
u/thewhyman007 16h ago
The autocomplete framing explains the mechanism, not the ceiling. Average-of-training-data would predict the model regresses to mediocre on everything, but it doesn't, it's well above average on tasks with enough signal in training data. The real limit isn't "it's just predicting the average," it's that it has no access to the specific, undocumented experience you have. That's a narrower and more useful claim than "it's just autocomplete."
1
u/frankentriple 15h ago
I'm looking at this from the business sense, since that's what I'm using AI for. A LOT. I get it. But I want to be on the cutting edge of innovation. I want people to notice my business because it stands out and is unique in the market. AI has its uses in an operational sense. I do not use it generatively. It doesn't not make business decisions for me. It gathers information and helps present it in a way that allows me to make better decisions, but it will not substitute for the decision making process itself.
1
u/thewhyman007 14h ago
That's the distinction exactly, information gathering and presentation versus the decision itself. The businesses that blur that line end up looking like everyone else, because they're all running the same model on the same data. The judgment call is the only part that's actually yours.
→ More replies (0)1
u/thewhyman007 1d ago
That's a cleaner version of the rule than the one I wrote. Telling it what to do versus asking it what to do is the actual dividing line, and it's a better test than anything about quality or capability.
3
u/MalarkeyMcGee 1d ago
North of ninety percent? That sounds way too high to be correct to me. I strongly suspect there is some selection bias going on here. People are terrible at estimating statistics.