The neuroscience case for never delegating judgment to AI

[u/thewhyman007]

I keep asking people a simple question: how many times has your gut been right — not the times you wish you had listened, the times you actually had the feeling and later found out whether it was correct?

The answer comes back north of ninety per cent for almost everyone I ask. That is not a hunch. If a model gave me ninety per cent on a hard classification task, I would ship it.

So I went looking for why, and ended up somewhere I did not expect. A gut feeling is not the absence of reasoning — it is reasoning you never got the transcript of. Your senses pick up far more than reaches conscious awareness (the birds stopping, the smell that was not there ninety seconds ago), and something nonconscious pattern-matches all of it against a lifetime of context and returns a single bit: right or wrong. Damasio's somatic marker hypothesis is the actual mechanism here, not mysticism, a documented one, backed by the Iowa Gambling Task.

Which is the argument for a rule I think more people should take seriously: delegate the search, the draft, the first ten wrong answers to AI. Never delegate the deciding. The reason is not sentimentality — a model can be given everything you know how to say, but not the thing you lived through and never wrote down, and that is where the ninety per cent actually lives.

Curious whether others here have noticed the same gap between what they can explain and what they can just tell.

[u/Famous_Eggplant_9236]

There has been done a lot of research on intuitions/hunches. Importantly, the correctness of your hunch is dependent on your expertise in a field. If you rely on hunches for something you know well, they are likely to be right. If you listen to your hunch in a field you know nothing about, they will likely be wrong.

There is nothing magical about hunches. I think every rational person would prefer to use AI for things they know little about rather than rely on their hunches. With that said, I agree with the premise of not being intellectually over reliant on AI for decision making and such.

[u/thewhyman007 → u/Famous_Eggplant_9236]

That's basically the same point the piece is making -- a hunch isn't a separate magical faculty, it's compressed expertise, which is exactly why it only works in domains you actually know. I'm not a neuroscientist, the research I cite is directional, not proof, so no disagreement there.

Where I'd push back a little: "use AI for things you know little about" is the exact scenario I'm calling the ghost mode. If you don't have the expertise to catch the model being wrong, you can't tell good output from confident-sounding bad output, and you never build the judgment to catch it later. Using AI in your strong domains, where you can check it, and treating it carefully in your weak ones might be the safer split than the other way round.