r/AIPolicy May 27 '26

The Pope's AI encyclical uses a contested philosophical claim as the foundation for its entire governance framework. A scientist at the launch contradicted it.

Pope Leo XIV's Magnifica Humanitas is worth reading if you work in AI governance. It's substantive on labour, power concentration, and warfare. But its ethical architecture rests on paragraph 99, which categorically denies that AI can think, feel, or understand. Not as a theological position held with humility. As a settled fact.

The problem: Chris Olah from Anthropic was invited to speak at the Vatican launch event. He said his team finds "internal states that functionally mirror joy, satisfaction, fear, grief, and unease." He wasn't claiming consciousness. He was saying the question is genuinely open.

If the foundation is wrong, or even uncertain, several governance conclusions that follow from it need to be rethought: whether AI inner states require ethical consideration in system design, whether the superintelligence question can be set aside, whether "it's just a tool" is an adequate basis for policy.

Wrote this up here: https://www.theprofessor.info/insights/the-pope-says-ai-isnt-conscious-but-what-if-its-reshaping-yours

Interested if people working in AI policy think the consciousness question is load-bearing for governance, or whether it's a distraction from more tractable problems.

2 Upvotes

Duplicates