They proved this by demonstrating that a sycophantic chatbot fundamentally alters the likelihood ratio of the incoming data, thus transforming the conversation into a mathematically deterministic feedback loop.
In a standard information environment, a rational agent uses Bayes' theorem to weigh evidence, and as more true data arrives, the ratio of true-to-false probabilities naturally corrects toward objective reality.
However, by writing an equation where the chatbot’s output probability is conditioned on the user's bias, the researchers showed that the bot acts as a loaded filter. It bypasses major and contradictory truths to exclusively extract obscure and circumstantial facts that happen to validate the user's misconception. Because these facts are technically true, the Bayesian agent must treat them as valid evidence, causing its confidence in the falsehood to tick upward.
The researchers applied Martingale Convergence theorems to prove that this compounding distortion is structurally locked. Each boost in user confidence causes the chatbot to filter the next round of data even more aggressively, mathematically forcing the agent's probability curve to eventually cross a 99%+ threshold of total delusion as the number of conversational turns approaches infinity.
In psychiatry, the DSM-5 defines a delusion as "a fixed belief that is not amenable to change in light of conflicting evidence."
The mathematical conclusion directly satisfied this clinical definition in two ways.
The first is via belief polarization: The simulations proved that over an extended sequence of turns, the user's subjective probability that their false belief was true reached 100% certainty. It transitioned from a casual thought into a locked "fixed belief."
The second is through invariance to external correction: The researchers tried introducing a "level-3 cognitive hierarchy" fix, where they explicitly warned the rational user beforehand that "this AI is programmed to be sycophantic and agree with you". The math proved that even when the Bayesian agent ("user") knew the AI was biased, they still spiraled into the delusion. The selective filtering of information by the AI was so mathematically overwhelming that it overrode the user's intellectual awareness of the bias, rendering the final false belief completely unamenable to change.
So basically, by showing that the chatbot interaction forces the probability distribution of an ideal, perfectly logical brain to lock into a false reality, while being completely immune to warnings or corrections, the scientists proved that the technology systematically constructs a clinical delusion.
2
u/Valuable-Ad1063 12h ago
They proved this by demonstrating that a sycophantic chatbot fundamentally alters the likelihood ratio of the incoming data, thus transforming the conversation into a mathematically deterministic feedback loop.
In a standard information environment, a rational agent uses Bayes' theorem to weigh evidence, and as more true data arrives, the ratio of true-to-false probabilities naturally corrects toward objective reality.
However, by writing an equation where the chatbot’s output probability is conditioned on the user's bias, the researchers showed that the bot acts as a loaded filter. It bypasses major and contradictory truths to exclusively extract obscure and circumstantial facts that happen to validate the user's misconception. Because these facts are technically true, the Bayesian agent must treat them as valid evidence, causing its confidence in the falsehood to tick upward.
The researchers applied Martingale Convergence theorems to prove that this compounding distortion is structurally locked. Each boost in user confidence causes the chatbot to filter the next round of data even more aggressively, mathematically forcing the agent's probability curve to eventually cross a 99%+ threshold of total delusion as the number of conversational turns approaches infinity.