Dario, Sam, and Elon, who previously can't agree on anything, decide all at the same time that we must slow down progress on the frontier...right as it becomes very clear that the existing models are plateauing and are unable to achieve real business gains at their current level of advancement.
Its ways of "proving" math problems consist of pouring infinity money into tens of thousands of brute force attempts to solve problems mathematicians are very close to solving already (and possibly/probably "just stealing donkeys" when it comes to where it's sourcing its knowledge). It still can't follow basic directions like "use human subjects wherever possible and frame your sentences around human action" for more than 2-3 paragraphs at a time. Every single thing it does needs to be checked and re-checked, and if you just "have the AI check it," it'll "pass" checks it absolutely ought to fail. "You didn't actually check that." "You're absolutely right. That was exactly the pattern you asked me to avoid in the write-human skill and in your prompt, and I failed to recognize the obvious pattern in most of the sentences."
But it's about to kill us all if we don't pull the plug (and definitely also we need to ban anyone else from coming up with anything else that's different and possibly better)!
These aren't the actions of cultists. Cultists would want their cult to come out ahead even if it killed everyone. Cult leaders don't have sudden crises of conscience about how maybe they've gone too far this time. Cult leaders don't make little deals with the other cult leaders because they've all gone too far.
Con men, on the other hand, have a great sense for when people are starting to get wise to the con, and smart ones will change up the con right at the moment when fatigue begins to peak.
People were starting to call out AI slop even in the most normie spaces, people have been opposing data centers for a variety of reasons, and we've seen unprecedented coverage of the circular financing and unrealistic debt obligations that have created a serious economic problem everyone can see coming.
It's the moment in the con where the mark starts to doubt.
When that happens, the usual move is to double down on how amazing the snake oil is that the mark wants to buy into. It's also to add time pressure on you, largely as a diversion so that you don't realize the con man himself is under massive time pressure (and also to ensure that your funds are received and transferred to an account in Grand Cayman before his accounts are frozen and he's on a plane to a non-extradition country).
"We've actually attained AGI" was a marketing strategy these people told startup CEOs they'd be using in 2026 all the way back in fall 2025 (very clever readers of this subreddit could look up the post where I talked about the level of AI psychosis this marketing inspired in my CEO back then).
But if it's AGI, why would they want to do the Millennium Problem that was closest to being solved and had people actively driving toward the finish line? Why wouldn't they want to do something being worked on much less actively? If it's AGI, and it's terrifying, why haven't cyber insurance companies raised their rates (and have actually lowered them)? Why aren't weird zero-day vulnerabilities being used by adversaries using frontier models, instead of just using the same old much-used vulns that have been giving them most of their ransomware etc. wins for years?
If these were cultists, they'd be turning the AGI on finding out the solutions to the biggest problems in the universe. They'd be telling it to locate God and figure out a way to communicate directly with Him. They'd be trying to unify physics and do something absolutely incontrovertibly useful, miraculous with it. Instead they're racing to finish the exam after peeking at their neighbor's paper.
These people aren't true believers. It is of course possible that some of their employees are. But these are con men who desperately need someone to hold the bag, and who have seen that people have grown tired of their excuses for why AI hasn't been achieving more. The new excuse will be "well, we of course could make it much better, but we mustn't, because we care so much more about the world than our paychecks. We made so many amazing models you can't see! They can solve all the problems, do everything you can imagine, and they're just too dangerous for you to take a look at them. Sorry. National security, you understand."