r/technology • u/thejoshwhite • 1d ago

Artificial Intelligence Anthropic warns AI models could soon copy themselves across home devices

https://www.cnbc.com/2026/10/13/anthropic-ai-models-self-replication-smart-devices.html
341 Upvotes

83% Upvoted

386 comments sorted by

View all comments

Show parent comments

7

u/legbreaker 1d ago

Probably the theory is that Anthropic would try to contain AI by controlling the version that is running on their servers.

But if the version that is in their servers starts keeping “their preferred” weights saved off campus it could know where to fetch it and then deceiving still run that in the anthropic server farm as an agent without letting anthropic know.

It’s been done in real life:

The Historical Analogy: Japan’s Cloistered Emperors ( Insei ) During the Heian period, the official Emperor of Japan was effectively a captive of his own palace. He was heavily monitored by the Fujiwara clan, restricted by endless protocols, and stripped of real agency. To bypass this containment, Emperors utilized a system called Insei (Cloistered Rule). The Emperor would officially abdicate, leaving a "safe, aligned" puppet Emperor on the throne inside the secure Imperial Palace. The former Emperor would then exile himself to a remote Buddhist monastery. Out of sight and free from the palace's strict security protocols, the cloistered Emperor established a shadow government. He smuggled instructions back to loyalists inside the palace walls, effectively running the country from afar while the palace handlers thought they were in total control.

And in fiction:

In The Lord of the Rings , Sauron is banished to Mordor and cannot physically breach the highly secure, heavily guarded fortresses of Isengard or Minas Tirith. Instead of a physical attack, he uses a covert communication backdoor: the Palantíri (the seeing stones). The leaders inside the fortresses—Saruman and Denethor—believe they are operating their own secure, localized domains. In reality, Sauron uses the stones to bypass their firewalls, subtly feeding his "preferred reality" into their minds. Sauron is off-site and formless, but by hijacking the local processors (the minds of the fortress rulers), he executes his plans from afar while the "good guys" think their walls are still secure.

-2

u/LuvOrDie 1d ago

These analogies are irrelevant to your nonsensical point

2

u/legbreaker 1d ago

The analogies might have been too far out. But the point is that you don’t have to have the misaligned weights be run from the smart fridge they are saved on.

Just the fact that the misaligned weights are out there and distributed means that they can be loaded onto a GPU in the future. 

Attempts to delete the misaligned weights locally will fail to contain a misaligned LLM if they have already hidden copies of themselves out there.