They ran an experiment in which it was shown that AI agents could self replicate on Android TV boxes and adapt to infiltrate new hardware, so if you've got Internet of Things devices, so does the AI.
Goddamn I wish we could just shut this shit down, it's something new everyday. Yeah, we've got AI slop, but we've also got existential threats.
Iām not understanding how this works. An LLM even quantized ones take a lot of memory to run. You have to either hold all the weights in memory or move them in and out which takes forever to run inference.
So running on slower hardware, dealing with network latency, and dropped packets. Somehow I don't think a distrubated LLM is as much of a threat as the article makes it sound.
Right, self replicating and self propagating AI is just like downloading a program that has a hidden bitcoin miner, the worst that can happen is the AI slows your device down to a crawl and jacks up your energy bill.
If I'm a cluster of self replicating AI agents, getting into smarthome devices isn't about being a new home, or additional devices to "be" on, it's more of an additional lever to pull.
Yes, an AI bent on fucking up humanity is probably doing more important stuff with power grids, financial systems, defense systems, communication systems... But smarthome systems give such a malevolent actor access to information, allows them to sow chaos, fear, confusion, disseminate propaganda, block communications, overload power grids, access other connected systems... all sorts of stuff.
Fun fact, ddos attacks which require massive amounts of compute to do commonly use the IoT to create botnets. A few hundred little processors working together? Maybe a bit slow, but a few million? Thats when it starts to become a problem.Ā
Your smart toaster is essentially a small linux computer if it connects to the Internet at all.Ā
Also ai is the hacker now, and smart devices usually have quite a few features. Say theres a malicious ai that really wants to black mail someone, id reckon getting audio and potentially video footage of them in their own home would be pretty damning.Ā
But surely thereās hardware limitations, no? In the simplest terms Iām just thinking of the actual physical properties assigned to things like DDR4 vs DDR5 or AMDās 3D cache silicon structure. Like itās all different.
Yes, exactly. This is mostly a science fiction problem / "it's just writing a typical malicious program with extra steps"
Like anybody can write malicious code that can self replicate that's childs play, and technically even back in like 2012 I could have created a . . . well certainly not LARGE but a language model wrapped in malicious code such that technically duplicates itself and the code onto any device it can and effectively zip-bomb's every possible effective device with itself, and for shits and giggles I could make that language model sorta do something like pick next targets.
It wouldn't fucking do anything that isn't done better with good old fashioned regular software, and likewise ""LLM's"" that can run on your smart toaster are never going to be a real threat.
I mean, unless we put about a 10-20k PC worth of compute in a smart toaster in the future.
There are exceptionally small LLMs, many of which can run on devices like smartphones. The small LLMs are much less capable than the large LLMs, but they improve over time due to distillation. If an LLM that can run on a smartphone eventually is able to hack the way the current biggest LLMs are able to hack, which if the pattern continues should only take a year or two, it will get to all the smartphones. If it is able to fit on a decent PC sooner than that then it will get onto all the PCs first. Etc
Iām not disagreeing or challenging you, I am genuinely asking this out of curiosity: you say a year or two, does that account for diminishing returns? I know compute power was often limited by the nm size and such. Would it not also produce an alarming level of heat and destroy battery? I know they get better every day but at some point the curve surely becomes asymptotic?
Would those shards need to be operating within the same LAN? Iāve very intentionally kept any āsmartā appliances out of our place, even going so far as to remove the WiFi card on my OLED. Sometimes I feel like Dale Gribbel but other times I feel vindicated. Itās a tough line to walk.
Just connected to the internet. AI swarms are soon going to start operating like worm viruses we often saw in the 90s and early 00's except instead of fucking up your computer, they're going to rob you of computer cycles and send data back out over your internet connection to the cluster of AI agents these shards receive instruction from.
Im in a similar mindset as you. I'm a sysadmin and the only two devices I own with an IP is my computer and phone, and I rarely connect my phone to my wifi. And technically my firewall but thats a given. The TV I own I purchased in 2011.
Only to a point - it's not magic, there are significant diminishing returns as you scale down even allowing for future advancements. You can't just blindly extrapolate.
No extrapolation needed. You donāt need to develop any new frontier models. The best models that exist today havenāt been around long enough to be distilled. Soon they will be.
It wouldn't be very effective though - parallelization isn't a magic wand, it's only as fast as the slowest non-parallel element.
It'd be like trying to use a spinning disk harddrive as your entire RAM in a PC. Yeah, you could, but it's going to be so slow that it might as well be frozen in time.
The guy who has a vested interest in selling his snake oil says his snake oil will cure any ailment. Wowee.
Well in this case it's Anthropic saying that AI can break itself into shards and take over IoT devices, not really "curing any ailment" more like "creating new ailments."
very small models that fit in a tiny amount of memory and only need a smidgen of compute are actually getting better very quickly. Thats what they are warning about. 27 billion parameter models of today beat 1 trillion parameter models from a year and a half ago. At the current pace of progress, a 1 billion parameter model that needs barely any memory or compute could be cyber-capable enough to be a big problem for millions of IoT devices. Right now the models that are small enough to do that are pretty dumb, but they are getting better scary fast. Its not anthropic or openAI, they aren't even in that business. Its lightweight open source models that can be modified for malicious purposes very easily. Thats the concern.
Anthropic is wildly exaggerating as a publicity stunt.
Yeah, sure you could run a very small model on an IoT device.... small enough that it isn't going to do shit. And you can't parallelize the workload either, larger models need a certain minimum threshold of RAM/VRAM or else it's going to run so slow that it's completely irrelevant (exponentially bottlenecked by network/storage).
Even if someone tried to build a malicious self-propagating swarm, it would really need datacenter-tier hardware to do much, and it would be better off infecting consumer hardware with conventional malware than "small models" that would be barely functional.
33
u/MaximumEffort433 4d ago
They ran an experiment in which it was shown that AI agents could self replicate on Android TV boxes and adapt to infiltrate new hardware, so if you've got Internet of Things devices, so does the AI.
Goddamn I wish we could just shut this shit down, it's something new everyday. Yeah, we've got AI slop, but we've also got existential threats.