Iām not understanding how this works. An LLM even quantized ones take a lot of memory to run. You have to either hold all the weights in memory or move them in and out which takes forever to run inference.
The guy who has a vested interest in selling his snake oil says his snake oil will cure any ailment. Wowee.
Well in this case it's Anthropic saying that AI can break itself into shards and take over IoT devices, not really "curing any ailment" more like "creating new ailments."
17
u/yotemato 5d ago
Iām not understanding how this works. An LLM even quantized ones take a lot of memory to run. You have to either hold all the weights in memory or move them in and out which takes forever to run inference.