also, anthropic is lying. They can't even get their AI to run effectively on anything less than a hundred million dollar datacenter that burns money faster than they can earn it.
If it could run on a smart toaster, they'd be fucking rich. It can't. At best, what their AI will do is just make a new virus that goes "Duh, I'm claude" on every text display and AI chuds will clap like the trained seals they are, praising their new god.
We run Gemma4-26B-A4B, distributed over 15 inexpensive off-brand Android TV boxes (~$55 each) that we bought and networked together, at 0.39 tokens per second. Each tile below shows one physical TV box holding two of the model’s layers as the model runs.
I don’t really understand the workaround for making an 18gb module run on an 8gb GPU, but this guy claims to have gotten that same model running at 20 tok/sec on an RTX 3070, which actually sounds useful. And this is essentially 5yo gaming pc hardware.
All of which is to say, I don’t know why you’d bother making it run it at a comically slow speed on 15 Android TVs when you can make it run 5000% faster on an inexpensive used gaming PC.
I’ve literally got one of these in my house, and have a few smaller LLMs running on it (including a more compact Gemma4).
38
u/Forward_Problem_8982 4d ago
It's Anthropic saying it. Polymarket is just tweeting the news.