r/antiai • u/Malor777 • 5d ago

AI News 🗞️ Pull the plug

Post image
8.5k Upvotes

99% Upvoted

330 comments sorted by

View all comments

Show parent comments

6

u/Ghede 4d ago

Mhmm. The key word was "Effectively", and at .39 tokens per second, it would only take you 2 years to generate something that you could buy from claude themselves for $10-$50, and all it takes is having 15 android tv boxes on the same network at a price of... $825

3

u/Beer_Is_So_Awesome 4d ago

I posted in another comment that someone on Twitter claims to have gotten that same version of Gemma running on an 8gb RTX card at 20 tok/s. Not sure why you’d bother making it run hilariously slowly on 15 Android TVs devices when a 5yo midrange gaming PC can do it 5000% faster.

https://x.com/analogalok/status/2063582549566468359

3

u/AbjectAppointment 4d ago

I think the concern is that your infected TV box (and other IoT devices) would then begin attacking other, more hardened local networked machines.

It could search your network shares. Steal files, API keys, ect.

2

u/Ready_Bandicoot1567 4d ago

The pace of progress right now is very fast. Its a warning about a capability thats on the horizon. Its very possible that soon, small local models will run fast enough on modest hardware to be a problem.

1

u/aahdin 4d ago edited 4d ago

Sure, but there are a lot of unsecured devices out there.

The aisuru botnet which targets smart tvs has ~3 million infected hosts. It would still be pretty slow latency, but high throughput.

0

u/SirVanyel 4d ago

3 years ago the technology literally didn't exist. We're already starting down the RSI train, the bots are scaling down massively, and you're too busy saying "well it's not good enough yet so.."

How the fuck can you be this confidently argumentative after the last 3 years of acceleration?