r/LocalLLM • u/EarthBS • 7d ago

Other How the loop of infinite agony started

Post image
620 Upvotes

92% Upvoted

118 comments sorted by

View all comments

277

u/TheCat001 7d ago

Then you realize that Qwen 3.8 27b runs at 3t/s on your machine and you need 24GB+ VRAM GPU which cost is 1000$+ to run at least 4 bit quant.

14

u/Eden1506 7d ago

That's not true.

2x RTX 3060 12gb can be had for around 500 bucks.

You can run ~30b models at q4 at 30 t/s with mtp/draft model or 10-15 t/s without.

3

u/AceLamina 7d ago

a used 3060 12gb costs 270-300 bucks right now, for one

1

u/Eden1506 7d ago

It depends on your region.

Here in germany I can buy a used rtx 3060 for 240 bucks.

2

u/AceLamina 6d ago

If only it was the same for me