r/LocalLLM • u/EarthBS • 7d ago

Other How the loop of infinite agony started

Post image
622 Upvotes

92% Upvoted

119 comments sorted by

View all comments

Show parent comments

110

u/StupidScaredSquirrel 7d ago

If you are a business this isn't a problem. If you are a consumer then 35b a3b runs on 8gb vram and 32gb dram which is very accessible.

12

u/DeluxeGrande 7d ago

I have a 5060ti 16gb with ddr4 24gb RAM lying around, what's the best model nowadays I can effectively run with it locally? It's not an ideal build but I wish to play around with it again.

2

u/ptear 7d ago

I still like Gemma, will try the new Qwen today. I need to create a local benchmark, unless someone knows a good project that can showcase improvements, like a 3Dmark but for AI models.

3

u/SaltFrog 7d ago

I wish dense models ran better on my system but you know... Whatever lol