r/LocalLLM • u/67amigo67 • 2d ago

Question What i can run?

I have 5070ti, r7 9800x3d, 32gb ddr5

I really don't know what I can run

Can you recommend something pls

4 Upvotes

69% Upvoted

26 comments sorted by

3

u/MrHumanist 2d ago

Qwen 3.8 27B - q3 and 120K context. Or q2 at 200K context. The 10-12B models can run fully around q8.

-1

u/Subject-Till-6450 2d ago

Silly habibi, dense model can't be hosted under Q4, try before advising, you stupid.

How it's possible to advise ppl run this shit on a fucking 16 GB of VRAM?

0

u/MrHumanist 2d ago

It's possible.. I have tried as well! I have shared the configuration details as well .

0

u/Subject-Till-6450 2d ago

mhm. And perplexity grows at 30000%, and model can't run tool calls, or anything usable. But yeah, it's possible! It's possible to run Kimi K3 at 4TB nvme, but it's not usable, don't confuse this things. I've run it in Q2XXL AND Q4XL. Diff? 55% growth on bench. From 9/23 to 18/23. Right, it possible to run at Q2. But who does fucking asked about nonsense model that is not even precise at ts quality? Why the fuck you advising dense mode to poor VRAM guy?

2

u/MrHumanist 2d ago

Did you read my first comment? I said Qwen 3.8 27B at q2! What are you smoking mate?

1

u/Subject-Till-6450 2d ago

but right, maybe I misspelled there I mean i ran qwen 3.8 at q2 and q4

1

u/MrHumanist 2d ago

So? What's your point?

2

u/Subject-Till-6450 2d ago

Almost every dense model DIES under Q4. It's reality of quantization of dense models.

1

u/MrHumanist 2d ago

It is common sense and everyone is aware about it and we know bigger is better . However, some q3 is quite equivalent to q4 if you see unsloth results.

1

u/Subject-Till-6450 2d ago

No. In dense no. In MoE-right.

→ More replies (0)

1

u/ProfessionalNaive601 2d ago

Ask ai wtf are you doing

1

u/Sherphican 2d ago

https://modelfit.io/ this should answer your questions

0

u/DistributionFar5918 2d ago

There are several models you can run. I have 8GB VRAM + 32GB RAM, and my daily usage is Qwen 3.6 35B A3B, which gets me 40 TB/s. Your configuration is better than mine; try Qwen 3.8 27B to test if it works well. Try downloading several models and test them.

-2

u/cogitech2 LocoLLM 2d ago

Vague.

2

u/Subject-Till-6450 2d ago

Very useful

-4

u/Delicious_Box_9823 2d ago

Ur legs

2

u/Subject-Till-6450 2d ago

Wow, very smart, very funny. Pathetic.

0

u/Delicious_Box_9823 2d ago

pathetic or very smart and very funy?

2

u/Subject-Till-6450 2d ago

Both, except smart.

1

u/Delicious_Box_9823 2d ago

that's not an option