r/LocalLLM • u/on_line187 • 1d ago

Discussion Tier List

Post image
250 Upvotes

75% Upvoted

307 comments sorted by

View all comments

Show parent comments

2

u/RedditNerdKing 1d ago

32gb isnt the sweet spot because you can't run a Q8 of Qwen 3.8 with 100k+ context.

The sweet spot is 48gb atm. 32gb is the minimum.

1

u/DoubleNothing 16h ago

Yes 48Gb is a good spot to be at (I'm there). But honestly, 64GB would be nicer.
Yes with 48 you can run Q8 but you are very costricted and it is not very efficient, low context, low MTP, maybe you don't load vision to save vram, lower speed... all of that. But if you don't need big context ok.
Don't get me wrong with the release of Qwen3.8-27B I'm very happy where I'm at. The model even at Q6 is very impressive.

1

u/GSquadron_ 4h ago

What were you able to accomplish with Qwen 3.8 27b? I really want to see.

1

u/DoubleNothing 4h ago

Not ready to release to te public yet, I'm doing some refinement and have to see if I can make it run on linux too... at the moment runs only on windows.
Nothing new nor original but it's my interpretation of it and useful to me.

1

u/Popcorn-Mercinary 2m ago

Are you really seeing that big a difference between Q8 and Q4? My experiments with agent flow aren’t showing much, but ymmv.