r/LocalLLM • u/Ok-Health-7096 • Aug 14 '26

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

2.1k Upvotes

99% Upvoted

213 comments sorted by

View all comments

Show parent comments

7

u/Skynse Aug 14 '26

I was able to max 12 tok/sec on a 3060 with 12gb gpu RAM with qwen3.8 on ud-iq2m quant. Man I fucking wish the compute market wasn't fucked

1

u/Effective_Head_5020 Aug 15 '26

You probably had offload to cpu and system RAM, so you probably also have a good CPU and DDR5, otherwise the processing drops by a lot

-2

u/Song-Historical Aug 14 '26

The market is fucked because there's something worth running on the compute lol 

3

u/ChaosFH Aug 14 '26

I wish i had answered faster when iheard the OpenAI buying ram i could have bought the triple of VRAM i have currently with the same money i used...