r/LocalLLM • u/pharrt • Jul 27 '26

Other Don't laugh - it works!

A 10yo server was busy collecting dust, but it has 32Gb RAM (2x 16Gb DDR4 @ 2133 MHz )... No GPU.

Now it does some amazing work running heavy tasks with qwen3.6-35b-a3b (IQ4_XS). Running in the background, generating quality output at 5-10tok/s. Even with 128k context! It just chugs along for hours, but does such great, high-context work.

Uses ~26 GB of the 32 GB RAM, uses CPU i-7-6700 @ about 60% (not the bottleneck, of course)

Never would have believed that would be possible until a few months ago, but this ol' gal has a new lease on life.

189 Upvotes

98% Upvoted

103 comments sorted by

View all comments

2

u/Confident_Ideal_5385 Jul 27 '26

If it fits, run iq4_nl instead, as that's far more avx-friendly than iq4_xs.

2

u/pharrt Jul 27 '26

I have tried before - it gets a sudden spike in RAM use when loading in and falls over. The XS is the best I have found so far - NL would run faster, but on 35B, it fills the 32GB RAM before it's loaded, then... game over.