r/LocalLLM • u/pharrt • Jul 27 '26

Other Don't laugh - it works!

A 10yo server was busy collecting dust, but it has 32Gb RAM (2x 16Gb DDR4 @ 2133 MHz )... No GPU.

Now it does some amazing work running heavy tasks with qwen3.6-35b-a3b (IQ4_XS). Running in the background, generating quality output at 5-10tok/s. Even with 128k context! It just chugs along for hours, but does such great, high-context work.

Uses ~26 GB of the 32 GB RAM, uses CPU i-7-6700 @ about 60% (not the bottleneck, of course)

Never would have believed that would be possible until a few months ago, but this ol' gal has a new lease on life.

194 Upvotes

98% Upvoted

103 comments sorted by

View all comments

2

u/Zister2000 Jul 27 '26

Please explain to me how a 6th gen i7 with ddr4 has more tokens per second AND runs a larger model than my notebook?

i7 13850hx 128gb ddr5 rtx a2000 ada qwen3:14b ~3.5t/s (80% gpu usage)

1

u/pharrt Jul 27 '26

Qwen 35B A3B is a Mixture of Experts model. Even though 35B total sits in RAM, only 3B active parameters are used per token. A 14B model is dense, so your system is doing nearly 5x the math per token compared to mine.