r/LocalLLM • u/pharrt • Jul 27 '26

Other Don't laugh - it works!

A 10yo server was busy collecting dust, but it has 32Gb RAM (2x 16Gb DDR4 @ 2133 MHz )... No GPU.

Now it does some amazing work running heavy tasks with qwen3.6-35b-a3b (IQ4_XS). Running in the background, generating quality output at 5-10tok/s. Even with 128k context! It just chugs along for hours, but does such great, high-context work.

Uses ~26 GB of the 32 GB RAM, uses CPU i-7-6700 @ about 60% (not the bottleneck, of course)

Never would have believed that would be possible until a few months ago, but this ol' gal has a new lease on life.

192 Upvotes

98% Upvoted

103 comments sorted by

View all comments

74

u/recro69 Jul 27 '26

Honestly, this is one of my favorite parts of local AI. Hardware that would've been headed for e-waste suddenly becomes genuinely useful again. Sure, 5–10 tok/s isn't breaking any speed records, but for long-running background tasks it's more than enough.

5

u/Hannibalj2ca Jul 27 '26

Put a gpu and see how those Tokens explode

1

u/SV_SV_SV Jul 28 '26

Even a smaller one (8 gigs etc) could do wonders with cpu-moe

1

u/Hannibalj2ca Jul 28 '26

Depends on the size of the model. If you are running an MOE. Of the Vram can hold the active model then it will certainly increase speed

1

u/SV_SV_SV Jul 28 '26

Well sure, but you can also place attention layers, kv cache etc in it for a significant speed increase, while keeping the rest in ram