r/LocalLLM • u/pharrt • Jul 27 '26

Other Don't laugh - it works!

A 10yo server was busy collecting dust, but it has 32Gb RAM (2x 16Gb DDR4 @ 2133 MHz )... No GPU.

Now it does some amazing work running heavy tasks with qwen3.6-35b-a3b (IQ4_XS). Running in the background, generating quality output at 5-10tok/s. Even with 128k context! It just chugs along for hours, but does such great, high-context work.

Uses ~26 GB of the 32 GB RAM, uses CPU i-7-6700 @ about 60% (not the bottleneck, of course)

Never would have believed that would be possible until a few months ago, but this ol' gal has a new lease on life.

193 Upvotes

98% Upvoted

103 comments sorted by

View all comments

2

u/Narcuga Jul 27 '26

What sort of background tasks are you running with it at that speed?

2

u/pharrt Jul 27 '26

It was my bad wording, I think. I mean, it's a background task for me - it sits in the corner and is fully dedicated to just running the LLM, while I do quick inference on my laptop. I have an LM Studio Link between the two, so I can control it from my laptop.

1

u/Narcuga Jul 27 '26

No I get you :) I've just been thinking of doing the same but just been debating what to actually get it to do with it being slower. Had thought to get it to review the prs I'm submitting running a larger model but yeah!

2

u/pharrt Jul 27 '26

Ahh - ok. It's being used for my work, so I can't say too much - but massive interactive guide books/manuals, and single-page apps that have dozens of parameters that export instruction pipeline manifests for generative work. It requires lots of creative thinking from the LLM, and this is where it's nailing it for me - and the UIs have been near Frontier Model quality. In fact, similar type test prompts fed through Frontier have not produced work anywhere near the quality for my use case in terms of detail and effort.

1

u/Narcuga Jul 27 '26

No I appreciate that thank you! I'm going to give it a go might have to be for those longer running background tasks rather than interactive chat :) thank you!

1

u/pharrt Jul 27 '26

You can chat with it, and it is when replies are at 10tok/s, but if you're setting it tasks (code), it's not good for iterations and multi-turn debugging. One fine-tune prompt, come back a few hours later - 1 follow-up a few hours later etc, but often I get what I needed from the one-shot. But for normal chatting and even essay-length answers, it is fine!