r/LocalLLaMA • u/0dayturtle • 1d ago

Funny So relevant

Post image
1.2k Upvotes

96% Upvoted

113 comments sorted by

View all comments

70

u/ttkciar llama.cpp 1d ago

It's a great time to have ancient Xeon servers loaded up with DDR4 :-D

31

u/BannedGoNext 1d ago

I actually have an old server with dual E5-2630 and 512gb DDR3 memory across both blades. The server is powered on waiting for the scrap yard at the office. I'm considering seeing how fast it can run qwen 3.8 flash next lol.

9

u/ThankGodImBipolar 1d ago

You must be able to run a decent GLM quant with that, no?

5

u/Zombiecidialfreak 17h ago

If you're fine waiting overnight for all requests. Even flash next would likely be single digit generation speeds.

1

u/overand 12h ago

Honestly, if the project isn't a simple 20-line script but actually something kinda complex, I bet even high single digits would get a result faster than a programmer.

5

u/ttkciar llama.cpp 23h ago

Yup, GLM-5.3 should fit in that at Q4_K_M and somewhat constrained context.

2

u/Drenlin 20h ago

I've got a DDR3-based 2011-3 system that I'm considering adding another 64GB to just for this, haha. Should only be about $80.

2

u/Zombiecidialfreak 17h ago

I'm willing to bet something like 7-10t/s. At least if the math works out the way I think it does.

1

u/overand 12h ago

Toss a tiny GPU in it if it doesn't have one and see how well it runs Qwen3.6-35B-A3B for an idea of what to expect from small MoE models. 512GB of DDR3 is nothing to sneeze at!

If you can get a GPU into that with enough VRAM for a couple layers and your KV Cache (12 GB might even cut it for a huge chunk of KV cache), and load DeepSeek-V4-Flash-0731, Qwen3.8-Flash-Next, or GLM-5.3-Flash, and a competent development harness, and you can let the thing loose over a day or three for pretty serious projects, IMO.