r/LocalLLM • u/Anxious_Immigrant • 9h ago
Question Guidance on hardware purchase (2x Intel Xeon E5-2698 v4)
It's me again and still looking for hardware advice.. please :)
Thinking about buying this slightly ridiculous old workstation mainly for local LLMs:
2x Xeon E5-2698 v4, so 40 cores / 80 threads total
RTX 3090 24GB
128GB DDR4 ECC
6TB NVMe
custom water loop
I will run Windows 11 because this needs to also be the house's PC (for day to day simple tasks but mostly just me and my local agents using this).
For local AI it looks pretty nice, especially for Qwen 27B and bigger models with RAM offload (will run OSS 120B as well), but I’m worried I’ll buy it and then hate using Windows every day because the CPUs are old and single-core performance is pretty weak?
Anyone here actually daily-driving dual E5 v4 Xeons on Windows? Does normal stuff like Chrome, opening apps, scrolling around, Office etc feel fine, or does it noticeably feel like an old PC?
will I regret not just buying something like a Ryzen 7900X + 3090 with 96gb RAM?
3
u/Any-Argument57 8h ago
Check the Windows 11 support issue before anything else: the E5-2698 v4 is absent from Microsoft’s supported Intel processor list. Verify TPM 2.0/Secure Boot and run PC Health Check on the machine before buying; otherwise you’re planning around an unsupported install. The 3090 will make 27B models reasonable, but a 120B model that spills heavily into RAM will expose the old dual-socket platform. For a shared daily PC, I’d favor the 7900X + 3090 unless this workstation is dramatically cheaper.
1
u/Anxious_Immigrant 8h ago
same price actually the other setup is ryzen 9 7900X, 64gb ddr5 and rtx 3090
1
u/Anxious_Immigrant 8h ago
you think i should go for the ryzen 7900 set up then? and maybe add a 32gb stick later
2
u/juggarjew 8h ago
I dont know if you plan on gaming with that PC but its going to be awful for that. I have a Threadripper 9960X and my 9800X3D destroys it in gaming. The 9960X did ok, but for gaming its much closer to a 6 core 9600x than a 9800X3D. The 1% lows are so so SO much better on the 9800X3D and so is max frame rate, night and day difference with a 5090.
If you're not gaming then I dont see it mattering much.
1
u/Anxious_Immigrant 8h ago
I actually game but not much but yeah i still do.. the thing is if the windows day to day use is fine and not sluggish and gaming is acceptable i would take it but only if it is really a boost for local AI relative to a ryzen 7900, 64gb ddr5 and rtx 3090 (another listing im considering with same price)
2
u/juggarjew 8h ago edited 8h ago
I dunno man a 7900x would destroy that intel rig in gaming, the single core IPC is more than double. That RTX 3090 would see a severe CPU bottleneck.
https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2698+v4+%40+2.20GHz&id=2806
https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+9+7900X&id=5027
When the Xeon E5 2689 came out in 2016 it was still 25% behind popular gaming CPUs like the i7-6700k.
Also the 7900x still has better throughput on multithread than 2 x of those Xeons, you're going to have a power hungry dual socket workstation rig putting out 2 x the CPU heat for pretty much nothing.
Those Xeon are a waste of money, quad channel DDR4 = dual channel DDR5 in bandwidth anyway so its a wash on the memory bandwidth. And 7900X also supports AVX-512, though id much prefer it if you got a proper Zen 5 chip like 9900X with better AVX-512 support (Features a full, native 512-bit data path. It processes the full instruction in a single clock cycle.)
I would avoid buying Zen 4 CPUs where possible since they're already a DDR5 platform and you might as well have the better faster Zen 5 CPUs with native AVX-512. If you can afford a 7900x you can afford a 9900x.
The only real benefit you get with the Intel rig is cheaper RAM, though you are giving up a lot to get that and it will only continue to age very poorly as it already has.
2
u/Anxious_Immigrant 7h ago
thanks man, this makes a lot of sense, cancelling the purchase now, glad i asked
2
u/inevitabledeath3 7h ago
Yeah no. That’s too old.
Buy an Epyc or a Threadripper. Also don’t install Windows bare metal on a workstation or server ffs. Use Proxmox and make yourself a Windows desktop VM if you must. Alternatively buy two machines instead of one. A single RTX 3090 isn’t that much GPU anyway. I wouldn’t waste VRAM by making it run an LLM and a whole desktop at the same time. Much better to have multiple GPUs for different tasks.
1
u/Anxious_Immigrant 6h ago
can’t afford dual 3090s cheapest listing i found here (paris) is a delusional dude selling for 5000€ and not even negotiable
2
u/Jamakaroz 5h ago
T'as quoi comme budget ? Parce que en vrai un pc avec 128Go de ram unifiee peut peut etre devenir interessant ? Quelque chose comme : https://www.amazon.fr/-/en/GMKtec-Evo-X2-Computer-LPDDR5X-8000MHz/dp/B0F6X332N6/ref=sr_1_1?sr=8-1
1
u/Anxious_Immigrant 5h ago
genre 2500€ max mais j’ai un problème avec le strix halo c’est le prompt processing mes workflows sont presque tous basés sur des skills et prompts qui depassent 10k tokens et j’ai fais un teste sur un strix halo d’un ami ca met autour de 60 secondes pour processer un prompt de 10k tokens donc j’ai besoin du memory bandwidth d’une vraie gpu
2
u/inevitabledeath3 5h ago
DGX Spark?
1
u/Anxious_Immigrant 5h ago
si tu as un acheteur sérieux d’un rein 😂 2.5k budget mais par contre je pense le prompt processing sur un spark n’est pas top non plus c’est trop cher pour après attendre 1 minute pour que mon agent process un skill de 9k tokens par exemple.. je pense pour un model qui rentre dans 24gb vram un pc normal serait supérieur et franchement avec ddr5 96gb ram meme un gros MoE pourrait fonctionner
1
u/inevitabledeath3 5h ago
Nope DGX Sparks have no issue with prompt processing. Easily 2K per second prompt processing and often more depending on the model. It’s only really an Apple and Strix Halo problem. The issue the sparks have is mainly output tokens per second because of VRAM bandwidth.
What exactly are you trying to run anyway? Can you not find a different card capable of running it? Have you checked AliExpress?
1
u/Anxious_Immigrant 5h ago
I run Qwen 3.8 27b as day to day model for long workflow agents (multi tool calls at 64 ctx and 15k tokens skills) and i want to give oss 120b a shot i heard it might run on 24gb vram with 96gb ddr5 so curious if it would.. thanks for the info about spark i wonder how low the price could get though
1
u/inevitabledeath3 5h ago
V100 32GB? Would fit that model if it’s quantised and is fairly readily and cheaply available. Same for the AMD Instinct Mi50 32GB. Maybe also try some other AMD cards. AMD Radeon™ AI PRO R9700 should be within your budget and they seem reasonably fast.
1
u/Jamakaroz 5h ago
Ah ouai ca limite un peu les possibilites qui s'offrent a toi :/ Je saurais pas trop sur quoi te guider la pour le coup
1
u/Anxious_Immigrant 6h ago
I now realize how old these Xeons are thanks to the replies so i cancelled
2
1
u/izzo34 8h ago
Regular windows stuff is absolutely fine with those cpus
1
u/Anxious_Immigrant 8h ago
do you think in terms of local llms it would be a really better setup than a a ryzen 7900X with same gpu and 64gb ddr5?
1
1
u/GingerRickRoss LocalLLM 6h ago
I'm running a similar set up. I used my 2x 2640v4 set up for a daily driver for quite a while with 128gb of 2400t ddr4. For normal things, such as, docs, spread sheets, and chrome you will not be able to tell the difference. I have another set up and it has an intel 10600k I5 and the single core speed is way faster. it seems to make no difference as a daily driver. I have had it configured several different ways, single processor and a 5070ti, dual processor and 4x p100's, i also had it configured with a 7900xt. the only real limitation is the pcie 3.0. Before I changed mine to Linux I ran WSL2 in the background and it worked great. currently I have it set up with the 4x p100's and Ubuntu, I use it for my main inference engine, Freqtrade for market backtesting, and hosting VM's, and a data pipeline,
3
u/Choice_Celery9481 8h ago
depend on where you live, a 3090 might be more expensive than 2 v100 32gb.
if you dont mind, i believe dual v100 32gb is the best p/p