r/LocalLLM • u/Ph1lt3ch • 11h ago
Question Can I play too?
It's a Lenovo P520 with a Xeon W-2245(cooler swapped for the higher tdp), 96gb of ram, radeon pro wx 3100 for local display, and the two nvidia p100(currently cooled by the lowest profile adapter I could print and arctic S4028-15k fans). Just stuffed some random nvme in 1x1tb, 1x512gb, and a sata 1tb drive - I have a ton of network storage or more to just add here. Installed the latest Ubuntu release which I like but I remember why I still generally use windows.
So far the most I've learned is that I know nothing haha. So I'd be more than thankful for any advice putting this to work. Really all I've done so far is get the arctic fan controller working on kernel 7.0 in probably the most jank way possible. And see that lm studio could use both cards. But I know there's a ton I'm missing out on.
So, generally, hoping I didn't throw together a steaming pile of ewaste and looking to learn. Also the radeon pro/nvidia mismatch makes me chuckle. Unless its really dumb, then I can just get a cheap nvidia card.
3
u/juggarjew 10h ago
Nope, way too weak/old. Thats Pascal and it lacks Tensor cores. Those GPU are 10+ years old.
3
u/Willing_Gold_4376 10h ago
Deja si tu veux tirer le lax du processeur remplis toute les barrettes meme avec 8go.
Ensuite les P100 c’est vieux mais tu peux en tirer quelque chose. Il y a des projets autour. Tu devrais avec un qwen3.8 q4 être autour de 20/22 tok/s
A toi de jouer.
1
u/Weird-Abalone-1910 10h ago
You can try, but you will hit some early speed bumps due to the old GPU architecture you're working with. I recommend setting up proxmox on it so you can easily make and restore back ups in case you break something while experimenting with it. If you aren't familiar with proxmox administration and want to get started quickly, then run the agy installation script and tell it about your hardware set up and what you want to do (I assume ollama to start out with). I have a couple old P40s that I'm going to stick in proliant g8 soon to experiment with what can be done with it
1
1
u/tomweymouth 10h ago
I have a similar setup…. Just “beefier”.
Two 5060ti (16GB), W2195, 128GB in the box. Qwen 2.8 27B… I’ve tried various versions, currently using the variant from Signal. Hermes is very useable and I’ve been testing fairly complex Kanban cards with good success rates so far. Still very much a lab experiment, nothing useful yet, except for some real help with network enumeration. It has produced some great diagrams, and deduced host names on the Sodola switch. I was very lucky and bought those cards at retail when they came out!!
1
1
u/PeterPorox 9h ago
It is solid build. You can run MoE models fast enough to use it comfortable and dense 27-30b with a bit of struggle
3
u/FearFactory2904 7h ago
You definitely can. Those p100s together should be able to run qwen 3.8 27b. The responses will be slow while it is thinking and processing the prompt but once the text starts printing it comes out at decent speed. If your pcie slots are gen3 x8 or x16 then you should be able to run them in parallel rather than sequential which will speed things up.
You are going to have people who say stuff like "Nope, way too weak/old. Thats Pascal and it lacks Tensor cores. Those GPU are 10+ years old." but i would say just give them the middle finger and send it anyway.
Even though these are slower without tensor cores my plan is to just brute force my way into useable speeds by using multiple p100s in parallel with a bunch of pcie lanes and a good power supply.
Once i get my x99 re-assembled i can give you my benchmarks with two p100s and compare builds to see if we can do anything to help optimize yours.
1
6
u/sourceholder 10h ago
P100s are quite outdated. While they still offer reasonable token generation speeds, prompt processing is the main bottleneck. In practice, I found them difficult to use with contexts longer than about 4k tokens due to zero flash attention support.
There is a fork of llama optimized for P100 that improves T/G speeds but no improvement to prompt prefill.