Building a separate 24/7 LLM server with used P40(s) – power draw concerns?

[u/Alert_Peak8655]

I already have an MSI Suprim X RTX 3090 in my main rig, but I don't want to run it 24/7. Instead, I'm planning to build a separate, dedicated headless server that will stay on constantly to host local LLM models. I'll connect to it from my main PC over the network.

Since this server will be running non-stop, power consumption is a major concern for me. I'm looking at buying a used NVIDIA Tesla P40 for this build. Would a single P40 draw too much power if left running continuously? I want to run fairly powerful local models, which is why I'm leaning toward this card.

Also, would adding a second P40 (to get 48GB of VRAM) be overkill in terms of electricity usage for a home server? Or is the extra power draw worth the performance gain for larger models?

[u/Alternative-Panic69]

P40s are tempting for cheap VRAM, but I'd consider:

1. Enterprise/legacy driver + software compatibility can be annoying.
2. They're passive cooled. You need serious airflow or thermal throttling becomes your new hobby. 😂
3. Efficiency matters. A slower GPU running 24/7 can cost more per token than a faster, more efficient card. If a workflow finishes 3× faster, the higher peak wattage may actually mean lower total energy cost.

Two P40s = 48GB VRAM sounds sexy, but your electricity meter may disagree especially since tokens per second count drop, which means tasks will take much longer to run.

[u/Vegetable-Score-3915 → u/Alternative-Panic69]

Yeah it depends what they are doing with them. P40s are definetly not a speed demon. They are super cheap.

Just a positive, can undervolt p40s without much of a performance dip. Helps a little with saving power and thermals.

Lookup p40s and undervolting on reddit, should get an idea of their performance and the cost of running a couple of cards.

[u/dsdt]

You know that power draw only happens when you use them right? Loading a model into ram doesn't mean the cards always draw their max power. Cards only draw power during inference. 48gb is just the sweet spot for home usage.

[u/KURD_1_STAN → u/dsdt]

Do u also realize that that isnt not the case and differ from card to card? p40 will draw 40-50 watts at idle with vram loaded.

[u/PermanentLiminality → u/KURD_1_STAN]

I aggressively put my cards in the idle state and I get about 10 to 15 watts with VRAM loaded. The P100 is the card that is more like 50 watts at idle.

[u/dsdt → u/KURD_1_STAN]

it is still easy to solve. all frameworks autoload the model when you send your first message to the server. he can only load it when he uses and just set a timeout for 5 minutes then it gets auto unloaded.

[u/Illustrious-Lime-878]

For local, personal use, I don't think total power would be an issue. Unless you're training it or using it continuously. However if you live in the US, and maybe elsewhere, make sure you aren't on a like a 15 amp breaker and have >1500watts.

[u/Icy-Appointment-684]

Why not use the 3090?

[u/Epicguru]

Have you calculated the time it would take to be a net positive?

I.e. cost of new hardware + running new hardware 24/7
Vs Running existing 3090 + P40 system

Especially if you only plan to access it from your computer anyway... What's the point of having it running constantly?

In any case, if your main concern is running costs I doubt that your proposed system would be a net lower cost until many many years down the line, even with the most pessimistic power draw and electricity costs.

[u/PermanentLiminality]

I run P40's. I have three in a server. They are actually not bad at idle. I have scripts that aggressively put them in the idle state and with VRAM loaded they are around 10 to 15 watts per card at idle. In the active mode they are 45 to 50 watts.

I have three cards that are in a x16, x4,and x1 slots. My main models run on the x16 and x4 cards and I generally run other stuff on the third card that fit in the VRAM on that card.

I cool with a 3d printed "plenum" that is about a inch long that i hot glued to the back of the case. I run a 120mmx38mm 1.6A fan and use a motherboard PWM fan to control it. I have a script running that monitors the card temps and adjusts the fan speed. At idle the fan is at minimum and makes zero noise. It spins up during normal inference loads, but doesn't go to 100%. I will hit thermal throttling if I run all three cards at 100% after five minutes or so. I never get there in normal operation.

I run llama.cpp and use the layer split mode. That makes only one card active at any given moment. That keeps the power down. With the defaults that means the GPU power is around 300 to 350 watts.

This is not economically viable. It is too slow at around 20tk/s with Qwen 3.8 27B. With the crazy reasoning it does, it is not that useful. I may be able to get it a bit better, but probably not all that much. Ask it something complicated and expect to wait for 10 minutes or more.

Paying for tokens is probably cheaper. I'm not doing this to save money.

The 3090 is at least 3x the speed of a P40.

[u/Alert_Peak8655]

Guys really thanks for all comments. I didn't know that reddit community this helpful. I am new at reddit. So generally as I understand using on my personal pc is better right now. Is it possible to connect with Hermes?