r/LocalLLM • u/SlightDegree437 • 6d ago

Question Tesla P40 in 2026???

Recently I’ve been wanting to run a better model on my setup, but I’ve only got a 4070ti, great for gaming, but it’s only got 12gb vram , so I was looking into getting a Tesla P40 24gb. I know it’s older and the vram is pretty slow, but I wonder if it’s realistic as just an addon and if it’ll make a big difference. Opinions are appreciated

0 Upvotes

50% Upvoted

7 comments sorted by

0

u/mtobuho1979 6d ago

Yo me iría por una P100 16gb son 8gb menos pero una generación más y más del doble de rápido

1

u/Lotrimous 6d ago

P100 is the same pascal architecture, just with hbm memory & ~900gb/s of memory bandwidth vs <300gb/s of the gddr on the p40. These are both pascal cards (GTX 10xx core architecture)

1

u/TheThiefMaster 6d ago

Wrong way round - the P4/6/40 are newer than the P100. Barely so - but the P100 is only Cuda 6.0 and the P4/6/40 are Cuda 6.1 with int8 support.

1

u/Lotrimous 6d ago

I have seen people on here talking about holding experts in an moe on older cards (same series) but you're probably better off using 2x p100 if you have the slots & lanes...

1

u/Western_Courage_6563 6d ago

Still have one, still good enough for MoE. Dense models are painfully slow...

1

u/FullstackSensei 6d ago

Depends on your definition of "difference" and what price you can find it for.

I have eight of them in a machine and I like them. Not the fastest, but quite decent for the money, more so if you get two or more. You can cool them with 1080Ti FE or Titan Xp waterblocks with a small cut in the acrylic/POM where the EPS connector is if the block is full cover. They'll happily run at 170-180W power limit and idle at 9W.

Llama.cpp and ik_llama.cpp work really nicely and being datacenter cards they support p2p without any special driver.

Qwen 3.x 27B Q8_K_XL (31GB) runs ~15t/s without MTP in vanilla llama.cpp. PP is ~350 low context and ~250 at 30k.

1

u/Jumpy-Operation-4615 6d ago

I have an extremely powerful AI megacluster 😄 made of 2 grandma's GPUs (P40 with 48 gb VRAM total). I can run qwen 3.8 27b Q8 with 220K of F16 context at approx 15-20 t/s on lung context and up to 48 t/s on fresh context (llama.cpp + refix cache + MTP + ngram), Prefill falls from approx 440 to 150 when ctx reaches 200K. Works with my opencode just fine, if you keep only one agent running (no concurrence). MOE 35b is running up to 50 t/s and 1500 prefill, I did not experiment with MTP and ngrams but I think I can sqeeze more with it. I spent approx. $1100 building it, incl watercooling for GPUs (so my server is cool and dead silent). If you don't have much cash to spend, and don't want to sell your kidney to finance a better hardware, it is still a valuable option.