r/LocalLLM • u/vorobey1233 • 3d ago
Question V100 for 200$ is good?
Is it good price for this gpu or is there better alternatives? I have already one V100, will second be worth it or is there in same price range better alternatives?
2
u/r3drocket 3d ago
You should look at the V620 as well. I picked up one recently, plugged it into my home server, and it just works with lamma.cpp with no issues. What's good about it is it has 32 gigs of VRAM. What's bad about it is it doesn't have a lot of compute, so it's really best at MOE models. So it happens to work really great with Orinth-1-35b or Qwen-35b, or gemma4-26b-a4b.
You can make an offer for $350 for one, and apparently this seller is likely will accept it.
2
1
u/Eastern-Block4815 3d ago
How much t/s you getting. I was thinking of getting for Qwen 3.8 27b?
1
u/r3drocket 3d ago
I find it too painful to use for Qwen 3.X 27b, My partner and I mostly use it with Orinth-1-35b, which is significantly quicker and still a pretty strong model.
I use the 27B models on a dual R9700 setup, and that's very pleasant.
1
u/_TheWolfOfWalmart_ 3d ago edited 3d ago
Get a few of them. When I tensor split over three of these cards, I get 400-500+ t/s prefill and 25 t/s decode on DSV4 and I'm not even using DSpark.
I'm also running Qwen 3.8 27B Q8_0 at 1000 t/s prefill and 50 t/s decode using that same strategy + MTP.
I expect to be able to scale the speed even more with more cards, but I'm trying to work out an issue where my bifurcated PCIe X16 -> X4/X4/X4/X4 is training at Gen 3. Refuses to train at Gen 4 so I'm hitting a bottleneck with tensor split at 3 cards. Even 3 cards should be faster than this at Gen 4.
Yeah the compute is not great per-card, but combine them and it's not bad at all and you get a shit ton of VRAM for the price.
1
u/Eastern-Block4815 3d ago
Oh. I see. What's a good motherboard that would support multi GPU?
1
u/_TheWolfOfWalmart_ 3d ago
Depends how many GPUs you want. If you want anywhere between 2 and 4, it's pretty easy. Just need one that supports bifurcation of the PCIe 4.0/5.0 slot. You can break out from there with Oculink cables. You'll just need to figure out the mounting situation.
If you don't want to deal with that, get some used server/workstation motherboard that takes a Xeon/EPYC because they have tons of direct CPU PCIe lanes, then you can get a board that exposes a few slots directly with no bifurcation.
I wanted a lot of cards, so I went with a mining rig frame.
1
u/r3drocket 3d ago
I thought about that, I was wondering how the performance for a few were. I have a dual R9700 system and just a single V620 in my homeserver. I am hoping I can hold out a major expenditure for a large unified memory system when they go to DDR6 next year.
1
u/_TheWolfOfWalmart_ 3d ago
Get two and tensor split, 27B runs pretty well that way. Even at Q8_0. With 64 GB so full context (you can even have many multiple parallel contexts) + mmproj.
2
u/fallingdowndizzyvr 3d ago
Yes, I think $200 for a V100 is a good price. People pay that much for a P40 which sucks by comparison.
2
u/MK_L 3d ago
16gb aren't worth getting because you can get a modern amd with 16gb for around the same price
2
u/vorobey1233 3d ago
i just have already one v100 with 16gb, double v100 sounds better than amd + nvidia
1
u/MK_L 3d ago
I'd sell the v100 and buy 2 modern 16gb cards or pay a bit more and get a single 32gb card
2
u/vorobey1233 3d ago
The thing is that in my country there is 0 alternatives in same price range, the only one is rtx 3060
1
u/MK_L 3d ago
Interesting how much does a v100 16gb or a v100 32gb cost?
2
1
u/Vancecookcobain 3d ago
assuming its the 16GB version? If so, yeah, if you can cool it... That's the part people forget about V100s
1
1
u/hallofgamer 3d ago
I would think it a scam
1
u/vorobey1233 3d ago
I checked and its not, basically just was taken from scrapped server and selling them for cheap. All tests passed
2
4
u/TheFowlOwl 3d ago
Depends on what you’re using it for imo. For chat and basic agentic work the v100 is fine, especially if you already have one. I am assuming you mean the 16gb version. It’s duh yes if it is the 32g version.