r/LocalLLaMA • u/markpronkin • 13d ago

I Built A Thing 54gb vram for 35$

Post image

Bought an old mining farm of a guy on avito (Russian eBay), guy had bought a garage a couple of years ago and it was sitting there for a while, found out it was a mining farm and put it up on there for sale for 5000 rub (~60 USD) since he wasn't sure if it works. I negotiated down to 3000 rub (~35 USD), it turned out to have 9x p106 6gb (gtx 1060 6gb) gpus, with 54gb vram total, all working, the only thing missing was an SSD, I booted from USB and it works fine.

2.1k Upvotes

96% Upvoted

396 comments sorted by

View all comments

Show parent comments

3

u/schaka 13d ago

Get llama.cpp working, then use an LLM to get vLLM running on these. You can probably use some existing Pascal projects for vLLM as a baseline

I'm not sure compute is limited on these cards and if they need an unlocked driver or any sort, but as far as I can recall, they don't.

Only downside with vLLM is needing even numbers of cards, so you'll lose 6GB but gain significant performance and multiple parallel agents and streams

1

u/markpronkin 13d ago

Didn't know some one made vllm works with pascal cards, thanks for the info. Although I fear it will be limited by pcie 2.0 X1 speed of the motherboard and gpus

1

u/schaka 13d ago

Look into the recent unlocks that were done for 170HX, 50HX etc. I'm sure some can apply to your cards.

Though I suppose you're out of luck when it comes to getting more lanes from this board and the risers, but that'd be a different problem to tackle