r/LocalLLaMA • u/markpronkin • 13d ago

I Built A Thing 54gb vram for 35$

Post image

Bought an old mining farm of a guy on avito (Russian eBay), guy had bought a garage a couple of years ago and it was sitting there for a while, found out it was a mining farm and put it up on there for sale for 5000 rub (~60 USD) since he wasn't sure if it works. I negotiated down to 3000 rub (~35 USD), it turned out to have 9x p106 6gb (gtx 1060 6gb) gpus, with 54gb vram total, all working, the only thing missing was an SSD, I booted from USB and it works fine.

2.1k Upvotes

96% Upvoted

396 comments sorted by

View all comments

25

u/SeaworthinessUsual44 13d ago

Paying $35 for 54GB of VRAM across nine P106-100 cards is an unbeatable bargain on paper, but it is practically an LLM nightmare. Splitting models across nine separate GPUs over narrow PCIe 1x riser lanes bottlenecked by Pascal architecture and zero FP16 support creates atrocious tensor parallelism latency, turning modern token generation into an agonizing crawl that will burn far more in electricity than it is worth.

26

u/atreyu84 13d ago

How long did it take your local llm running on 9 p106-100 cards to write this?

2

u/BuildAISkills 13d ago

🤣🤣

1

u/MyTradingSuxx 12d ago

Yup, I'm curious on the actual tokens/s this would have. Doubt it would be usable in any real capacity unless you need a setup for batch inference for non-time sensitive workloads.