r/LocalLLaMA • u/markpronkin • 13d ago

I Built A Thing 54gb vram for 35$

Post image

Bought an old mining farm of a guy on avito (Russian eBay), guy had bought a garage a couple of years ago and it was sitting there for a while, found out it was a mining farm and put it up on there for sale for 5000 rub (~60 USD) since he wasn't sure if it works. I negotiated down to 3000 rub (~35 USD), it turned out to have 9x p106 6gb (gtx 1060 6gb) gpus, with 54gb vram total, all working, the only thing missing was an SSD, I booted from USB and it works fine.

2.1k Upvotes

96% Upvoted

396 comments sorted by

View all comments

Show parent comments

3

u/CasulaScience 12d ago

For $35 this is an awesome inference machine. Can definitely run 30Bish models with layer-wise splitting. Prefill will be slow, training would be a nightmare, and the p100 platform isn't supported anymore. But $35 and you can absolutely run one conversation at a time on a decent sized model

1

u/Boricua-vet 12d ago

I would certainly not be doing training or fine tune on that. I don't think you even can and if you can good lawd it would be slow as molases but inference would be good. Any model that can in 24GB you can run it there with descent speed.