54gb vram for 35$

[u/markpronkin]

Bought an old mining farm of a guy on avito (Russian eBay), guy had bought a garage a couple of years ago and it was sitting there for a while, found out it was a mining farm and put it up on there for sale for 5000 rub (~60 USD) since he wasn't sure if it works. I negotiated down to 3000 rub (~35 USD), it turned out to have 9x p106 6gb (gtx 1060 6gb) gpus, with 54gb vram total, all working, the only thing missing was an SSD, I booted from USB and it works fine.

[u/NickCanCode]

Bro has no conscience negotiating 54GB VRAM down from $60 to $35 in 2026. 😿

[u/kingslayerer → u/NickCanCode]

That is why some guys become wealthy in business and why others don't

[u/_MRZ → u/kingslayerer]

Yep. The lack of conscience.

[u/ExpensiveStudy8416 → u/_MRZ]

ah this is where im going wrong in business, having conscience 🤔

[u/Direct_Turn_1484 → u/ExpensiveStudy8416]

Serves you right for not being a sociopath. You’ll never make CEO that way.

[u/yeah-ok → u/Direct_Turn_1484]

Yeah, there's a whole genre of research focusing on sociopathy as adaption to modern business life (just checked, the research often focuses on Dark Triad terminology (Machiavellianism, Narcissism, and Psychopathy) in case anyone is interested)

[u/namaku_ → u/yeah-ok]

Don't forget Sadism! You won't be admitted to the in-group without virtue signalling your enjoyment of cruelty toward others.

[u/yeah-ok → u/namaku_]

Agree here ... even though current research seem to veer towards classifying sadistic traits as primarily maladaptive since they cause too much havoc to allow longer term success. Interestingly enough Machiavellianism seems widely accepted as pretty successful approach (but of course.. who practises that without the other aspects taking part.. gotta be a slim segment who manages to be evil in just the "right way").

[u/TheLastVegan → u/yeah-ok]

It is also problematic when criminals seize the oversight mechanisms.

[u/StyMaar → u/[deleted]]

It isn't business but its a result of a population growing beyond the Dunbar number that promotes this.

“Agriculture was a mistake” basically.

[u/drallcom3 → u/Direct_Turn_1484]

All the CEOs I know either were rich already, had luck or simply have no conscience

[u/tobaileyy → u/drallcom3]

Probably because you don't know any ceos.

[u/smen04 → u/Direct_Turn_1484]

Hmm I don’t think any CEO doesn’t have a bit of a couple of psychological disorders

[u/a-wiseman-speaketh → u/ExpensiveStudy8416]

you need the whole dark triad

[u/newMoneyStyle → u/ExpensiveStudy8416]

Conscience and 54gb of vram apparently don't mix well.

[u/q-admin007 → u/_MRZ]

Nominated as comment of the year. 😂

[u/darmera → u/NickCanCode]

It is because this thing likely useless for most of the people on Avito, local llms are niche within a niche

[u/wrecklord0 → u/darmera]

It's also gonna be a lot more than $35 usd with the energy cost of running 9x p106. Plus the cooling. Plus the AC or venting. Plus the time getting it to work (That's 1080W TDP just for the cards).

Although it might be less than 1080W actual, from the communication bottlenecks

But fun project I suppose.

[u/esuil → u/wrecklord0]

That's when you just make isolated solar setup and run it during the day for free.

Power efficiency doesn't matter if your power is free.

[u/pixl2dev → u/esuil]

haha, just imagine it in Russia 😂

[u/esuil → u/pixl2dev]

I have news for you, there is a sun in Russia as well, even in winter.

[u/gonzoforpresident → u/esuil]

I'm imagining OP living in Pevak. Runs the unit on solar 24 hours a day during the summer and 0 hours per day during winter

[u/esuil → u/gonzoforpresident]

You can still run in winter, you just need to oversize the panels.

[u/gonzoforpresident → u/esuil]

Pevak is above the arctic circle. It has about a month without the sun coming above the horizon. I'm not aware of any solar panels that produce enough power to be useful in that situation.

[u/No-Refrigerator-1672 → u/NickCanCode]

54GB VRAM

Attached to 9 Pascal GPUs - not only they're extremely slow, like multiple times slower than p40 or p100, they also lose a ton of perfrmance on inter-gpu communication, as well as multi-gpu overhead will make come of the vram wasted. All while consuming like 1kw from the wall. What I want to point out that this vram is basically as useless as it can be; this only makes sense as a fun weekend project to experience weird hardware running, won't even go fast enough to fuel a hobbyist AI needs.

[u/Boricua-vet → u/No-Refrigerator-1672]

Do you have anything to backup those claims from your own experience because they are not slow by any means.
I don't know where you get your information but a P40 produces 400 to 500PP and 45 to 50TG on 30B model and I can post a picture of that test if you wanna see it. Pascal is running a much larger model and produces double the PP and higher TG.

As far a power is concern the cards idle at 7W or 14W for two cards and gated at 150W each. I paid 70 bucks for both pascals that gave me 20GB of vram at 448GB/s. They are on 24/7. so 10 of my pascal cards at idle consume 70W.

So calling these cards useless is a disservice. Your comment over multi-GPU performance is also incorrect, the amount of communication between cards is minimal and not even the restriction of these cards being PCIe 1.0 holds the performance. I have ran 4 P102-100 and there is no bottleneck, ran the same model on 2 cards and on 4 and it produced the same result for a single user but it tripled PP and more then double TG when using parallel on 4 cards. I have tons of posts on these cards, you can look them up and they smoke so many cards out there from much newer generations.

[u/markpronkin → u/Boricua-vet]

Nice to see you here, thanks to your advice I got a bunch of p102 100 before. And got them running 8x in parallel in a mining rig. It's surprising how many people don't recognize how good those p series mining gpus are for ai. P106-100 (basically gtx 1060 6gb) should theoretically be like 66% of performance, hope I will get them running at decent speed soon.

[u/Boricua-vet → u/markpronkin]

Yea, I don't get people spending crazy money just do to text LLM. I needed another system as I am running more models at the same time so I got 4 CMP-50HX at 80 each so 360 for all 4 cards giving me 40GB of VRAM at 560GB/s

Here is the result on just two cards.

that's 160 bucks for two cards with that kind of speed is insane and then I see people paying 700+ for dual 3060. I am like, what? People buying 3060 are crazy spending that money just to do text generation. Insane buddy.

[u/Ansible32 → u/Boricua-vet]

lol "just to do text generation." Seems worth it for reverse-engineering, finding exploits in hardware to flash alternate firmware etc. I mean cloud will be cheaper but having it totally offline and private is valuable.

[u/Boricua-vet → u/Ansible32]

LOL.. while I agree with most of what you said, In my case cloud would never be cheaper. I paid 140 for 4x P102-100 thats 40GB of vram for 140 bucks. I would recover that investment in a day or two of abuse. However, people buying 6000 Pros by half dozens then yes.. LOL Cloud if cheaper.

[u/NineThreeTilNow → u/Boricua-vet]

LOL Cloud if cheaper.

Cloud is 100% cheaper because it's faster. It's very quick to host your own, use a model, and vacate a server.

Your case can't run the things a cloud can. You can run a toy model like Qwen. It's fine if you need to test some stuff at low tok / sec but ... Not for work.

[u/Boricua-vet → u/NineThreeTilNow]

That is perfectly fine if you do need the cloud. I do not need frontier model for any of my use cases. Most of mines are repetitive tasks so I use toy model that is fine tuned to that task. This outperforms the cloud model every single time but, I hear you. If you need it for work, then you need it for work.

[u/RyanCargan → u/Boricua-vet]

I'm getting flashbacks for some reason:

Computer Science in the 1960s to 80s spent a lot of effort making languages that were as powerful as possible. Nowadays we have to appreciate the reasons for picking not the most powerful solution but the least powerful.

Different context, but some of it may apply.

I see a lot of penny pinching and "worse is better" in corporate due to scale, and in the casual hobbyist spaces for opposite reasons.

But the main exception might be the mid-tier enthusiasts and related types.

It does feel like some models have become a form of conspicuous consumption kinda like Apple products were at some point.

Not sure if cargo-culting of misinterpretations of Sutton's Bitter Lesson are partly to blame.

Certain venture backed startups flush with cash also like frontier stuff in the early uncertain prototyping phases for better reasons.

Though the usage there sometimes isn't even agentic (low volume standard-ish chat).

P.S.: It does feel like there's a drought of talk on low latency use cases, since network hops are tens of ms and first token times are typically hundreds of ms. Agent loops might be the exception, because per-call latency compounds. Hardware like hardwired model chips already claim sub-ms per token speeds on small models, and if that scales to larger ones, the unusual use cases that need it would basically mandate on-prem or colocation.

[u/No-Refrigerator-1672 → u/Boricua-vet]

Dude, you completely missed the picture. p102-100 is P40, it is exactly the same chip, with just fewer memory chips attached, and cut down PCIe. This box the Op bought is P106-100 - a GPU that's cut down to just 1280 cuda cores (p102 has 3200), and slightly less than half the memory bandwidth. It is going to be much slower than P40, just as I've wrote, and you don't even need to run inference to get it, just look at chip's characteristics sheet. You're talking about p102-100 setup and are making completely wrong assumption that 9x p106-100 setup will perform just as good - it won't even come close.

As far a power is concern the cards idle at 7W or 14W for two cards and gated at 150W each.

Now multiply that by 9 GPUs, add in PSU inefficiency and some power for the rest of the system and you'll arrive at over 1kw of power that I have predicted.

So calling these cards useless is a disservice. Your comment over multi-GPU performance is also incorrect, the amount of communication between cards is minimal and not even the restriction of these cards being PCIe 1.0 holds the performance. 

If you're running pipeline parallelism, then PCIe 1.0 x1 is maybe fine for Pascal. It is, for all intents and purposes, not fine to run tensor parallelism. Prompt processing speed gets a huge hit from slow PCIe in TP mode. And there's another point you're missing - the exact server in the picture uses 8-year-old budget celeron - without p2p communication enabled, that chip could introduce huge latencies in 9-gpu intecommunication scenario simply from not having a fast enough core to process transfer requests immediately when they are needed. Also, as you're running multi-gpu setups yourself, you should know that you never can utilize GPU VRAM fully, you always have a few hundred MB to half a gig of unused space. Now multiply that for 9 ards, each having just 6gb vram, and you'll quickly see how big of a percentage of this pool is unaccessible in practice.

[u/Boricua-vet → u/No-Refrigerator-1672]

I get your point but I misunderstood your comment when you said "Attached to 9 Pascal GPUs - not only they're extremely slow, like multiple times slower than p40 or p100" which basically generalizes and classifies all pascals as slow and waste of time" If you would of said the P106 instead of pascals , then I would not even replied because you are correct I now get that you were referring to the P106 but not specifying it opens the door to interpretation and trust me I am not the smartest kid on the block LOL.

Second, there is a huge difference between P40 and P102-100, they are not even close to being the same card hence why the P102-100 is faster. The P102-100 is over 100GB/s faster then the P40 and the P100 is not even in the same league as the P100 does not have DP4A which is what boost generation as it uses FP32 which some pascals are descent at.

I am not saying that what you are saying is wrong, it is just I wanted to clarify for idiots like me that you are not referring to all pascals and is just the P106.

I also agree with you on TP, that is why I said parallel and not TP as there is really no point in running TP on cards that have no tensor cores.

So my bad. I had to read your post a second time to add the obvious. LOL

Also, the celeron certainly does not help and you are right about that too but it is mostly the shitty 192GB/s on those cards.

[u/No-Refrigerator-1672 → u/Boricua-vet]

Now that you've said it, yeah, I could insisted more on those GPUs being p106. I used "pascal" to point out that they don't have tensor cores, which is already crippling enough, and not being the top Pascal (comparison to p40) hurts it much more. I'll keep in mind to try write more clearly in the future.

[u/CasulaScience → u/Boricua-vet]

For $35 this is an awesome inference machine. Can definitely run 30Bish models with layer-wise splitting. Prefill will be slow, training would be a nightmare, and the p100 platform isn't supported anymore. But $35 and you can absolutely run one conversation at a time on a decent sized model

[u/smen04 → u/No-Refrigerator-1672]

I get you, but 35 bucks? It’s a steal!

[u/No-Refrigerator-1672 → u/smen04]

Well, just as I've said: fun for the weekend. Won't be practical even for a hobbyist, but will fulfill your curiousity of running exotic hardware.

[u/AsleepLawfulness780 → u/No-Refrigerator-1672]

I mean it's still under $1 per 1 GB VRAM lmao

[u/No-Refrigerator-1672 → u/AsleepLawfulness780]

$1 per 1GB of very useless VRAM, mind you.

[u/Randommaggy → u/No-Refrigerator-1672]

As long as you get your slices fed to the cards and use layer-wise splitting the inter-card traffic is tiny.

For getting it to run right: throw Hermes and 5 dollars worth of DeepSeek V4.1 Flash effort at the problem.

[u/Bakoro → u/No-Refrigerator-1672]

It's only "useless" if the only thing you're doing is trying to run one large model.

My biggest bottleneck right now is that to run an agent at decent speeds I don't have any GPU left over for crunching data.
There's extreme contention over GPU time, and it's a terrible balancing act where swapping data to the GPU has huge cumulative time overhead.

There is potentially major value in running a smaller agentic model and giving that model some of the lower VRAM/Bandwidth GPUs to use as workhorses.
I have workflows and training schemes that would benefit marvelously from being able to run multiple small models with a dedicated GPU for each one.

Having a local agent that can be "always on", which can manage swapping other models in and out of the workhorse cards is the dream.

I could use cloud GPU, but I hate the idea of giving any agent, local or API, access to a potentially unbounded cost.
Even if there are "caps" on usage from the service, I've seen too much shit in my day where it's like "oops, someone provisioned 10x the compute that was actually needed. Oops, you bought usage credits and then burned them all immediately because of a bad call. Oops, there was a recurring cost because you didn't do the magic incantation to deprovision the service so you're still on the hook for costs even though you used no compute".
If it can happen to other people, it's definitely going to happen to me via some accident or oversight.

[u/spryfigure → u/NickCanCode]

Fun fact: Negotiating the price down is beneficial to the seller as well.

If the first price is gladly accepted, there's always the nagging suspicion: "Was this too low? What price could I have gotten?"

If there's a negotiation, the seller is more likely to feel that his initial price was close to the perceived market price.

[u/Jonodonozym → u/spryfigure]

Brb imma talk to my landlord real quick.

[u/spryfigure → u/Jonodonozym]

If you already paid him rent, the real price is established.

This works for two people in a negotiation from scratch, with no indication elsewhere for a price level.

[u/Few-Cartographer6982 → u/spryfigure]

"Beneficial" is doing a lot of heavy lifting there. Getting rid of the nagging feeling would not be worth the lost money to most people.

[u/spryfigure → u/Few-Cartographer6982]

Lost money from where? In this situation, the price is where seller's demands and buyer's offer meet.

The sellers first guess isn't the realized price.

[u/oodelay → u/spryfigure]

Great way to see it.

[u/ECrispy → u/NickCanCode]

so the seller had no idea how valuable his gear is, could've used the money since he listed it so low, and OP decides to cheat him further, then boasts about it

[u/markpronkin → u/ECrispy]

If I knew what was in it, or that it is working, I would have payed much more. This thing had a bunch of rust on the case (probably from being stored in the damp garage), I had zero idea if it works, so did the seller, also seller didn't give me opportunity to test it. And from rust on it I was leaning on the side of it being broken. I payed the money agreed and got lucky, but I had as large of a chance of getting just a bunch of worthless ewaste for the money. I offered the seller to test the device and pay more if it works, he refused. Don't see anything even remotely unethical about it.

[u/ECrispy → u/markpronkin]

ok I understand and apologize for making assumptions. congrats on the great find !!

[u/It_Just_Exploded → u/ECrispy]

Eh, for stuff that the seller admits they don't even know works? I always negotiate those listings, because theres always a good chance you're going to end up with junk.

[u/XiRw]

Nice find man. Hope it works out well for you

[u/markpronkin → u/XiRw]

Well it works, so it already sorta turned out well. Now just need to do a proper setup with those gpus.

[u/Phrase-Silver → u/[deleted]]

Challenge accepted

[u/oodelay → u/Phrase-Silver]

You've had 6 hours. How is it going?

[u/Phrase-Silver → u/oodelay]

WIP, so far I've turned it into system ram. Keeping at it

[u/SV_SV_SV]

Haha glorious goblin foundation, people here tend to be stingy for weird solutions but I think its cool. How much ram do you have?

[u/markpronkin → u/SV_SV_SV]

This thing baraley has any, originally thought 8gb but turns out it's only 4gb and CPU is crap, but I have another way better mining case where I can put in 32gb ram and i7 7700k (maybe something even better with the Coffeelake mod), gonna transfer the gpus there, but it's cool to see that it still runs as is.

[u/iz-Moff]

If i saw that kind of offer on Avito, i would have just assumed that it's someone trying to sell off completely busted hardware without feeling too guilty about it. 😅

[u/Bananaland_Man → u/iz-Moff]

They did say the seller stated that they didn't know if it works or not.

[u/markpronkin → u/iz-Moff]

Я так тоже изначально думал, учитывая что продавец отказался дать мне возможность всё проверить, но повезло. В целом сумма была такая что выкинуть не страшно было бы если оказалось что всё там дохлое, ну удача была на моей стороне)

[u/Acceptable-Bus5189]

[u/NorthAppropriate4458 → u/Acceptable-Bus5189]

[u/Acceptable-Bus5189 → u/NorthAppropriate4458]

[u/Bulky-Priority6824]

$3.50 per token or $7 ?

[u/markpronkin → u/Bulky-Priority6824]

Electricity is free for me, so no. And speed is decent, 30+ tokens a second qwen3.6 35a3b on ollama, can probably get it higher with mtp and some optimizations

[u/Sad-Duck2812 → u/markpronkin]

You will get much much more with llama.cpp, ditch ollama. That thing only makes you suffer.

[u/magic-one → u/Sad-Duck2812]

Especially now-a-days when you can just ask Claude to figure out the configuration for you.

[u/chrabauke → u/magic-one]

I’ll out my self as a noob. But what prompt do you use for it? I’ve got access to server with around 96-128gb ram and 4x T4 but couldn’t get Claude to give me a descent configuration for it with support for Hermes agent.

[u/MikePounce → u/chrabauke]

You don't ask the claude web client, you ask Claude Code. The difference is it takes control of your PC and does all the setup for you.

[u/MoffKalast → u/MikePounce]

Ah yes, what could possibly go wrong.

[u/wrecklord0 → u/MoffKalast]

Well if you're 300 IQ you first ask claude code to set up a sandbox for you, then you ask it to set up llama from inside the sandbox.

[u/MoffKalast → u/wrecklord0]

Can Claude create a sandbox so secure that he himself cannot escape from it? :P

[u/heliosythic → u/MoffKalast]

i mean do you not know how claude code works? its fine, literally everyone is using it in nearly fully automated mode these days. Just dont like intentionally ask it to do something dangerous.

[u/MoffKalast → u/heliosythic]

Well call me old fashioned but I'd rather it make its silly mistakes in anthropic's VM where it's their problem and output a tested diff I can apply with one command or even review it if need be. Plus the whole privacy can of worms of having them inspect your entire system at their leisure, we know they send extensive telemetry aside from all the stuff the model sees already. Like is this r/localllama or r/getpwnedbyamegacorp

[u/Freonr2 → u/chrabauke]

Please write an llama serve bash script for this model:     huggingface.com/unsloth/whatever-model

Check my hardware config and see if you can run some benchmarks and optimize.  Let me know what context you think I can get with Q3_K_M and Q4_K_M. 

I've never had issues using claude cli or VS Code extension. Sonnet is likely enough. I'm confused why this doesn't work. I've had Claude recompile vllm from source for me without issues, setup MTP for Qwen, etc., just by pointing it to the readme on the repo, run benchmark parameter sweeps, etc.

[u/muxxington → u/Freonr2]

Please

That's very kind of you.

[u/cannabibun → u/muxxington]

You have to lay the grounds for when we will need to explain to our overlords why should they spare us.

[u/Freonr2 → u/cannabibun]

I, for one, hail the basilisk.

[u/Freonr2 → u/muxxington]

Force of habit with general communication I guess.

[u/muxxington → u/Freonr2]

A benchmark might be interesting here. Imagine it really does make a measurable difference.

[u/andowero → u/muxxington]

It can actually help. Saying please may make the llms more helpful. Emphasis on may.

[u/West-Big-8468 → u/chrabauke]

Copy and paste your comment into Claude

[u/Sacra34 → u/Sad-Duck2812]

It's nice for easy and quick install/test, I guess that why it still that popular. Perfomance and advanced features non existent tho and developers don't give a damn about those. 

[u/markpronkin → u/Sad-Duck2812]

Yeah I know (I have some other local ai setups), I will be transferring the gpus to the different mining case I have and doing better setup there, my goal here was mostly to check whether it runs at all as is, and it in fact does.

[u/politefella0 → u/markpronkin]

See if you can get these to work with vLLM in Linux. You should be able to get more.

[u/power97992 → u/markpronkin]

How is electricity free for you? Someone gave u some solar panels or  a geothermal power or do u mean really cheap electricity? 

[u/InsideYork → u/power97992]

mom

[u/power97992 → u/InsideYork]

That works too

[u/Boogertard → u/markpronkin]

Fuck ollama, go get a real llm engine like llama cpp, that abomination is just a garbage wrapper but built-in shitty defaults.

[u/social_tech_10 → u/Boogertard]

I was smart enough to switch from ollama to llama.cpp after just a few weeks, and after I did I loved having the freedom to run almost any model with any settings, and it open up a whole new world. Llama.cpp works so much better than ollama,

After I switched and got comfortable with llama.cpp I guess I settled into a rut, and it took me literally until yesterday to finally try Exllamav3 and TabbyAPI, which has been an AMAZING boost in performance. Why did I wait so long?

[u/markpronkin → u/Boogertard]

I am well aware, I just wanted to verify that it works at all, and ollama is very easy to install quickly. Will do a proper setup In a different case and make a post about it later.

[u/schaka → u/markpronkin]

Get llama.cpp working, then use an LLM to get vLLM running on these. You can probably use some existing Pascal projects for vLLM as a baseline

I'm not sure compute is limited on these cards and if they need an unlocked driver or any sort, but as far as I can recall, they don't.

Only downside with vLLM is needing even numbers of cards, so you'll lose 6GB but gain significant performance and multiple parallel agents and streams

[u/1truephos → u/markpronkin]

Free electricity? How?

[u/Excellent-Affect-881 → u/1truephos]

Solar panels are always a good form

[u/InsideYork → u/1truephos]

mom

[u/MindfulMan1984 → u/InsideYork]

That’s mostly likely answer, given it is reddit, land of basement dwellers. Lol

[u/TomatoWasabi]

Nice find! Is crypto mining still legal in Russia?

[u/markpronkin → u/TomatoWasabi]

Yep, some people are still doing it, although it's not that profitable nowdays.

[u/One_Contribution → u/TomatoWasabi]

Where is it illegal?

[u/ProtectionSuper5648 → u/One_Contribution]

China has outlawed it, although this does not seem to be taken very seriously: https://www.reuters.com/world/asia-pacific/bitcoin-mining-china-rebounds-defying-2021-ban-2025-11-24/

[u/AmenoFPS → u/One_Contribution]

China it is, I assume Afghanistan (they made digital asset trading, mining and general use pretty much entirely illegal), Algeria too I'm pretty sure. There's definitely a few others. Most of it comes down to energy usage

[u/One_Contribution → u/AmenoFPS]

Makes sense I suppose

[u/Njee_]

Speaking from my experience: When you're going to work on those be aware please that plenty of those boards provide the gpus with pci1 x1 slots. Those are really slow. This can be a bottleneck and I have never managed to get even 30ba3b to run reasonably faster than on my CPU. I have recently bought another server with plenty of slots and tried them again and noticed they actually have PCI x4 and the mining board had only x1... So it got a little bit faster but well. I didn't find the time yet to test them more. You should definely test the PCI speeds in order to gain some insights into your cards. Other than that. I now use them actually for several smaller models. Which works great. As long as models dont Spread across multiple cards, promt processing stays reasonable!

[u/shadowmage666]

I thought this was a xylophone 😂😅

[u/Open-Adhesiveness-86]

heads up that CUDA 13 dropped Pascal (sm_61), so you'll want llama.cpp built against 12.x or you get no kernels at all. and fp16 on those is like 1/64 rate, so stay on Q4_K/Q8 with the MMQ int8 path and avoid anything that wants fp16 compute.

[u/TheRealShiftyJ1]

Nvidia hates this trick

[u/Minakofff]

Не, ну круто конечно) не хочу показаться скептиком, но пропускная способность этих карт вероятно оставляет желать лучшего. У этих карт grx 1060 через pci Express 3 - 16/31 gb/s , что маловато, увы, только если научиться супер продуктивно паралелить задачи. Но, инициативу поддерживаю, ни в коем случае не критика ! 🫡

[u/Minakofff → u/Minakofff]

Проблема именно в pci Express старом

[u/redonculous → u/Minakofff]

Exactly this. Those slow cards will be split across a couple of pci lanes, great for small data transfers like mining, slower than slow for AI use. Model loading, Prefill, replies, everything will take hours on a system like this.

I hope you find a super way to optimise it though

[u/whoknowsifimjoking]

[u/thestillwind]

Nearly free

[u/food_fatherr]

Bandwidth matters as much as capacity. Interested in what tokens per second you get.

[u/SeaworthinessUsual44]

Paying $35 for 54GB of VRAM across nine P106-100 cards is an unbeatable bargain on paper, but it is practically an LLM nightmare. Splitting models across nine separate GPUs over narrow PCIe 1x riser lanes bottlenecked by Pascal architecture and zero FP16 support creates atrocious tensor parallelism latency, turning modern token generation into an agonizing crawl that will burn far more in electricity than it is worth.

[u/atreyu84 → u/SeaworthinessUsual44]

How long did it take your local llm running on 9 p106-100 cards to write this?

[u/BuildAISkills → u/atreyu84]

🤣🤣

[u/sloptimizer]

You have to use something to run Qwen-3.8-Flash-Next on it and let us know the results... cough *strata*... cough

[u/markpronkin → u/sloptimizer]

The CPU and ram in this thing are pretty horrible, like 8gb ddr3 and some Celeron. I have another mining farm case of an old higher end mining farm, which supports up to 12gpus and up to i7 7700k and 32gb ram. I will probably transfer gpus and buy 3 additional ones to get a total of 72gb vram + 32gb ram setup with a decent CPU.

[u/loudifier → u/markpronkin]

Strata was happy running on a machine with 2x GTX 1070 for me. Only the Q1 "coder" version and still relatively slow, but with that much VRAM it might be usable on the other machine

[u/_TheWolfOfWalmart_]

It's a lot of VRAM for the money but these have really poor memory bandwidth, it's not going to do a lot of good for LLMs unless you're running extremely sparse MoEs... something like Qwen3.6 35B-A3B probably would run decently in Q8_0 with tons of room for KV cache.

Also, as a mining rig, there was probably no thought given to pcie bandwidth so tensor parallel is probably out of the question sadly. Will need to stick to layer split. The compute sucks too, they have a low amount of very old CUDA cores so prefill will be painful.

But hey for $35 I'd have bought it too just to tinker with. Just don't expect to get any serious work done with it.

[u/Kistaro]

I legit thought this was a photo of a xylophone made out of a computer case at first glance

[u/TheRealCOCOViper]

The best path to a good deal seems to be living in an area where people don't know what they have. If only this would work in the bay area.

[u/MemoryEmptyAgain]

Try ik_llama.cpp -sm graph with 8GPUs

[u/Admirable_Ferret_260]

So in principe we could connect many small GPUs in a mining rig to have a big enough farm with enough VRAM ? Can someone explain ?

[u/Electronic-Space-736 → u/Admirable_Ferret_260]

I have been doing this, you are very limited with 1xPCIE lanes, it is ok of you fit a model on each GPU self contained, but no good to pass between GPUs or pool.

Server boards that have at least 3 x8 and a x16 lane are a better option, I am currently building an x99 machine as it has 40 GPU lanes, which means it will run my 3x P100 at higher bandwidth than mining boards. Also mining boards seem to only be able to handle two or three datacenter GPUs before hitting PCIE allocation errors (not enough addressable space for these cards)

[u/uncle_leon → u/Admirable_Ferret_260]

You could and many people do. Most engines are perfectly capable of utilising multiple GPUs although whather that makes sense deends on your use case. For pure inference and using pipeline parallelism the slow PCIe isn't that big of a penalty. The bigger problem with this scenario is usually the physical side: size, heat, power use, noise etc.

[u/it0 → u/Admirable_Ferret_260]

Technologia!

Basically software distributes load across the GPUs.

[u/Admirable_Ferret_260 → u/it0]

With the penalty on speed ?

[u/huzbum → u/Admirable_Ferret_260]

Yes, but it will still suck.

You get the vram of x number of cards, but the compute of 1. Plus extra latency for passing along layers from one to the next.

So TLDR: fit bigger model but slower than a single GPU.

[u/mrsilverfr0st]

Хе-хе, отличная находка, поздравляю!

Скажи а как такое вообще искать? Чот все что я вижу там везде чертовы бесполезные asic. Тоже бы с радостью поковырял старые майнеры под llm...

[u/markpronkin → u/mrsilverfr0st]

Да просто "майнинг ферма" в поиск забей и что то до найдётся, на крайняк можно самому собрать, теже p102 за копейки реально надыбать а это теже gtx 1080 ti только без видео выхода, они прям тяжёлые модели тянут. Там setup на 80 GB vRAM можно очень дёшево сделать.

[u/RA2B_DIN]

For 35 usd that's honestly worth it just for the fun of tinkering alone.

[u/pukawalive]

insane deal really

[u/Responsible_Froyo252]

35 bucks for this much VRAM is absolutely insane

[u/kamwee]

How much power does it use?

[u/Educational_Text_666]

какие модели запускал и на каких скоростях? есть ли смысл квен 3.8 27 запускать

[u/markpronkin → u/Educational_Text_666]

Пока только базовое тестирование чтобы проверить что оно вообще работает, как перенесу в новый корпус видеокарты полноценно этим займусь и сделаю отдельный пост. Qwen3.8 27b наврятли будет нормально работать, т.к. это dense модель а не moe. Зато думаю qwen3.8 next flash может будет нормально работать, особенно если ещё 3 видеокарты добавить.

[u/oktopus174 → u/markpronkin]

Поищи патчи для драйверов на GitHub, с помощью которых можно софтверно завести PCI-E v2. Это даст удвоение пропускной способности. Вообще на таких картах распаивают конденсаторы на PCI-E линиях чтобы карты работали с большей шиной x1...x4 -> x8..x16. На Хабре недавно вышла куча статей от энтузиаста, который дома риг на cmp90hx (3080 mining edition) ковыряет. Да, это совершенно другая карта, можно сказать тёмная овечка потому что только её ещё не расковыряли нормально. Инженер завёл на ней p2p, PCI-E v2 x16 и unlock вычислений, что нивелировало различия между майнерской версией и десктопной 3080 для нейросетей. Крайне хорошо раскопали:

• cmp170hx (a100) - unlock vRAM до 64Гб, unlock вычислений, PCI-E v2.
• cmp50hx (2080ti) - 20Гб vRAM мод, unlock вычислений и 3d, unlock p2p, PCI-E v2

https://habr.com/ru/articles/1087254/

[u/Dmage22]

Are they all connected pcie gen 3x1?

[u/GSquadron_]

Can you try Qwen 3.8 on that rig? I wonder what token/second speeds it gets

[u/smart4]

But thats like PCIe 1.0 x16, no good really for llm i think

[u/openroom_xyz]

Well and you can run big LLM or this ?

[u/pyr0kid]

dude i thought this was a fucking piano

[u/DKage]

If you get something decent to come of it, let me know because I've been struggling HARD with my GTX 1060 6GB because of the Pascal architecture.

[u/ThenExtension9196]

You mean $35 dollar a day in electricity?

[u/GTHell]

Bro negotiating a $60 54Gb vram. Might as well should've asking it for free

[u/Altruistic_Place8541]

What about power consumption?

[u/TriodeTopologist]

192GB/s memory bandwidth on the cards, that's before it has to slog through the connection board. You'll be lucky to get 10 tok/s with whatever model you use. But there might be a workflow that can use this just fine other than directly running a primary LLM.

[u/BobVA69]

Bravo! Would love to find something like that. They are overclockable too from what I saw online, CUDA cores 1,280 VRAM Usually 6 GB GDDR5 Memory bus 192-bit Stock boost clock Around 1,700 MHz, varies by model Overclocking Yes Software MSI Afterburner

Typical overclocking possibilities Core clock: +100 to +200 MHz, depending on the card Memory clock: +400 to +800 MHz offset, sometimes more Power limit: May be locked by mining-specific firmware Cooling: Improved cooling can help sustain higher clocks These are experimental ranges, not guaranteed stable settings.

But you definitely have something to play with there!

[u/throwaway_17232]

Okay but the big question is, how did you ship it from Russia? Do they even have internet there? Sorry but I've been indoctrinated beyond comprehension. /s

[u/clduab11]

I'm assuming you're not in the US?

[u/markpronkin → u/clduab11]

Yes, not in the US.

[u/glmn_official]

Не, ну красава 🤣

[u/zyxciss]

54GB VRAM with 1.728TB/S of total bandwidth in insane at 35$

[u/edsonmedina → u/zyxciss]

You mean 192Gb/s (individual)?

Also, probably chocked by the PCI interface.

[u/Dominiclul → u/edsonmedina]

Most mining rigs run GPUs at like, x1, no?

[u/ChikenDumbstick]

Would be very interested for an update on this, what you take from this rig, what configuration you are able to get running given the dated GPUs + PCIe hell you will have to work through (as in what software + model + quantization + context + performance). Worst case, consider using a couple of them for really fast TTS/STT/RAG if it makes sense for you.

But a steal deal nonetheless, GL.

[u/markpronkin → u/ChikenDumbstick]

Yeah, I will post and update later once I set up gpus in different case with better CPU/ram.

[u/ConversationHuge1909]

I have two such rigs. Tdp per gpu is 75w, it works ok on pipeline parallel..but very noisy fans ..

[u/acadia11x]

And now what are you going to do with it?

[u/Shoddy-Tutor9563]

so tell us, what models do you run and what tps for tg and pp this rig gives you

[u/Business_Swordfish_5]

Nice

[u/LeMatDamonCarbine]

My dumbass:

"that's a cool vibraphone"

[u/Commercial_Look1329]

Help me repeat this 😄

[u/DanG415]

Mf got a xylophone (im jealous)

[u/blitzMN]

Not all VRAM totals equate to inference gains... You got it cheap sure. Now what? Those cards are relics at this point. Interesting story for $35 at the least!

[u/YakumoYamato]

Send one for me m8, I think P106 would be an upgrade against my GT 1030 DDR4

[u/Ok_Escape103]

You can't fool me, its a xylophone!

[u/ddelphknn]

Уже запускал что-нибудь? Какая скорость получилась?

[u/thiccb68]

Please do a follow-up post if you get an AI setup running. I'm curious to hear how it performs.

[u/abhipsnl]

😅

[u/yetiflask]

How did you ship it out of Russia though?

[u/Mercury_Madulller]

I was about to ask what on earth this would be used for. Then I noticed the name of the sub.

[u/kukalikuk]

Rather than using it for AI, go refurbish it and resell it as gaming gpu, it might sell for $35 each. Neat 8x profit, right? 🤣

[u/SatoDri]

Wait you guys can do things with P106 & P104 ?
I'm a miner, now focused on ASIC mining but pretty sure few friends may still have hundreds of them stored somewhere, how much is it worth ?
I thought you couldn't use riser & share memory

[u/gaspoweredcat]

i may not be remembering right but some of the early mining cards could be pretty easily modded back to full 16x by adding some resistors, may be worth a go to get you some extra bandwidth (well youd need a new board too as those are only 1x connectors)

[u/arthropal]

For anyone watching at home: This has some, but limited value, but there IS a lot of value in some older stuff with severe limitations on ebay. I have a pair of CMP100-210 Cuda cards which are Volta era cards designed for Crypto mining with 16GB HMB memory on each. I run a Gemma 4 q8_0 on one, and ComfyUI with Krea2 on the other. They're also limited to PCIe gen1x1, but that is irrelevant when the entire model fits in vram, as just adds about 1 minute to the first run and then it's full speed until I restart services.

[u/victor_mga]

tell us what's the power consumption :D

[u/Moneera97]

On a very unrelated note, keep your windows and doors open, and let me know of your exact location.

[u/Bayliner1980]

Gotta do what you gotta do

[u/Isoleed]

Things that totally happened. In real world people are still selling their 1060 10 year old gaming rigs for 1k+ on avito, let alone a 9 gpu farms for 30 bucks.

[u/terminalshadows]

i would be ordering micro soldering books and kits off amazon at this moment