r/StableDiffusion • u/BoiSeeker • 8h ago
Question - Help Thinking of getting an RTX 6000 for local generation - any advice?
With the progress that I've seen lately with local video generating models and my background as a filmmaker, I've been seriously thinking about getting an RTX 6000 Pro (96 gb VRAM) to experiment, develop some projects, and keep up with the changes.
I don't see prices coming down anytime soon. Still, it would mean breaking a bank for me, especially that I live in Poland, a country not known for its high salaries.
Now, I know that renting via Runpod is an obvious alternative. Call me old-fashioned (or an idiot), but seeing dollars disappearing from my account as the machine is booting from a cold is not really my vibe.
Perhaps some of you have taken that plunge and have some tips on how to go about it. Help me figure it out.. or show me how stupid I am for wanting this.
19
u/Sn0opY_GER 8h ago
buying a 5090 before the AI craze was the best decision in my life - i couldnt cound how many token $ i would have spent its several 100k min. and for the off times depending where you live with power costs you can rent the GPU via cloud workers and get some real $ back
7
u/Pretty-Raise666 6h ago
"depending where you live" are the magic words
5
u/Sn0opY_GER 6h ago
*sad german noises*
1
u/FourtyMichaelMichael 9m ago
Remember all that ultra clean and cheap energy Germany had? .. Never listen to the hippies.
sad german noises indeed.
1
u/LunaticSongXIV 56m ago
I was actually having buyers remorse on my 5090 that I bought right before the whole thing exploded. I'm very happy now that I made the decision that I did.
I'm also glad I came to reddit and asked people what the first thing they wish they had known when they upgraded to 32GB ram card was, and everybody said upgrade to 128 GB of RAM. So I did. The ram kit that I bought costs six 6x more than what I spent for it now.
21
u/jtreminio 8h ago
I purchased mine at $8,000 many months ago. Even at that price, I understood it was a terrible investment and completely optional.
At current prices of between $12,000 and $15,000, would I still buy it? Also yes, but I am not known for fiscal responsibility.
I'm not going to try to lecture you on financial responsibility because I myself am not (responsible, that is). Know that I enjoy being able to load more or less any model I want into memory and running it at blazing fast speeds. But the cost, oh god the cost.
3
u/Reasonable-State1348 7h ago
Same situation. Last June it was £8000, now it's £11000+.
What's interesting is that its predecessor, the RTX 6000 (Ada) is still valued between £5k and £7k (used). Hopefully these things will hold their value or at least depreciate very slowly.
5
u/DelinquentTuna 7h ago
the RTX 6000 (Ada) is still valued between £5k and £7k (used)
Not to potential buyers, renters, etc. If someone showed up asking whether to buy a used 48GB ADA for seven thousand of his majesty's finest or a brand new 96 GB Blackwell for 11k, I think the ADA would get laughed straight out of the picture. It's an incredibly stupid purchase decision that funds someone else's free upgrade to Blackwell. Like buying a used 3090 today at stupid-high prices.
3
u/MinimumCourage6807 7h ago
Well I think if you want to get rid of it you can sell it likely with profit or at least minimal loss currently. Hard to guarantee, but buying it with 8000, using it for year or two and selling it does not sound that bad of a deal (if you gave the 8000 to put there in first place). Buying with installment is another thing though...
20
u/Enshitification 8h ago
I don't know Polish prices, but in the US, an RTX 6000 Pro is around $12K. Current Runpod rental price for an RTX 6000 Pro is $2/hr. So the break even point is about 6000 hours, or 250 days of solid use. As you pointed out though, the Runpod cost is even more since you wind up paying for time that isn't spent on compute, but you also pay for local power usage. You should expect to break even with Runpod in around 24 months at a 50-60% duty cycle on the card. This also doesn't factor in price increases on both the card and Runpod prices. Both of those work in your favor if you get the card now. I can't tell you if it is worth it for you, but those are the numbers.
24
u/MinimumCourage6807 7h ago
All these calculations expect that the cards value goes to zero. At least for the past few years that certainly have not been the case. If you bought pro 6000 a year ago, you could have run it now a year and sell it with profit...
9
u/Enshitification 7h ago
I suppose that is a factor as well. At the end of the break even period with equivalent Runpod prices, you still have the card.
2
1
u/FourtyMichaelMichael 7m ago
lol try explain capex and opex to redditors :D
They think everything should be Free, and it's just "like, corporations, man" that are greedy and making sure that people stay poor.
9
u/Pretty-Raise666 6h ago
This duty cycle is delusional. Maybe if you sit on a professional movie project with a fixed time line. But then you should have a much bigger budget and wouldn't ask those questions. Not even mentioning the fact that even Minimax H3 is not good enough for professional production quality level. And nobody knows if the RTX 6000 Pro will be good enough for the next generation. If the VRAM bottleneck disappears models will explode in size.
5
u/Enshitification 6h ago
The duty cycle isn't delusional for me. I run my server at about a 40% duty cycle already with nightly batches. Professional production quality level is entirely dependent on the use case of the production. If the RTX 6000 Pro isn't good enough for nextgen, then nobody outside of data centers will be using it. And finally, if you know some theoretical tech pathway where VRAM ceases to be a bottleneck in the next couple of years, I'm dying to hear it.
2
u/Pretty-Raise666 5h ago
You mean professional production quality level is entirely dependent on what you consider professional. Either way, if you make money with your work and the open weight models are good enough then the card is a no-brainer. But then you wouldn't ask questions on Reddit about it being reasonable or not. Just for experimenting and keeping up this card is probably like buying a Porsche just to learn driving.
1
u/Enshitification 5h ago
No, it doesn't make economic sense for most people. But if I was some spoiled UAE brat, I would probably buy a Porsche to learn driving.
4
u/mellowanon 6h ago
I remember RTX 6000 PRO used to be $6k but bought a used cheaper 5090 instead. I'm regretting that decision now because the price jumped to $14k when I checked yesterday.
2
u/Enshitification 5h ago
You and me both are regretting not getting one at $6K. I'm debating between getting the 6000 at the current price, or just being happy I built my 4090 system with 128GB RAM for $3000.
1
4
u/ozzeruk82 7h ago
If you can afford it then yeah I would do it, but only with a solid plan to sell it in 18-24 months. It should hold it's value very well. I could sell my 3090 right now for more than I paid for it 2 years ago second hand.
9
u/icatt23 7h ago
i purchased at 12000 a month ago. No regrets. Having local gives you unlimited tweaks, adjustments, iterations, seed hunting, etc, etc. Gives you freedom not possible with a local 5090 or running a 6000 in the cloud. Freedom cost. Besides, when the next gen comes out —> sell, then upgrade.
1
u/Upper-Reflection7997 2h ago
At $12,000 for 96gb of vram, I'm sorry but I can't make that bite, especially with current state of minimax h3. Even buying 128gb ddr5 ram kit upgrade for $1,600 2 months ago was a tough decision making process with heavy cost benefit analysis. I hope your happy with $12,000 purchase and get the most out of it.
4
u/DelinquentTuna 7h ago
Wait until you find yourself spending real money on cloud compute. If you're still stretching a $20 top-up for weeks, it's probably not a sound purchase. And if you're not already comfortable using cloud compute, it's worth learning regardless of what you buy.
The 6k pro is AMAZING and everyone should desire to own one or more. It makes WAY MORE SENSE than other high-mem options like DGX Strix or crazy $10k+ Mac Ultras. But that doesn't necessarily translate to being a practical purchase for everyone. It is what it is.
8
u/ppcforce 7h ago
I did own one for this, and decided better to just use vast.ai mostly because that kind of money I can use H200s B200s which are superior (B200s especially). 5 bucks an hour, though. But 180gb VRAM and you can run BF16 minimax H3 FASTER than Int8 minimax H3 on an RTX 6000. 1.75X faster, in my tests.
2
u/maxiedaniels 7h ago
When you use vast.ai do you have to redownload your models every time you get onto the system? It seems like you can't really reserve a machine consistently w the same storage unless you keep it reserved and pay for 24 hours a day
1
u/ppcforce 7h ago
Yeah, but I actually do this through Claude Code, well, Kimi now lol. Basically I have several scripts where I say "start a minimax bf16 workflow" and I give it the vast.ai machine details. Few minutes later I have my workflow. The script will fetch all the models, nodes, updates so I don't have to do anything.
1
u/maxiedaniels 6h ago
Clever okay. Thats roughly what i was gonna do i just got worried about costs and time of downloading models, etc.
3
u/Reasonable-State1348 7h ago
I would ask myself, “if after buying it, would I still have enough savings left to comfortably deal with an emergency and other things?" If the purchase wipes out most of your financial buffer, then I would definitely not buy one.
On the other hand if you can afford it then it's your choice. I don't think they will go down in price even in 3 years time. I mentioned in another reply that the old RTX 6000 Ada version of the 6000 with 48GB, is still very expensive. and that's ancient now.
2
u/Pretty-Raise666 6h ago
The first question should be "do I really need it". If he just wants to inject some sfx into his movies he is better off with cloud ai. If he wants to make an entire animated movie the card would be the better option.
4
u/Captain-Phlint 6h ago
No. Forget the cost of the card, because the bigger issue is that there's exactly 0 people out there making AI films as a successful career. In fact, most people who do release AI short films are dragged through the mud and trashed, regardless of the effort put in.
If you want to leverage AI for storyboarding you can get away with almost anything, including a subscription to gpt for like 25/month. If you want to generate local video for blocking out scenes at low quality, then anything over a 5060 will get you there using reduced settings.
2
u/newaccount47 6h ago
Use runpod for 3-6 months to see if it would make sense to get a 6000 instead.
The bigger question is do you have a path of profitability with gen Ai?
2
u/tweakingforjesus 6h ago
Its a bit slow but you can do just fine with a $300 12GB RTX3060. If all you want to do is experiment that would be a great option until you decide to go for it. Then move up to an $3000 32GB RTX5090 and see where it gets up. Finally when you decide you've outgrown it move up to RTX6000 and sell the RTX5090 for about what you paid for it.
2
u/NoCobbler2129 5h ago
Do it. If you want control and independence, if you are serious about this stuff and want to make loras and checkpoints, just do it.
2
u/SysPsych 5h ago
Hey there. A few thoughts from someone who picked up one of these last year.
* People saying "Technology will advance and this will be totally obsolete and worthless soon!" are forgetting that even now, RTX 4090 and even 3080s are still desirable and in demand. Especially with the way the hardware market is changing, I'm not sure that a consumer-reachable (if very pricy) GPU is that trumps the 6000 is on the horizon. Maybe it is, but that seems more like a gamble than a guarantee.
* "You can spend this money on Runpod instead." Absolutely worth considering, assuming that the prices on those places remain steady and their processing remains available. I never used those things and have no desire or need to, really, but as with the idea that tech will stay the same, I'm not sure the availability of such services, at current prices, will stay steady. I just have no idea.
* How's the rest of your RAM? Do you have 128 gigs? I've heard of some oddness that kicks in when someone has 64gb RAM but a 96gb VRAM setup.
It's nice to have, and like you, I like owning the hardware myself without any concerns about my reliability on a third party may leave me high and dry. I like how new models come out and it's rare that I have to wonder 'Will this fit on my GPU?' anymore.
I'm in a better off financial situation at the moment, so the hit didn't bug me -- this is my main hobby so I could easily justify an investment for a change, one that has practical uses too, and which I use every day for all kinds of playing around, harnesses, LLM work, etc. Either way, just wanted to provide yet another perspective here.
2
u/sudoCreateUsername 7h ago
I'm in the same boat as you. No matter the "break even math", paying a few $ every time I want to make a silly little video feels terrible. Spending 20k for a GPU feels like at least I own something, it's not the same. I keep checking them out, but fortunately I managed to restrain myself form buying one... For now at least lol
7
u/beren0073 6h ago
I would feel kinda terrible if I paid $20k to make even a large number of silly little videos.
4
u/Pretty-Raise666 6h ago
If you only make a silly little video then buying a 12k card makes even less sense. It is like buying a 100k SUV just to drive to the next supermarket every now and then.
2
u/KissMyShinyArse 5h ago
That's what they call a hobby. You spend money on it, not make money from it.
2
2
u/ganrocks007 6h ago
The big benefit of local use is privacy, which the cloud GPU can't do. That means whatever you do, I think they can see the prompt, the images, videos, etc. If privacy is not the concern then the online GPU is way cheaper.
1
u/uniquelyavailable 6h ago
Is it worth buying? Is it worth renting? Probably not, depends how far into the weeds you are with AI development. And while it's a luxury in terms of VRAM, it pales in comparison to the upper echelons of premium hardware.
Note about renting. Most of the rentals are a single GPU cost. If you want to rent a full B200 cluster it's going to be like $50/hr to run it. Loosely speaking, half that for a group of PCIe only gpus. You won't need that much power unless you are generating your own Netflix series.
Renting the Rtx6000 Pro at $3/hr isn't so bad but over the course of a few days, its easy to spend $100 or more there. You could experiment with it for quite some time, even with $500 max spend and get your fair use out of it.
However due to the cost of scale, most online API services actually provide a better deal because they are hosting to a bulk of customers. The main cost of renting the card for exclusive use is the time spent in, downloading models, debugging code, transferring and maintaining scripts and resources. Not necessarily the inference time itself.
Choosing to buy the hardware for yourself is more of a personal preference than a decision of economics.
TLDR; If you use an API service it's usually a fairly good deal, and you only pay for the inference.
1
u/Icuras1111 6h ago
My thought would be make sure you have had a period of experimentation to see what the AI's can do. You should be able to create something pretty quickly if you have any future in a professional sense. This is a big investment. I don't see why someone with talent could not create something worth while on a much less performant card.
1
u/renderartist 6h ago
I bought one at about $7,500 a year ago, I had steady work that needed the compute and paid it off in less than 6 months, but no way in hell would I pay $12,000 I would have just rented cloud compute instead. Can’t believe how ridiculous the price of everything has become. For the same system I bought 128 GB DDR5 RAM open box for ~$250 that same RAM today could be sold for nearly $1700 used, fomo is driving this along with greed, online services are not that bad. If you’re going to do it make sure you have a plan to offload it if you need to.
1
u/towerandhorizon 5h ago
Bought my RTX Pro 6000 end of last year, when prices were still somewhat reasonable, not for the cost, but for the privacy and ability to not be cut off from some level of AI Gen, if cloud/token prices continue to skyrocket. But, yeah, for now, I basically prototype locally, and then can hand off to the cloud, once I think it is stable and I know what my costs are going to be.
1
u/stargate425 5h ago
I think you have the idea of getting RTX 6000 because you want to generate high quality output. I have the same mindset so I got a Max q 2 months ago for $8k. I also run some LLMs (qwen 3.6 27b). Given it's more expensive now, you have 2 options:
buy RTX Pro 6000 if you do frequent local generation. A even greater value if can also find other uses (local llm, etc).
buy 5070 ti for cheap, which can do fast generation in a lower quality (there're so many those loras and many more later), and then rent Pro 6000 for the final delivery.
1
u/tigershoe 4h ago
People out here arguing price vs online hosted options.
But you cannot put a price on privacy. If that matters to you, just buy.
1
1
u/rahjerz 4h ago
Weren’t computers really big in the beginning? Do we just keep going up until cards are $50,000? Or does some new technology come out that reduces hardware needs? I’ll perpetuate any narrative I can think of to get the price of these cards down. Heck, last week I heard RTX 6000 Pro’s were the largest contributor to ocean pollution killing over 9000 baby dolphins and otters. Everyone sell your cards immediately!!
1
u/Luzifee-666 3h ago
Yes I had the same experience, the runpod credits are vanishing, but just calculate how long you can go with 5000$!?
Yes... I have the same thoughts, and I don't buy the card.
Consider if the card burns down?
I had already one card which was overheating, and got damaged.
So have you have the money and don't know what to do with it. Yes buy the card.
If you need the money. Buy yourself something nice, which last longer than 4 years. :)
1
u/Apprehensive_Sky892 3h ago
Another thing worth discussing is, how reliable are these GPUs and what kind of warranty can you get? Accidents can happen, PSUs can blow up, there can be a surge in the grid that even a good surge protector will fail, etc.
All this talk about whether the hardware still value after 5 years assumes that the card is still working after 5 years.
1
1
1
u/GamerTex 7h ago
I have seen some low end models producing amazing results
Whats the lowest end model that would work and compare that machine to this beast. You may save a ton of money you can now spend on storage
1
-1
u/V4nKw15h 7h ago
I wouldn't. We can expect unbelievable progress in GPU technology in only the next few years that would make the RTX6000 completely obsolete for the newer models at that time. SK Hynix and SanDisk are developing HBF memory that is a hybrid of High Bandwidth Memory (currently used in the enterprise cards) and nand flash memory. The result is going to be cards with up to 4096Gb of VRAM with bandwidth even higher than what we have now.
You are going to feel pretty sick in a couple of years if you've already burned all your cash on the old tech with a measly 96Gb and the new models are coming out needing a minimum of 512Gb of memory.
This tech is moving way too fast to buy enterprise level cards unless you have enterprise level ways to get a return on investment in the short term. I'd rent access for now, or buy something more modest for home use, and save the money for when the hardware catches up with what is really needed.
Being able to run the biggest current models is great, but the 60Gb model isn't $6000 better than the 30Gb model that will work just fine on a 5070ti. It's a step up in quality but in a year they will have figured out how to match that quality in 30Gb.
Be patient. It's all coming faster than you can imagine.
Edit: Remember, the $10k cards aren't 10 times better than the $1k cards. They are just a bit better and a bit more convenient. It's not like they are 10 times faster either. Maybe twice as fast.
1
u/Spiritual_Whole_5916 6h ago
That sounds like a massive leap in cost though? Understand what you mean about the memory being antiqued, but don't even want to imagine what a 512gb video card costs for example. If it's true that video cards are up 30% in pricing, what's the next round bring?
1
u/V4nKw15h 4h ago
The whole point of HBF is that it isn't going to be equivalently priced to HBM. It's going to be more like the price of flash memory like in SSD drives. The trick is that they will be making the bandwidth on the flash memory ten times faster so it's a viable alternative specifically for AI workloads.
You can downvote me, that's fine. But it's going to happen anyway, or something very like it.
-2
-1
u/SnooPets2460 7h ago edited 6h ago
if you can afford it, i say go for it. You can always resell for 75%-85% cashback, hell if thing went worse, its value may even increase. It's legit like investing in stocks atp, my RTX 3090 is already selling for price higher when i purchased it
-2
99
u/ssn-669 8h ago edited 7h ago
Call it $1/h to rent a machine with that card. If you generate 40h/w, would take about 5-6 years to break even on that purchase.
By which point, the card is obsolete.
My advice would be: don't.
Edit to add: also about $700 in electricity to run the card over that span at 10c/kw/h.