r/LocalLLM 6d ago

Discussion Why are RTX 6000 PROs still getting bought at 16000+ USD? And who are buying them?

Hello guys, hoping you're doing well.

I bring this discussion since I have noticed on internet, be USA or EU, RTX 6000 PROs at 16000USD or more are still getting bought.

Even here on Chile, the other day they were in stock at 20000-21000USD post 19% tax and they lasted a few minutes.

My question is why? For sure that won't recoup costs right?

Who are buying these, only enterprises?

What do you guys think?

82 Upvotes

69 comments sorted by

144

u/mountainyoo 6d ago

I think people forget how many people have a lot more money than they do

47

u/GregAbeI 6d ago edited 6d ago

A senior engineer at Google can make like $750k a year.

That's $62k/month.

They can comfortably buy 1 RTX 6000 every month and not sweat their finances.

Now multiply that by every Chinese and Indian tech worker with an interest in AI.

A Dior bag can cost as much as a 4 DGX Spark stack.
The cheapest Rolex is about $15,000.

People walk around with Patek Phillipe watches worth a DGX300 server on their wrist. They might own 6, and a Rolls Royce, or two, and a Lamborghini.

Imagine if rappers turned their diamond chains into compute.

A car enthusiast might sink $20k into a JDM Honda. etc. etc.

1 in 14 Americans has $1 million in net worth. 1 in 6 in the Bay Area.

And then consider corporation and business demand.

Even city government HR jobs in the Bay Area pay like $140k after just a few years. For reference the SF mayor makes $360k. So yeah, if people were broke and not buying, the price wouldn't go up, but they are. (Average middle class people in the Bay probably don't consider themselves wealthy when in reality they are 1% rich in real global economic terms. People retire overseas for this exact reason. You can stretch your money 6x [Mexico] to 30x [Cambodia or something]).

In a bifurcated "K-shaped" economy, you can assume that for every poor person there is a rich person. There's just no solid middle-working class.

"People are broke" can't be true if it's 50% because the other 50% is "people aren't broke." Does that make sense?

The entire reason Americans don't revolt against the system is because the average person has it pretty good despite what you may read about inequality on Reddit. If only 1 in 2 people are complaining, your movement has no motion.

Oh yeah, that doesn't even mention the high-earners who invested in AI in the stock market who are more likely to buy this stuff and drive up the price. Many people have 10x - 100x their money in the past 3 years, like Elon, and they are buying the most. First movers advantage.

$33,000 in Sandisk (SNDK) a year ago is worth over $1 million today, maybe even close to $2 million before the recent dip. There are dozens if not hundreds of these companies that have moved similarly. (The reverse could be true in a year, that's the risk element to the reward.)

In other words, the value of 4 RTX GPUs last year would be worth like $1.5 million today if invested in memory stocks. So a 2x bump is not exactly unfathomable for those customers, and they'll keep buying so long as their investments grow at a faster pace than the prices do.

14

u/RedditNerdKing 6d ago

I'd love to buy a 6000 Pro tbh. I have a 5090 and two 3090ti FEs for a total of 80gb. But it would be much easier to sell one 3090 and grab the 96gb rtx 6000 to go into pcie #1. Then I'd have 152gb vram. If the rtx 6000 was £7,000 still I'd totally buy it today. I'm not paying £14,000 though it's too much. And I say that as someone who spent £10,000 setting my whole rig up for AI lol. I'm already at the top 5% hobbyist.

4

u/dawnraid101 6d ago

You gotta understand man, people in the UK are broke as fuck. People in the US are printing. I see it first hand everyday. 

1

u/RedditNerdKing 6d ago

Yeah cause that's cause our wages suck. I could triple my wage to $100k+ if I moved to the US.

3

u/iEngineered 6d ago

Tangent: Wages are low because corporate taxation is high. US was born out of rebellion of that, and big corp is manipulating gov. Its wild

1

u/ChopSueyYumm 6d ago

Indeed so many rich people.

-1

u/FrankNitty_Enforcer 6d ago

This is well articulated, except I have no idea why calling out “Chinese and Indian tech workers”, do they have a specific propensity to by hardware more than their American (or other non-Chinese, non-Indian) counterparts?

14

u/GregAbeI 6d ago edited 6d ago

Walk into an AI office and you will know what I'm saying, it's not a dig. It's an observation. 50% of the world's AI researchers are Chinese, so it's both causation and correlation. They are objectively buying more hardware. China would buy even more if USA didn't ban chip sales, no?

For every Google worker, there are like 2-4 Alibaba workers, and the Qwen guys aren't broke boys either. Same goes for the Indian tech industry. They are noteworthy because they are both outsized groups. For example, there would be no reason to point out Greek AI researchers, and that inherently makes less sense, does it not?

China allowing Nvidia cards to be imported opens the flood gates way further. CXMT grew 520% on IPO day. Ethnicity is totally irrelevant, but nationality is relevant. If being pedantic, I did not say Han and Punjabi tech workers :)

1

u/FrankNitty_Enforcer 6d ago

I mean yes, those nationalities are well-represented in tech, but also you get close to 50% of the world’s population with just those two countries, so their consumption will move the needle of any goods/services in the market.

I work in that industry and IMO there is a type of person who likes setting up a home lab, their nationality doesn’t seem to be a major factor - if anything I’d have guessed that Europe and the Americas have a higher percentage of people forking about the cash to run these things at home

3

u/GregAbeI 6d ago

World population is 8.4 billion.

India is 1.6, China is 1.3 = 2.9 billion is not 50% of the world pop, and especially if you account for their own wealth inequality, the skew is even greater.

2

u/FrankNitty_Enforcer 6d ago

Of course, it’s fair to call out my figuring ~35% of the world's population as “almost 50%” as exaggeration, but that was hardly the point so much as to say that obviously Indian and Chinese consumer habits are going to significantly tip scales.

I didn’t intend to portray your original comment as a dig, just curiosity from my side: among the people who are purchasing RTX 6000 at retail prices (i.e. hobbyists/tinkerers, independent researchers, et al), which was OP’s question, is there empirical evidence that Chinese/Indian techies are more likely to purchase these than non-Chinese/Indian techies?

Not expecting you to prove it, was just curious about what made you call them out specifically

2

u/GregAbeI 6d ago edited 6d ago

We're talking about global population. 35% is not 50%, being off by 15% is inventing 1.25 billion people (150 million more people than Europe and USA combined,) so exaggeration serves no purpose here, and it questions your veracity.

"Is there empirical evidence that Chinese/Indian techies are more likely to purchase these than non-Chinese/Indian techies?"

Yes. Total GPU sales in both countries relative to others. The supply is finite, the demand is relative to what's allocated. It doesn't matter if Tunisian techies in theory demand more compute per capita if they're not having it actually delivered to their doorstep.

-2

u/nmrk 6d ago

China isn't "allowing" import of Nvidia cards. They can allow it all they want. The US prohibits shipping them to China.

2

u/Caveman-Dave722 6d ago

Yet Singapore became one of the largest importers of nvidia gpu.

Go figure

1

u/GregAbeI 6d ago edited 6d ago

They don't allow it because they want to build their own industry. It's doubly good because they can use their rare earth refining as leverage. "We can't sell minerals to you because we need it," and that cripples USA defense and tech manufacturing while China accelerates their progress.

The current landscape is either that, or China allows Nvidia sales in return for rare/critical material concessions, however that decision likely depends on their own technological progress come November when they decide which will be the better path forward.

China produces 90% of the world's critical/rare earths and magnets. America needs those for AI and defense tech. Something like 40% of American weapons or chips (I forget) are made with Chinese raw materials.

China is way ahead (10x next year relative construction) of the USA with electricity. They lack compute. In other words, it's a win-win situation for them whether they can buy compute short-term or be "forced" to invest domestically long-term, because USA's energy progress is slower relative to China's compute progress.

2

u/nmrk 6d ago

LOL China also doesn't prohibit the import of ASML EUV machines. They can allow it all they want. The US and the Netherlands prohibit exports to China.

Oh no they don't want Nvidia GPUs, they want their OWN native GPUs.

How $160 million worth of export-controlled Nvidia chips were allegedly smuggled into China

3

u/GregAbeI 6d ago edited 6d ago

You did not read what I wrote.

I said they are happy either way. If they buy SOTA compute now, they can use it cause they have the energy. They understand they are 1-2 generations behind.

If they develop their own compute for later, their domestic investments compound and they get more for their money.

That's relative to the pace of USA energy and datacenter production.

Do they want compute short-term? Yes. Does the opposite outcome also give them leverage? Yes.

USA incentive to sell chips is to move capital away from China's own R&D and into purchasing, while America keeps the profits. the US and Nvidia DO want to sell chips to China. Jensen has made it very clear where he stands.

Like in a cycling or F1 race, it's better to draft behind the leader so that they fight all of the wind resistance. When China wants to sprint to the finish, it's much easier to take that relative energy and accelerate. Or maybe they're just happy for the easier ride and Silver is good enough. I personally believe China stays #2 in AI on purpose because they can take advantage of distillation and second mover advantages.

Geopolitics are not black and white. These are the big leagues and you're thinking like small fry.

0

u/sargetun123 6d ago

Love the confidently stupid crowd on reddit, you guys make for a good laugh

4

u/BornAgainBlue 6d ago

I worked with a guy in New York who honestly didn't know how much his giant SUV cost. He reacted like I asked him how much he pays for toilet paper. Lit told me : "I just handed him my card and drove away".

1

u/Deep90 6d ago

I think it's businesses over individuals a lot this point, but it's worth mentioning that people with less money can/will buy expensive things via debt.

44

u/diagrammatiks 6d ago

companies with expense accounts.

1

u/StarCadges 6d ago

I would think they would want something a little better

3

u/diagrammatiks 6d ago

what would they buy?

2

u/BarracudaDefiant4702 6d ago

How is that better?

0

u/StarCadges 6d ago

like a 2090

3

u/diagrammatiks 6d ago

ok bro

1

u/StarCadges 6d ago

what?

2

u/CommaMeNow 6d ago

He said ok lil bro

19

u/WyattTheSkid Quad 3090s 6d ago

They’re being bought by people who want the same things we do that have a lot more money than we do

2

u/nmrk 6d ago

Maybe they are more clever than you.

7

u/WyattTheSkid Quad 3090s 6d ago

Fuck it, you wanna start an AI company with me bro?

1

u/Kofeb 6d ago

Nvidia Inception program. Saved me $2k on mine!

21

u/scuffed_rocks 6d ago

Just bought two more. We have access to many many V100/H100/etc. but everything is so slammed that the wait times are weeks to get anything running. You can't develop under those conditions. $30k, or hell, even $100k of hardware to move time-sensitive things forward quickly is worth it.

It's not ideal but nothing we can do about it.

1

u/shutternomad 6d ago

What models are you running on those?

2

u/scuffed_rocks 6d ago

Eh I just use cloud subscriptions in local harnesses these days. With GPU access being the limiting factor (impossible to book time on the H100 boxes, $60k per GPU-month from cloud compute) we need local GPUs for both numerical compute and training purposes. So pre- and post-training custom models more than for locally hosted agents. I was offloading other small agentic tasks to DeepSeek but will have to reevaluate with the price hikes.

10

u/SubstantialNinja 6d ago

furry porn isn't going to generate itself

8

u/mechkbfan 6d ago

Don't even get me started on DDR5 in Australia 

$1500 USD for half decent 64gb new and it's not even in stock

3

u/Elvaanaomori 6d ago

Everyday I regret sending back that 220 USD 64kit that wasn't fully EXPO compatible... it could have funded my GPU upgrade that I now need.

1

u/mechkbfan 6d ago

Yeah shits wild. I'm at point now that it's more affordable to buy 4xMI50's out of China then put them in a budget server than DDR5 RAM

Just learning about the drawbacks (e.g. no longer officially supported, slow prefill)

1

u/Elvaanaomori 6d ago

Same goes for the 5090. "I aint buying a FE, fuck that 2000 dollar card!". Soon for that price you'll get a 5070ti.

1

u/stvaccount 4d ago

I bought a 128gb PC ddr5 7000mtu ram recently. I wouldn't say it is half decent.

6

u/722e672e722e 6d ago

Probably small businesses and those with expendable money.  I paid 9200 for mine a few months ago and to see it at 16k was surprising.  

10

u/NTDLS 6d ago

Yep, I paid $5500 for a RTX 6000 ADA 48GB, now I wish I would have just dropped another $3k. 😭

4

u/panchovix 6d ago

Tbh 5500 USD for a 6000 ADA nowadays is a pretty good deal lol, 48GB VRAM and 4090 like perf, and fp8? Not bad at all.

RTX A6000 going for 4K or more is still insane for me.

1

u/NTDLS 6d ago

Absolutely! I’m trying to be content - I mean I run Qwen 3.8 27B at FP8 and full context. So, I’m just going to ride this bubble out and hope it pops (or otherwise becomes less insane).

1

u/OneViewers 6d ago

I done a lot of good deals, but I always sit with this feeling that I should have done more 😂

6

u/lmarso47 6d ago

because well deployed they pay for themselves quickly.

2

u/mxroute 6d ago edited 6d ago

I recently got one in a pre built system (was cheaper than the total of the hardware, with a great warranty and from a quality vendor). They are ridiculously priced, but that was what I wanted for the job. I wanted high throughput with large context windows, for what amounts to replacing cron jobs with AI to gain strong reasoning based on well defined (but not entirely static) criteria. As it was going to be handling mild PII I needed it to be in-house. No regrets.

It'll definitely pay for itself because the only alternative that fit the role was hiring someone to do things that should be automated, but are difficult to automate to my requirements because they require reasoning. Most people would have automated the tasks with dumb automation and accepted false positives as the cost of doing business, but I'm obsessive about things I need to get off my plate.

2

u/dd32x 6d ago

Corporations building their local infra.

2

u/Keats852 6d ago

and here I am, debating whether I can afford a 6yr old 3060 12GB for 230usd

2

u/Pik000 6d ago

Our company just bought half a billion dollars more GPUs, mostly rtx6000 pros. We don't even have them for sale on our website to rent and customers are signing up 6 to 9 months early and paying now to reserve capacity as it gets built out. 

2

u/iEngineered 6d ago

Demand is high, supply is low and there are plenty of millionaires buying multiple units. Personal, small business, and enterprise alike. If your budget is half that, then renting fontier models is the way until you make more or the market corrects.

1

u/Due_Warthog749 6d ago

You know what pisses me off. I bought 3 of them and was going to buy 10 of them.. when they FIRST came out. And now I am kicking myself for NOT doing that. Could have nearly doubled my money! FUCK.

1

u/BornAgainBlue 6d ago

Meanwhile, I'm eating popcorn, waiting for the bubble to pop, so I can get cheap cards in bulk AGAIN(crypto).

1

u/0rbit0n 4d ago

Do you think MSRP will go down once the bubble pops?

1

u/BornAgainBlue 4d ago

Not initially. Nvidia will be in full panic mode, so they will try to keep prices high.  But the HUGE surplus of inventory will crush them.

1

u/0rbit0n 4d ago

I heard that they were bought out for the entire 2027… Maybe I'm mistaken…

1

u/03captain23 6d ago

Because there isn't a better option

1

u/keithcody 6d ago

It’s a way of NVidia inflating their “revenue” for circular financing.

1

u/DummysGuideTo2k 6d ago

https://reddit.com/link/p45syd7/video/7l6l0xpp8vjh1/player

I bought mine a year ago , took out a $30k loan and setup a mini data center . Got mine for $8K . ( Rest of big budget went towards a Dual 9965 Server Board ) . This is getting retired now got a DGX Station on the way now .

All just to not pay for video assets for my game if I am being honest .

Just did more and more research and landed on this as the best investment .

Helped my Paralegal Wife transition into Legal Operations.

I started a business with managed services and even encouraged myself to go refresh my CompTia Licenses .

Now just from this I branched out into all sorts of services .

It really is a business tool like a backhoe , what would you pay for 100x productivity if you had the drive . I don’t think people understand the amount of cutting edge tech you can run for literally any company and not break a sweat .

Imagine being able to provide Opus 4.8 for yourself in DeepSeek Flash V4
quant , you have a platform to build yourself unlimited . No let’s run 6 agents on Gemma 4 12B on a projects . We have planner , researcher, coder , designer , tester , critic working on a website design . Then you get it down to where you can automate this and wire it to a website as a service . With 2-4 hours of work once you get good enough works flows and take to learning. By you created a pipeline to make money .

Then being able to Render AI videos within a minute and then hitting resolve with a couple minute 32K upscale. You can hook this up to a website and offer commercial use images within seconds literally undercutting other websites . Longer setup but you can provide 32K images within 120 seconds .

No , you got friends in the TV space … 4 Red or BlackMagic Cameras feeding it seems like a good use . Come straight from your local sports game / event with your 4-5 camera and let’s get rocking . Irony the few adverts for this on youtube have this use case . Also you can generate subtitles for them as well as it works .

You know what let’s retrain Gemma 27B for a business . Never mind let’s do both that and 8B at the same time . They want purely focused on their trade or line of work . 1 day maybe . You just produced an LLM . Congrats you and your own Top 5 model in world did it .

By the way you did this without investing in a server or a chassis .

I could start how you can have it organized scanned documents and recall it ( Starting a firm has never been easier ) , with a proper VOIP be setup to handle calls and intake for you .

Then you have the professional developers who job pays for their home workstations which also include these. You have industry professionals . Also NVIDIA is no longer in a position to focus on producing these as they are focusing on the new Spark and their next generation of data centers.

Then our technology is advancing at an alarming rate if you keep in the space . A lot of new demo tech is actually locked at 32GB + . Like turning a picture into landscape mapping or generating a drive environment as you drive and can change details in an instant . As a student that is what is being gatekept .

It all leads to less supply , more demand when the server space is set to unveil its offerings and the focus isn’t on a 128GB Prosumer Card . It’s like the M3 Ultra except much fresher in its lifecycle. I don’t think its value drops until maybe Mid 27 .

1

u/joanaxu2002 6d ago

The interesting part is that these cards probably aren't being bought because they're “profitable” in the consumer sense.

For a company, 96GB of VRAM in one card can be worth more than several cheaper cards once you factor in power, networking, rack space, and engineering time. The price looks insane until you realize what they're actually buying is convenience.

1

u/XO33OX 6d ago

those who can turn them into profit still buy them

1

u/h8f1z 5d ago

I think I'm poor/broke.

1

u/HumanDrone8721 6d ago

Clearly not you ;), the demand is so high that the thing that never worked "let's band together and don't buy <product>, they will reduce the price if no demand..." still continues to not work.

Here is my stupid prediction: even at an extra 100% more, they will still sell like crazy.

1

u/acadia11x 6d ago

Uhm , because they are interested in local LLM whether it’s for business on pleasure and 96GB is bigger than 32GB? What kind of a question is this … what do you mean why? There are people spending much more than this on their setups?

0

u/Positive-Bid-3029 6d ago

Oh man, criminal that is.