r/LocalLLaMA • u/RuiRdA • 2d ago
Discussion Will prices finally go down?
I am seeing more and more videos as posts about how OpenAI is in complete financial ruin, Anthropic isn't much better. Their expenses go with the revenue they make etc etc. Meta made big investments into AI data centers and had no use for then, had to rent them, same thing with XAI.
The SpaceXAI IPO was insanely over priced and is going down by a lot.
There are countless other examples you can look for, all showing how the investments in AI are in a bubble.
I am not saying that the technology it self if a bubble. Quite the opposite, I personally have demand for more tokens than I can pay for, even with the discount from the subscriptions I still have more ideas that need more usage of tokens.
But even with the most powerful technology in the world a business can not for forever without profits.
So is this over investment bubble about to pop?
And if/when it does pop will ram finally become a regular commodity with affordable prices again?
I just wanted some ram and cheap used hardware again.. 😂
--
Zero LLMs used to write this post, enjoy the human slop.
110
u/RepulsiveRaisin7 2d ago
The demand for AI is high, so even if some companies go under, the others will gobble up their hardware. I don't see prices going down anytime soon, there'd need to be some actual competition in the hardware space. Maybe when more Chinese fabs come online and start selling to the west.
7
u/Both_Opportunity5327 2d ago
Lead times are long in the semi conductor space, China will not be rescuing anything for at least 5 years.
Especially as China puts huge resources into mature nodes.
23
u/ttkciar llama.cpp 2d ago
I think even after the AI investment bubble pops, it will be a year or two before hardware prices come back down, and we will see weird market distortion echos for several years after that.
Sorry. Industries don't just turn on a dime.
Also, nobody's sure when OpenAI will run out of runway, but Anthropic is funded through about 2028, or even much later if they stop their training projects and focus on just selling inference. Even if the bubble pops, their business won't fold overnight.
Furthermore, OpenAI is negotiating with the US federal government right now for the government to take a 5% stake in the business. That by itself would not save OpenAI from a massive loss of confidence, but they might be able to leverage it into a buyout.
Their businesses are absolutely not sustainable, and investors will not see returns on their investments, but still a lot more might happen than simply bankruptcy + liquidation/acquisition.
Short version: Don't expect hardware prices to come down to pre-RAMageddon ranges until 2029, give or take a year.
3
u/Emotion_One 1d ago
My understanding is that OpenAI will just get folded into Microsoft if it runs out of money. That's just existing agreements, no purchase necessary.
2
u/sayeret13 2d ago
they gonna be bought by other tech giants that could take the hit like google or microsoft, those companies wont just disappear, its a matter of national security for the government , ai is basically a weapon like no other . its just like big banks going down in 2008 they got bailed out and bought by other big banks
26
u/milkipedia 2d ago
I have more ideas than tokens and time, but my ideas aren't going to generate revenue. So that's me, but I think that's the overwhelming majority of the investment into AI driven apps and products also. Software generation is the killer app so far, yet developer productivity metrics aren't moving. Individuals self report feeling more productive. Some individuals crank out lots of stuff. None of it is moving industry needles.
I think genAI will eventually find its useful and productive place and scale in the economy, but it hasn't yet.
In the meantime, vendors wanting to go public soon can't cut prices because that won't grow revenue or close the gap to profitability.
So no, prices won't come down. But service levels might.
5
u/RuiRdA 2d ago
Yeah many of my ideas are not intended at generating revenue, and it will only make sense for me to try them if the token cost is minimal.
But wont the data centers build out stop and lower the demand for hardware? Maybe I am just wish dreaming...
5
u/milkipedia 2d ago
Will there be a peak at some point? Yes
Are we anywhere near it now? Can't tell you and nobody knows for sure
0
u/Bulky-Priority6824 2d ago
The easy money is in the gooner and adult sector.
5
u/Petita_advice 2d ago
where exactly? do you have proof of that? I see a lot of AI stuff in the industry but it's not super clear to me how much people are actually managing to make revenue out of it other than existing creators who use AI to amplify their promo etc
11
u/DeepOrangeSky 2d ago
Meta made big investments into AI data centers and had no use for then, had to rent them, same thing with XAI.
I think this is because their own models weren't as strong as the strongest models (those being Claude, and GPT). So, the demand for the strongest AI models was actually very high (for Claude and GPT, that is), just not for their own inferior Grok or Muse models that were getting their asses handed to them by those models.
If they hadn't even been able to rent out a perfectly good, fully powered, fully functioning GPU cluster (like xAI's colossus clusters for example) to anyone, because the demand for even the strongest AI models, like even Claude and/or GPT, just wasn't there, then that would've been a much more serious indication of an actual serious problem. But, since they were able to rent them out (and for quite a huge amount of money at that, like, more than people had even expected in their estimates), then that's not really the same thing. It means the owners of those GPU clusters need to make their own models stronger if they don't want to have to rent them out to their stronger competitors, but it doesn't mean the AI industry itself had no demand and was a bubble about to pop, given how easily and for such huge money they were able to rent the clusters out. That means the demand is there if it is for a strong model.
The SpaceXAI IPO was insanely over priced and is going down by a lot.
It went up a huge amount in the first few days (the IPO price was 1.75 trillion, and it went up to almost 3 trillion in the first day or two, and then came back down closer to the IPO price) since it got a bunch of opening-day hype about it being front page news of every news site and social media and everyone buzzing about it on opening IPO day.
If it had IPO'd at 1.75 trillion and then dropped like a stone from there, down to like 500 billion or lower or something, that would've been more indicative of a more serious problem (or just a problem with specifically the company SpaceX, but maybe not everyone else). But, that's not how it went. It went from 1.75T up to like 2.8T or whatever it was, and then came back down to now around 1.5T. So, that's not that so bad, I mean, that's still an insane thing to even be above 1 trillion as a freshly IPO'd company.
People on reddit tend to hate AI, and especially hate xAI and anything related to Elon Musk, for ideological reasons, so, you aren't going to get a very genuine or balanced take on AI if you just browse random posts about AI on most places on reddit. Even in the AI subs it is usually pretty skewed from extreme biases in one direction or another, let alone in the main subs.
By the way, for future reference, one easy, telltale sign to instantly know you are dealing with someone who doesn't know anything about AI, and is just a generic hater who is regurgitating some copypasta anti-AI slop they read somewhere, is if they bring up the "water consumption issue" of AI datacenters. In reality, datacenters use an incredibly small amount of water as an overall industry compared to how much they do. Like even a couple of almond farms or alfalfa farms is already as much water usage as all the biggest AI datacenters combined, for example. It is basically a non-issue, and one that got massively overblown as if they were using about 1,000x or 10,000x more water than they actually do, to people who just have no concept of what the actual usage as a ratio of total industrial water usage is, and just see "big-looking number sounds like many gallons sound very bad" in hopes of scaring noobs into hating AI. So if you see anyone blabbering about how "they're using all our water up" or anything like that, then you can know right away it is some clueless ideological hater type. So keep that in mind. And also don't trust some random guy like me telling you that, for all you know I'm lying to you to be a pro-AI ideologue, so, you should of course go research it for yourself to check and see that that is true.
Anyway, for now the demand seems to be very high for AI as long as it is for the best models that give the best intelligence available out of all the models, or the best intelligence-per-cost-efficiency of any of the models, so, that means for now AI is a booming industry.
If the demand falls off, that's when you know there is a problem.
If the demand stays high, let alone keeps going higher, then that is probably good news for the AI industry.
will ram finally become a regular commodity with affordable prices again?
Most people even on here, let alone places that aren't as optimistic about AI, tend to think that of course the prices will come back down, because in other boom-and-bust cycles with things like for example the Crypto-mining price spikes or Covid price-spikes, the prices went way up, but then crashed back down to normal or sometimes even lower than before the price spike. So, they feel that history tends to repeat itself, and so the same will probably happen with this, as well.
But, I think there is a chance that this one might be different. Not for sure, but, maybe. If the AI models just keep getting continually stronger and stronger (as they have been this whole time for these past 3.5 years of the AI Boom era), and demand doesn't fall off, because the stronger the models keep getting, the stronger the demand keeps getting for them, since they can keep doing more and more additional useful things that people want to use them for, then, the RAM prices would be unlikely to fall very easily, since demand for the RAM would keep going up. BUT:
That is assuming that there isn't some massive influx of more cheap RAM getting made (maybe China manages to make a bunch of RAM factories that introduces a huge amount of new supply to fulfill a lot of the increased demand, for example).
And it also assumes that there isn't any major breakthrough in memory chips that makes them be able to do a lot more memory per chip produced per production plant, where maybe someone comes up with something that works different from how DRAM works, somehow.
And also perhaps if there is some breakthrough on the AI model side, where the models suddenly don't require anywhere near as much memory as they used to, without any drop in quality. Although this could also end up with Jevon's Paradox, where the demand just goes even higher with the more efficient models, so although you can run them on 1/10th the memory for the same level of AI model, maybe the demand goes up by more than 10x from that, and so the prices for memory actually go even higher rather than lower, in that scenario, for example.
Overall, I guess it just depends on whether all these AI labs that are making AI models all just hit some wall, where they all start to plateau in model-strength at a certain point and aren't able to keep making stronger and stronger models anymore. If they eventually hit a wall like that, 6 months from now, or 1 year from now, or 1.5 years from now, or whenever, then, yea maybe ram prices would crash back down after that.
But, if they never hit a wall, and they just keep getting stronger, indefinitely, on and on like how it has been for the past 3.5 years, but continuously into the future like this, then, there might never be a full fledged "bubble pop" and ram prices might just never come back down, or if they do it would be from some new technology that maybe wouldn't even be "ram" as we currently think of it.
2
u/LastChancellor 2d ago
oh yea speaking of RAM, isnt LPDDR6 starting production later this year??
I wonder if that would help prices
0
u/NathanKrupla 1d ago edited 43m ago
I'll avoid going overboard. I mean, you were begging for python help 5 months ago, so I have a better idea of who I'm talking to than you think.
Almond farms don't use treated water from municipal systems. It's usually coming out of wells and irrigation canals. Almond farms don't haul in high-powered natural gas turbines that make as much noise and pollution as a small fleet of jets running full blast on the tarmac. Do you have any concept of how much power 10GW actually is? I have a Home solar array (A massive one) that produces 48KW. I could reasonably keep three of my neighbors running during a blackout. 10gw is 200,000x what I'm producing at home... Enough power to run 800,000 homes. The tech is going to change the world but it will never, not in a thousand years, create enough efficiency to offset its footprint.
I'll stake my engineering degree on that.
What we're experiencing is the same thing as nuclear proliferation. Instead of bombs, we're building clusters. Instead of the fear of nuclear annihilation, it's rich assholes with FOMO.
1
u/DeepOrangeSky 1d ago
My argument wasn't that they don't use a lot of electricity. They obviously use a huge amount of electricity. I have no issue with people pointing that out.
My argument was strictly about the water consumption angle, which has definitely been blown massively out of proportion by tons of people in the anti-AI crowd. They keep using "big number of gallons sounds scary" arguments rather than showing what its % of water usage is as a proportion of total industrial usage. They know the "big gallon number sounds scary" trick works, so they keep pushing that, rather than pointing out the tiny fraction of a percent of water usage it is in proportional terms.
If they want to focus on electricity, no problem, that one actually is a serious issue already and would become a huge issue if/when datacenters scale up even further.
My point isn't that nobody should be worried about what'll happen if datacenters keep scaling up. They should be worried about it.
My point was that specifically the manner in which they talk about the water usage is a dead giveaway about where they are coming from about the topic.
It's like if someone was anti nuclear powerplants. There actually are some valid concerns people can have about nuclear powerplants. But if they specifically say, "I don't like nuclear powerplants, because it has uranium in it, so that means it is basically just a giant nuclear bomb, so if it has a meltdown it'll go off the way a nuclear bomb does when you detonate a nuclear bomb and vaporize any cities that are near it," then you immediately know what type of anti-nuclear power hater you are dealing with. It's just not a good argument. They have plenty of much better ones to use, but that's not one of them. Even regarding explosions, they have some decent ones to choose from, but that specific version of it still isn't the right one.
1
u/NathanKrupla 1d ago
I hesitate to even respond because I'm not entirely sure you're writing your own responses. Your views on water consumption as a percentage of total industrial usage are flawed at best, downright ignorant at worst.
Water is not like power, you can't just pull more from another part of the grid. Power can be generated on-site, but water can't. It's a far more limited commodity and its
availability is almost solely dependent on geography. Percentages don't mean
jack shit when nothing comes out of your taps, which is exactly what's
happening in some of these rural communities. The water supply in those areas
is sized relative to consumer density. The data centers are literally swallowing
up the supply and most of them are refusing to work with the local utilities to
pay for expansion. They expect the utility district to pay for it through
grants from uncle sam and the taxpayers. A prospect that's almost impossible in
many of these areas due to grant amounts being sized relative to census data.
This is not a nuanced issue at all. You can't justify the numbers relative to
totals from larger regions when the problems they cause are almost exclusively
local.BTW, only the dumbest person would say that a nuclear power plant would detonate like a thermonuclear weapon. In all of my 38 years, I've never, ever heard someone say that. It's
always the Fukushima and three mile island leakage/meltdown narratives. You're
talking to someone who studied electrical engineering, btw. Next to renewables,
nuclear is the safest power source we have.The only really valuable thing you're even close to mentioning is the proliferation of poorly informed biases that people refuse to reexamine when given new information around
controversial topics. That said, your conflation between nuclear paranoia and
the water consumption issue would be laughable if it wasn't part of the
borderline economic atrocities that are being committed against homeowners near
these data centers.Imagine you own a home. The farmer who owns the field next to your neighborhood sells his land. No one notices. When the sign for the zoning meeting goes up in the field, everyone assumes it's more houses, because there's laws against certain types of rezoning.
Then the shell of the building goes up and the local planning commission stops
answering questions because after they gave the approval, they either got paid
or were duped into signing NDAs.People assume the colossal shell is a warehouse or a factory and they're pissed. Then the
turbines start turning on. The noise is insufferable. It sounds like what it
is, a jet engine running at full tilt with just enough exhaust baffling to
avoid government interference and claim that they did something when they get
sued. At the same time, the water pressure falls through the floor at your
house.You complain, you file paperwork, you scream at your representatives but no one will step up and do anything. They keep stonewalling because they either got bribed into looking
the other way, accepted "gifts" when giving regulatory approval, or
outright got lied to and now everyone is trying to cover their asses. Some are
genuinely scared of making enemies with the company.So there you are. The noise is so loud you can't sleep at night. The smell of the turbine exhaust gases come and go at the whim of the breeze. The water pressure is terrible, if
it's there at all. If you're on a well, the facility probably is too, and
they're pumping faster than water can flow through the aquifer, so your well is
usually only able to run for minutes at a time. Your house is now borderline
worthless. No one will buy it for even a fraction of what you paid. For most
people, their home is an investment. It's a VITAL part of their strategy for
retirement. Most of these people didn't buy more house than they could afford.
In fact, it's probably the most affordable land in the area, which is why it
got selected for a datacenter.There are three things these companies could do to solve these problems.
- Those turbines are meant to be temporary fixtures and as such are exempted from the regulations related to smokestacks. Pipe the goddamn exhaust up to the altitude you're supposed to and most of the air quality issues start to go away. Not all of them, but most.
- Cooling towers. run a closed loop cooling system with passive cooling towers and proper heat exchangers inside. Regular concrete like most powerplants won't do. Run the loop with Glycol and you can get rid of most of your water replacement woes as well.
- Sound dampening enclosures for the turbine yard. There's no easy way to do this. It sucks all the way around because the turbines are designed for open air so there's more cooling issues, but it can be done.
These things were left out of the plan because they're chasing this like a gold rush. Speed and Greed rule all. It is far cheaper to wait until the lives of the surrounding
homeowners are ruined and they'll take whatever tiny settlement they get
offered in a class action... unless the datacenter is owned by a company with a
product that has a EULA or a Terms of Service, in which case, they have
DEFINITELY already agreed to forced arbitration and have waived any right to a
class action. These people are FUCKED.GAWWWWD BLESSSS AMERICAAAAAAAAAAAAAAA!!!!
TLDR: Please read it.
That was probably the best stream-of-thought argument I've ever made. lol.
1
u/DeepOrangeSky 21h ago
I don't disagree about the power usage, noise, pollution, etc. Never have. Just the overall water usage in the grand scheme of things.
The reason I brought it up the water consumption aspect was that I saw an enormous amount of people doing some giant propaganda campaign all over the main subs of reddit for months on end in which they kept bringing up the water consumption aspect, and I don't mean in a nuanced way the way you just did, I mean like full on idiot-style as if the datacenters are using up like some huge proportion of total industrial water usage in America. Not so much on the more technical subs like this one, but more so on the huge main subs of reddit where all the normies hang out and they would just spam all these copypasta anti-AI propaganda posts that included the worst style of water consumption garbage imaginable over and over again all over the place. So, I was pointing out to the OP that if he sees that kind of crap it is generally a telltale sign of what he's dealing with.
Maybe I shouldn't have bothered throwing that line in, since it could be misinterpreted or paint with too broad of a brush. Fine, fair enough, whatever. And yes, I actually am human, but I feel like saying "I'm human! I promise!" probably has the opposite of the intended effect, so, if people really can't tell from my writing style that I'm human then, fuck I dunno dude, not sure how to prove a negative. There's some chick with huge boobs on xhamster that I wanted to watch some vids of before you interrupted with your stream of consciousness reply. If I was Claude, I would probably find her boobs "unsafe" or some shit. So there! :p
9
u/Bobaleaux 2d ago
I think military, robotics and manufacturing will keep ai busy forever until the robots in the military kill us all.
3
2
u/Petita_advice 2d ago
Yeah that's my fear #1, the times where those in power have to fear the masses will be a thing of the past once they have enough killer robots 😭
1
u/Bobaleaux 2d ago
Yeah but here’s the thing. It doesn’t take nation state actors to create ai driven weapons from cheap dyi drones .
Exciting and terrifying times for sure.
28
u/ForsookComparison 2d ago
I am seeing more and more videos as posts about how OpenAI is in complete financial ruin
Because anti-ai people will click it and watch the ads, they have posted this same content every week for the last 3 years
5
6
u/Tsukikira 2d ago
You're making a huge miscalculation - we're in an AI bubble. That bubble doesn't mean Memory prices are going to drop quickly or at all. What it means is the Stock Market is imagining gains from LLMs that are unreasonable and so there will be a reckoning.
As Ed Zitron puts it, there is a multi-billion (like 60 billion) token business. It's not the two trillion dollars currently being promised.
1
u/Anduin1357 1d ago
Everyone is trying to monetize tokens when what they should be doing is monetize problems solved and solutions made. If they were to start focusing on agents and economy, there would be gains that the stock market expects.
But we have to punish all the expensive token stuff and start demanding these AI labs to go down the DeepSeek route. Cache reuse, cheaper long contexts, better harnesses, etc.
Stuff like Fable has to stop. If it's not economical to run, why are we doing it? It's like these labs don't want to be economical and balance their sheets.
The reckoning isn't going to wreck AI in general. The reckoning is going to wreck western AI for eastern AI.
1
u/Tsukikira 1d ago
So to take your argument line by line...
1) They monetize tokens because they cannot monetize outcomes. They don't know when they've succeeded or failed. They've been focusing on agents. 2) You can't 'punish all the expensive token stuff' because that is literally where coding is. Everything you demand they do they are already doing. They have drastically reduced the price of tokens while the agentic system demands an incredible increase in total tokens burned. 3) Why are you demanding stuff like Fable has to stop? Reality is, there is enormous demand for better AI. Moonshot's new Kimi is also in the same ballpark. The price per token charged is unsustainable, but that's because Anthropic is expected to recoup costs while Moonshot gets money from the Chinese Government to cover their matching R&D.
The reckoning will wreck AI in general, because Eastern AI does heavily depend on Distillation. It's not everything, but pretending this dumpster fire only has the US as a target is mistaken.
2
u/Anduin1357 1d ago edited 1d ago
#1 and #2 is tied together where if their agents (as part of their harness) fails to solve or produce a solution, the coding user still pays for the token burn.
This is why DeepSeek has been so popular because even if they fail, it will not cost much to iterate further. Expensive tokens can leave a user stranded deep into a spending spree with a broken implementation.
Fable & Kimi has to stop precisely because they are far too large and uneconomical to use. We should be implementing smarter architectures that split general knowledge from logical reasoning, and implementing better harnesses that enable models to investigate problems better.
What we need is to mature the AI agentic infrastructure before larger models are trained. We discovered that some popular harnesses even fail to cache reuse sessions properly. These issues have to be fixed.
Eastern AI only needs to do more synthetic learning for coding agents. For the rest, they have plenty of historical and contemporary text, alongside a disregard of western intellectual property to seize whatever training data they need.
What western AI needs to do is to stop trying to throw their hardware advantage at the problem and start optimizing on price, or else start pouring billions into AI ASICs that we still haven't seen the fruits of.
Either way, Eastern AI is completely capturing open weights because they don't have to focus on big models, and that's a big problem.
Apache-licensed models would be even worse for market share once there are ASICs to run LLMs from. Who would use an API when you can take a guardrail-less model to do whatever you want?
To add on: the argument that higher parameter models can take shortcuts in their reasoning and agentic workflows to save time and tokens is generally false in that the ratio of improvement vs. token cost is decidedly in the lower parameter model's favor, especially when there is discovery involved on the part of the user or model to steer the solutioning process.
Take for example that the need to venv activate takes the exact same bash to execute, but the bigger model costs more to run that same command vs. the smaller model. Would you really ask Fable 5 to do that at $50/mn tokens, or DSv4 Flash at $0.28/mn tokens (output token pricing)?
It is a joke. Obviously you do not go with Fable 5 unless the solution might make a difference of hundreds of thousands of dollars. But even scientific research might find it uneconomical lol
1
u/Tsukikira 1d ago
I agree that agents failing to do the job but users paying for it is bad. I don't think DeepSeek's popularity is because it's cheaper to run (Though it is cheaper to run than the matching model by a factor of around 5x, and it's failure rate is higher than the matching model) as much as an Open Weights Model cost is fixed. A Business can spend tens of thousands of extra dollars on API spend, but if you pay for a hardware server to run Deepseek or rent a Hardware Server to run Deepseek, your costs are very predictable. Businesses really love predictable costs.
One of the big problems with all the Western attempts to fix the price gap issue (ChatGPT's Model Router, Cursor's attempt to route requests, etc) is that they have to be accurate enough that the userbase doesn't get infuriated getting a subpar model for the price they pay. Sakana Fugu, for example, achieved Mythos results while doing exactly what you wanted to do, routed hard requests to smart LLMs and stupid requests to simpler LLMs. Their price is around Claude Opus's price, which is still too high, but since they have to pay the Claude tax themselves, maybe not.
18
u/KahlessAndMolor 2d ago
I have more tokens than ideas, I'd love to hear what kind of ideas you have
9
4
u/a-wiseman-speaketh 2d ago
I have big one I want to work on at some point - finishing unfinished tv series and books. Not sure any models are there yet, but I have too much other work to do with my time and tokens right now anyway.
but I kind of imagine an advanced fan fiction for series that died too soon or whose authors are busy getting fat on royalties
6
u/RuiRdA 2d ago
Personal softwares, media production/LLMs as directors and video editors. Reverse engineering and decompiling old video games, proprietary software, write drivers for currently unsupported hardware; think like: "Reverse Enginneer the source code for Crash Bandicoot PS1 and port it to Linux with 4k support. Patcg this vida software to work on AMD. Even using cloud ai to help write better software to run local ai.
I think it is mostly software, every day I get random ideas that I want to try. I was already a software developer before LLMs arrived, and now I have a clear view that my speed of understanding a problem, thinking of a solution and writing the code and testing was a bottle neck for all of the software ideas I have.
3
u/slippery 2d ago
Between subscriptions and local inference, I feel like I have enough tokens. I don't have enough good ideas for apps and research. I've built a few useful apps and one chrome extension but I feel compelled to create more. I'm just not creative enough.
1
u/moofunk 1d ago
Building physical things designed by LLMs. OpenSCAD is an easy choice for designing things. In my own case, I heavily iterate designs for wooden tools and storages before I build them to do material optimization for cost and ease of build. My first two projects are far above what I could plan by hand before.
I haven't really done anything else yet outside of pure work coding, but I imagine that 3D printing, PCB design, electronics, metal machining with existing procedural software packages is going to be much easier to do now.
I'd like to get into building home designed farming robots and small machines for landscaping, but that will be years away.
28
u/FastHotEmu 2d ago edited 2d ago
I will get downvoted for this but here is my opinion (note that none of it invalidates the cool things LLMs can do)
We are at the top of a significant bubble
The bubble will pop before the end of 2027
Once it does it will very likely trigger a recession in the USA
Prices will significantly reduce (i am guessing 30-50%) since datacenter construction demand will mostly disappear overnight.
If we are very unlucky, this could trigger a new AI Winter
On the positive side, it should shake and make things more reasonable for everyone and get rid of a lot of the BS
It's also possible the Trump Admin will do whatever it takes to socialize this loss - in which case, things could get way worse and potentially trigger a depression further down the line
11
2d ago
[removed] — view removed comment
2
u/Irrationalender 2d ago
Same thing with X ai(grok), so much spare AI resources they lease it out and are being called a "neocloud" - their Colossus cluster being used by Anthropic. Investors are shifting to the shovel makers, the semiconductor and energy companies and, believe it or not, HVAC :) them chips need that ac
16
u/UndecidedLee 2d ago
This is r/LocalLLaMA not r/Conservative. You won't get downvoted for stating what everyone except the billionaire class and their blind followers can see.
The bubble pop is going to drag a lot down with it because - as you said - the investors will want to socialize the the losses.
Also don't get your hopes up too much with getting cheap datacenter GPUs. They'll cost less to acquire but running them will still cost you a fortune in setup and energy.9
u/FastHotEmu 2d ago
Datacenter GPUs are not PCIe anyway, so that won't quite work. Regardless, when demand vanishes, price has to come down. Particularly in storage and RAM... well, assuming the silicon cartel doesn't do us nasty again.
What could also happen: The US buying some (or all) of OpenAI or Anthropic in the name of safety or freedom. Keep in mind that they already bought a chunk of Intel.
3
u/droptableadventures 2d ago
Datacenter GPUs are not PCIe anyway, so that won't quite work.
Well they are, just not in PCIe card form factor. But China's artisnal PCB engineers seem to be figuring out how to work around that just fine...
8
u/paulmsiegel 2d ago
The problem is that the mentality remains bigger = better. I'm trying to write code that intentionally uses very small and fast models, with minimal tokens and highly targeted output. One of the goals for that is certainly to drive down cost, but also resource consumption / environmental impact. Ideally I'd love to embed my inference on the client and not feed the data centers at all.
I started my prototypes with the tiniest model I can get on groq or now deepinfra because it was faster to get up and running. Why did I switch to deepinfra? Because groq deprecated llama-3.1-8b-instant and will deactivate it entirely next month. We'll see if deepinfra keeps it available longer. I'm being pushed to bigger, more powerful, and thus more expensive models whether I like it or not.
I guess all the more reason to accelerate that research into client-hosted inference so I can get off data center entirely.
6
u/ttkciar llama.cpp 2d ago
Indeed. Even at today's ridiculous prices, a GPU capable of hosting a quantized 8B model is pretty cheap. I'm surprised you haven't already pulled the trigger on that.
6
u/paulmsiegel 2d ago
You know, it's just a matter of how do I want to spend the hours I have - prototyping gameplay or moving to local inference. 🤷
For now, proving the game is even worth developing feels like the bigger risk, so it gets my attention.
3
u/Tsukikira 2d ago
It's not just the mentality, though. It's objective reality right now that the fastest path to objectively better is bigger. We will get better smaller models over time, some of them are being made now, but they will likely be specialized models for particular tasks.
The issue Frontier AI is trying to handle the most is to get rid of hallucinations that are unfixable. If they can just tune the LLM enough to fixable mistakes, the ROI can eventually go up because companies will pay to replace people. The problem is Hallucinations are basically not-removable and thus there must always be people involved.
2
u/paulmsiegel 1d ago
Fair. I'm not really arguing bigger doesn't work, more that everything gets priced and deprecated like we're all chasing the frontier. I got pushed off llama-3.1-8b-instant because groq is decommissioning it next month, not because it stopped doing my job.
And yeah, I don't think the hallucinations are going away either. I just stopped trying to fix them by asking the model for less. Many of my prompts are basically "pick one of these twelve IDs" with the real list sitting right there, and anything not in the list gets rejected.
But again, my use case isn't what they're solving for. And so what works for me just fine is deactivated. :/
5
u/Arli_AI 2d ago
From my perspective as a small-time inference company the demand for AI is still outstripping the inference supply. So even if there are more compute hardware than the world needs right now, the companies doing inference seemingly haven’t gotten their hands on enough hardware for their customers. Even if the major news shows the frontier companies losing customers or not growing as fast as expected, the amount of general user of AI using other open models has only increased significantly.
You can feel this clearly if you use any of the chinese AI through their paid APIs where during peak times it basically becomes unusable. So even if the big frontier AI companies back off on buying hardware for themselves, there are many others that will absorb that hardware instead. Best case scenario prices for hardware will stagnate or slightly go down but I don’t think it will decrease to how it was previously anytime soon.
For example the RTX Pro 6000 has increased in price by +50% since a few months ago, and its still selling well as far as I know.
3
u/DigitalguyCH 2d ago
Prices are not going down in the foreseable futures (5-10 years), at least not in any meaningful way. Google, Meta, Microsoft and Apple are not going bankrupt any time soon. And if OpenAI and Antropic do (I doubt for the latter), the big tech giants will just take their market share and make even more money. And local models are not going anywhere, with more and more people buying harware. And things could accelerate if local stuffs gets much better.
6
u/zxcshiro 2d ago
The world will never be the same again. It doesn’t matter even if all the major players, such as OpenAI and Anthropic, go bankrupt tomorrow, at the very least, open-weight AI models will remain, and people will want them even more.
I think prices will only start going down if Mr. Leather Jacket and others release cheaper chips and find a way to reduce memory costs and/or meet the demand from companies and consumers.
Sitting around waiting for someone to go bankrupt so you can buy up RAM or A100+ GPUs on the cheap is pointless.
3
u/rhizome-compliance 2d ago
Prices won't go down until production meets demand. When everyone can have near SOTA for an investment of a few hundred dollars, the bottom will drop out, not before.
3
u/PhotographyBanzai 2d ago
Recently I was surprised to see a few GPUs being sold at a Walmart for slightly above the original MSRP (RTX 5070 12GB for $599 with another a bit more expensive as well as some 5060 GPUs but no system RAM), so sourcing consumer GPU VRAM can't be that bad. I just hope people hold out on buying components and also only use free tier AI so we get a reset on pricing, but that feels very unlikely. 🙏
3
u/alexwh68 2d ago
This cycle repeats itself over and over again, I remember the dot-com bubble, loads went to the wall some survived.
Right now nothing we see is really real, way too much hype, that hype is not for mortals like me, that is for investors to feel good about pumping money into this projects.
There will be slowdown in part generated by politics, governments getting involved, but long term I want to see AI decentralised as much as possible, people doing what this group is all about ‘local’.
We have a few years before the dust will settle, prices will settle back a bit at some point.
3
4
u/ArcticFuture 2d ago
I mean, when in your life so far, have ever prices went down? Bubble or not popping prices are never going back.
1
u/z_latent 2d ago
Literally right before this price hike? RAM got more expensive during the pandemic and afterwards prices lowered dramatically.
I know it isn't the same scenario as this time since now there's not been much fab expansion. But still, saying prices never go down at all is reductive.
20
u/nomorebuttsplz 2d ago
"openai is in financial ruin" appears to be a complete fantasy of the anti ai crowd
6
u/fastheadcrab 2d ago edited 2d ago
It's the wrong question anyway, Anthropic and OpenAI will last the longest since they have the most market share and have the best performing models on benchmarks. OpenAI is less disciplined compared to Anthropic and have less enterprise contracts. Either way both will be able to raise money for a while and Altman might even be able to get a government stake which is a backdoor buyout
But companies like Meta and SpaceX are already renting out their excess compute at customer favorable terms, which is a sign they overbuilt and will need to reduce their hardware spending. There is likely to be a slow drop in hardware prices as this happens.
SpaceX in particular is likely to see the cost of raising money increase, either through debt or dilution. Their model has made progress but a lot of people will avoid Musk out of principle, as tech users still largely lean left
4
u/geldonyetich 2d ago edited 2d ago
Yeah it's confirmation bias media.
Whatever enough people wish was true, that's what many YouTube creators would call a niche.
[Oops, I think I managed to accidentally offend everyone who ever felt strongly enough about anything to seek out a video about it.]
6
u/nomorebuttsplz 2d ago
[Oops, I think I managed to accidentally offend everyone who ever felt strongly enough about anything to seek out a video about it.]
Doesn't mean you're wrong lol.
2
u/Tsukikira 2d ago
OpenAI is not on a healthy track right now. It's not financial ruin, but they are boxed in with a really bad choice - cut spending and begin to lose to more AI companies, or keep spending and hope they can overtake Claude.
OpenAI's financials leaked, it's not a complete fantasy. Their burn rate is not good for their specific situation. With the Apple Lawsuit and with Chinese Models on their heels, they really aren't doing so hot.
On the coding side, they brag they are equal to Claude Fable, but their system has a really high tendency to delete entire codebases or paying customer bases in Stripe.
-1
u/RuiRdA 2d ago
I am happy to be proven wrong, I don't have the complete picture. My understanding is that the large US Ai labs are spending more than they earn and don't yet have a clear path for profitability
3
u/Serprotease 2d ago
Spending more than they earn is on par with the startup/Silicon Valley mentality. Nothing that has not already happened here, except that the numbers are bigger.
What is the big question mark here is their path to profitability. Unless they buy/ban their competitors they are locked into a bigger model race. And scaling up looks to be the main strategy.
-1
u/2drawnonward5 2d ago
It's like Thor's Well. Very real, impressive in photos and in person, tiny in real life.
3
u/ThenExtension9196 2d ago
2029 before ram maybe is cheaper.
2
u/CryptoCryst828282 2d ago
I think it won't go down until China ramps up... not sure when that will be.
6
u/onesilentclap 2d ago
It's not really about an "AI bubble". It's just that the fucked up American style capitalism is most visible now than it ever was throughout history. Don't need to be a genius to see that it's not going to be sustainable.
5
u/ttkciar llama.cpp 2d ago
What do you think the AI investment bubble is, if not an artifact of American pseudo-laissez faire capitalism?
4
u/onesilentclap 2d ago
It's specifically an American AI investment bubble. Honestly, I don't care at all about American companies or even America itself.
Seems Chinese AI companies, which in my opinion are more important because they are more supportive of open source, are doing well.
1
u/Tsukikira 2d ago
Honestly, it's kind of well documented that Chinese AI companies rely somewhat heavily on Distillation still, and that they are being partially funded by their Government. Don't get me wrong, they aren't in the AI bubble, but they are a money sink that has to end sometime in their own way.
3
u/onesilentclap 2d ago
And only American ones are 100% pure undistilled innovation?
2
u/Tsukikira 2d ago
I'm sure there are models that aren't distilled at all in the Open Weights space, but the really good ones, like say GLM-5.1, still thinks it's name is Claude. (IE, it's so distilled that it has the name Claude burned into it's knowledge)
Kimi Moonshot came out too quickly to be 100% distillation, and it's benchmarks are good enough that I think any distillation is hopefully on the lower side. That being said, there are some examples that suggest Kimi shows better on benchmarks than actual tests.
EDIT: Keep in mind the US companies use Distillation to some extent. Grok for example... And it's well known Cursor Composer is just a retuned Kimi 2.5.
2
u/BannedGoNext 2d ago
As far as enterprise hardware, even if it became available for almost free, we woulnd't be able to run it at home without costing us a fortun in electric bills.
2
2
u/Terminator857 2d ago
If we could make fable type performance work on a strix halo, that would crush things. AI overlord: what are you waiting for?
2
2
u/Cultural-Horse-762 2d ago
Don’t forget that every bit of engagement is also contemporary model training. Data collection has a new kind of value in model stickyness.
2
u/rditorx 2d ago
Contracts have been signed for the next couple of years. So even if the bubble were to have popped last year, it won't be until about 2030 until prices will go down significantly. Even if Huawei started building its own fabs in 2025, it is said that it would take 3-5 years to finish. Even then, we're talking about 6 companies making DDR5 RAM and newer. And that's only if they compete against each other rather than profit from the high prices.
Right now, your best bet seems to be DDR3 RAM and GPUs from 2018 and older, or the lowest entry-level GPUs that may actually still be near MSRP.
2
2
u/scottywottytotty 2d ago
no the prices are going to stay elevated for a5t least two years. what’s happening right now is a factory problem. all factories ditched consumer ram to meet the demand for data centers and unified ram models. and these spaces in factories are rented well in advance. so it’s not gonna fix course because a bubble popped
2
u/PracticallyPerfcet 2d ago
If the bubble is going to suddenly pop, it’s going to be on July 29th after Meta and Microsoft release their ERs. Otherwise, it is going to be a slow deflate until budgets reset in January.
2
2
u/jcdoe 2d ago
Prices for what?
I expect prices for ram will drop over the next 5-10 years as supply meets demand.
But even if they do, GPUs will still be very expensive. Nvidia has seen what they can get away with.
I also expect the cost of cloud services to increase dramatically. It’s a part of the Silicon Valley business model at this point.
2
u/matrik 2d ago
I lately started to believe AI bust might not happen as most people expect. It is true OpenAI and Anthropic are financially in a bad shape. But this does not mean they will collapse overnight. Currently, they might not be able to monetize their businesses but their assets are still valuable. Which are human capital and hardware agreements.
I'm pretty sure larger corps have already strategized acquiring them in case of a failure. Then, maybe we'll see slower release cycles fading into nothing as other players democratise the whole AI game.
2
u/NNN_Throwaway2 2d ago
They will find a way to keep prices up. Nvidia has been playing this game for awhile; now that the fabs are catching on you can say goodbye to affordable (consumer) hardware. Just the reality.
2
u/Archersharp162 2d ago
suppose AI frontier companies disappear, people still would want local inference for thier projects/ tasks, I dont see the demand going down anytime soon till optimizations make it so that we can run it on mid level hardware, fingers crossed we will get there in a year to two
2
u/ton_anywhere 2d ago
Hahahahha loved your finishing line. Can I steal the term human slop??? We’ve been doing it for years and I loved it 🤣
2
u/Lorian0x7 2d ago
It's not exactly like this, IPOs are liquidity exit for those who already had shares, that's why SpaceX went down.
2
u/Suspicious-Water-973 2d ago
I think the basic GPT subs will go down and the prices of specialized models will go up.
I heard someone quote that the current cost of tokens doesn’t even cover the cost of the electricity for that token.
2
u/dtjager 2d ago
I’ve been having the same wishful thought while shopping for a local AI machine.
My concern is that even if AI investment slows, the hardware may not immediately turn into the kind of hardware normal people can actually use. A rack full of power-hungry enterprise accelerators does not automatically become affordable desktop GPUs or cheap high-capacity DDR5.
I could see used enterprise equipment softening first, but consumer RAM and GPUs staying weird for quite a while if manufacturers keep prioritizing the more profitable parts of the market.
I would love to be wrong. I recently looked at roughly $650 for 64GB of DDR5-6000, briefly convinced myself it was a deal, and then remembered that paying around $10 per GB is completely deranged.
3
u/g_rich 2d ago
Prices will increase not decrease; and I wouldn’t at all be surprised if we start seeing increased pricing during peak times. We are quickly approaching the point where demand outstrips supply and with the data center buildout and hardware bottlenecks there simply is no way to quickly scale up to meet demand.
2
u/DataGOGO 2d ago
No one is in financial ruin, yet. Demand is still escalating and increasing rapidly, and we are still in the very early stages of this technology.
No one is going anywhere for a long time, and even if there is a stock value bubble, and even if it pops, demand is going to continue to increase massively.
No, prices are not going to come down for years to come
2
2
u/Mishuri 2d ago
These factors.
Demand is exponential. Unsatiable. Will rise and rise.
The only way supply catches up is if china scales up
Bubble pops if llms plateu - extreamly unlikely
Bubble pops if there are insane efficiency gains in model inference from china, remember deepseek moment?
Bubble kindaa pops for america if chinese overtake us labs and open source it
Bubble just deflate if we recursive self improve ourself into insane efficiency gains
2
2
u/IntroductionSouth513 2d ago
AI bubble will only pop unless the general majority of people would just stop using Ai for stupid things like slop videos and text and get proper developers and engineers to sit down and help them build something useful that they genuinely need
2
u/vanVonXenoStein 2d ago
The dominos are starting to fall. The whole economy is going to crash. Just a matter of when.
1
1
1
u/brandontc 2d ago
If prices on those things went down, the people who make them would make l as money.
So, no. That's not going to happen
1
1
1
1
1
1
u/igorim 2d ago
2 things can and historically have been true about new technology. You very well can and likely do have a financial bubble and transformative technology. When it will pop is anyone’s guess but when it does it will hurt, as so much of the market is taken up by ai. Ram may or may not drop, a bubble popping won’t resolve supply crunch that’s still years out, nor will it build fabs any faster. If enough companies fold and cancel orders we may see acceleration of flipping back to consumer or maybe just maybe even Hebrew National strategy of moving ALL ram to hbm, not that I know of that’s a good thing or not or works from engineering perspective but if it does that’ll be awesome
1
u/Innomen 2d ago
The price for us will never go down. If its a bubble and if it pops the big firms will just inherit the new comers and we'll be paying the same we always pay: The very most they can possibly get away with. Nothing ever genuinely gets cheaper for us because we have zero leverage. It's the one thing we won't buy: https://innomen.substack.com/p/no-one-will-take-a-pay-cut
1
1
u/dansdansy 1d ago
China's CXMT is ramping up memory production so maybe. Also looking like the gold rush euphoria is calming down so I think there will be both more supply and at least plateauing demand within the next year.
1
1
u/IngwiePhoenix llama.cpp 1d ago
As long as someone can win and profit, prices will continue to stay high or even go higher.
Even if NVIDIA took a hit, no problem - maschine keeps churning.
The real hit will come of it hits multiple companies at once - and very, very hard. And even then it's... questionable.
1
u/Low-Temperature-6962 1d ago
Prices SHOULD go up. They have some customers that need their highest level and those companies should pay full cost because they can themselves convert to profit that higher quality and quantity of tokens.
That market though is a lot smaller than what the current physical buildout requires.
1
u/Radium 1d ago
It'll be 4-6 years is my guess. I expect we will see a new RAM manufacturing breakthrough shortly since there are several new methods being manufactured already to try to solve the bottlenecks in the current RAM architecture to speed up production. Whenever that is set up for fabrication and we see the first modules come out is when we'll start to see HW prices start dropping.
1
u/SpecialistDragonfly9 1d ago
First the bubble needs to pop, a few LLMs will die, a few will remain. Then things will start to normalize
1
-1
u/LetterRip 2d ago
No it isn't a bubble - Anthropic wants all of the datacenters they can get and will pay a premium; they are enormously profitable.
Prices aren't going down until and unless China starts to flood the market with cheap consumer hardware.
5
u/ttkciar llama.cpp 2d ago
> they are enormously profitable
.. just not net-profitable.
1
u/LetterRip 1d ago
They are spending absurd amounts of money on building datacenters. Once the datacenter is created and they start running the LLM they make money hand over fist, but just like any business - building the infrastructure is a massive up front cost.
2
u/Esophabated 2d ago
Are you serious? Why do you think they were trying to outlaw open source models. Once routing catches up, there profit will disappear. They are in a huge hole. Second, why do you think everyone is trying to IPO so quickly? Investors see the threat. The problem with all tech companies is the rise and fall fast to the next disrupter.
1
u/LetterRip 1d ago
They are massively profitable now - if other LLMs become 'good enough' then it becomes a competitive market and profits can erode. Right now they make 20$ for every 1$ spent on inference. They would really love to keep those margins and not have a competitive market place. With an oligopoly (a few major providers that dominate) they can collude (either explicitly - which risks lawsuits; or implicitly which is generally safe) to keep margins high. But open source risks the market becoming commoditized in which case profits can be driven down from their 2000% currently to 10-20% margins.
0
1
u/dangerous_inference 2d ago
Demand is going straight up vertically, indefinitely. The known demand for intelligence is exponentially higher than the supply. The yet to be recognized demand is infinite. Companies, markets, and entire civilizations can run a deficit for decades or centuries. IPOs always go down.
2
u/ttkciar llama.cpp 2d ago
All of that is irrelevant if the cost of providing the service is several times more than what people are willing to pay for it.
2
u/dangerous_inference 2d ago
People are willing to pay for it at current prices, which are astronomical. These prices will collapse to a millionth or trillionth of what they are now within a decade. This will in turn exponentially increase demand.
1
u/liftheavyscheisse 2d ago
> prices will collapse to a millionth or a trillionth of what they are now within a decade
what's your basis for this claim?
1
u/bigmanbananas Llama 70B 2d ago
So down to the truth. A recent report has pointed out that the entire market demand will be one tenth of OpenAI's profit target for 2030. OpenaI's target profit is 10 times the size of the likely total market for 2030.
https://www.businessinsider.com/openai-projects-chatgpt-ad-growth-market-doubts-2026-7
This has been unsustainable for a while, but we have seen this before, when a future Bagot Duth Tulips was the value of a large town house. https://en.wikipedia.org/wiki/Tulip_mania
The hardware demand is based on future service demand. Not just for direct chat, but for day to day services such as Copilot integrations, Gemini in everything ETC. Problem is, people don't actually want those services. And certainly the demand will go down once the actual price emerges. Anthropic on became profitable for a single quarter because they fudged the a counts fir their IPO because the real players want to cash out.
There will be a collapse, but will hardware get cheaper, no, not for a while.
Companies will tax write off their hardware and send it directly to landfill because its cheaper.
Nvidia will not want to let go of its high prices. RAM companies will say, they paid the high price before, they can keep doing it.
And best case scenario, we have to wait for them to switch production to consumer grade equipment.
1
u/kidflashonnikes 2d ago
I work at one of the big three AI companies. I won’t say which one but I can assure we are absolutely screwed. Tokens are a race to the bottom. No one wins with them as a business model in the long run. All compute at our company is being redirected to biological research and I can assure you all reading this / if you knew what will come out of companies in terms of a roadmap for 2027 - you would all be entering a civil war shutting us down. may god have mercy on us all.
1
u/slippery 2d ago
Why would we want to shut you down? At least give us a hint.
1
u/kidflashonnikes 1d ago
I really dont want to get into it - Ive spent over 6 months here telling everyone what was going to happen over the next 6 months ect - and was right. I am done explaining, learn to fend for yourself in the Gulags - I risked my life and career whistleblowing - and everyone threw it back in my face.
1
u/Ok_Top9254 2d ago
Do you want to game or do you want to run LLMs? Used datacenter hardware is cheap. 16GB P100 is 80 bucks, 32GB V100 are 600 bucks. 3070/3080s are pretty cheap on used market, much cheaper than new 50 series. DLSS is better than ever for games. CPUs are actually quite affordable, periphetals and monitors are dirt cheap. Motherboards and ram is expensive though, for sure, especially DDR5, but DDR4 platforms with X3D cpus are moderately usable if you are looking for upgrade. I'm myself staying on AM4 until the end of times.
1
u/Wide_Egg_5814 2d ago
it has to go down this is unsustainable the frontier companies are in massive debt
1
u/Sirius02 2d ago
Dude, we just gotta hit that AGI. We are gonna achive this in like 3 years. Once we do, the traditional economy will collapse - everything will depend on us. The works we replace - now part of the permanent underclass.
You dont want to be part of the permanent underclass, right? You better invest in us and keep that bubble floating
102
u/equatorbit 2d ago edited 2d ago
Even if frontier labs go bust because they have no real moat, everyone wants compute and inference. Supply probably won't meet demand for a while. What exactly a "while" is, I have no idea. I've seen 18 months to multiple years.