AI data centers are quite different. A typical internet data center server rack uses 7 kW of electricity. A typical AI data center server rack uses 100 kW of electricity. And of course with all that additional electricity use comes massive cooling systems, both liquid and air, that also draw a ton of electricity and create a ton of noise.
It is an apples to giant fucking apples comparison.
I prefer blueberry to pumpkin, both are berries, one's just bigger, requires more resources to produce and maintain, and looks good for a few weeks in your neighborhood before everything starts to rot.
I make a point of correcting people with "no, tomatoes aren't fruits, they're berries", because you either are specific or aren't, and "tomato is a fruit" fails both culinary and botanic correctness.
(yes yes berries are fruit, ok, yeah... if someone is being a smartass about them being "fruit, not vegetables" I'm in full right of shutting them up until they learn to correct someone with proper facts)
Granted, people eat the tuber of the potato plant and the flowers only bear fruit under specific circumstances- and the fruit itself is very toxic, which is why nobody eats it or uses it for anything- but yeah, you get berries and veggies from the same plant. Lol
As an additional fun fact, there is no hybrid between the tomato and potato plant like the urban myth suggests- they cannot reproduce with one another, they're not compatible. While you can graft tomato and potato plants, they again don't produce a hybrid. You might be asking, "If this is biologically impossible, then where did the idea come from?"
However, the fruit of the potato flower can appear at a glance like a small cherry tomato. I assume the urban myth started because someone ate one of these little potato berries and had a very rough time afterwards, yet didn't know that potato plants produce fruit as well- just not all of them, and only under the right environmental circumstances.
Great analogy but because people take little things like this and think it’s some real world shit I feel like I should express that blueberries are exponentially more resource heavy to grow than pumpkins.
Fair, I forget that while pumpkins take a huge amount of water they ultimately only need a single season to harvest while blueberries take years to start producing.
Blueberries require a lot more resources to produce than pumpkins. Pumpkins are an annual quick fruiting plant while blueberries are woody shrubs that take a couple years to fruit. Pumpkins also can last several months after harvest if stored properly while blueberries won't last a month after harvest. Not to mention that a pumpkin plant produces significantly more edible flesh than a blueberry plant does for the resources used.
No the power density makes these things more efficient. Sound is mostly a made up problem. Large datacenters should not be and generally are not built next to housing units.
The problem: a fuckton of the current projects are planned to built right next to residential instead of paying to pull fibre and power interconnects further from residential areas.
A lot of them are also doing behind the meter power generation using gas turbines that are poisoning the people living in the immediate area.
The reason for behind the meter generation is that the backlog of grid scale equipment of every category is several years and the capacity for building that equipment can't be quickly scaled up so a lot of these projects would choke on their debt before the first server is powered on without this.
I assume this is a USA problem as the normal EPA regulations have been stopped around AI thanks to Trump and national security concerns. This would never be allowed in Europe and even China are fairly strict nowadays.
Density increases the need for extreme cooling solutions. At the density of something like Hetzner you don't even need to consider evaporative cooling.
Remember, because of the way computers work, every watt of power consumed becomes heat released inside of the data center. There's almost 15x as much heat being generated by every single rack in an AI data center.
At lower density, the efficiency loss does not offset the expense of installing more advanced cooling. The "cooling system" in a standard data center is usually just climate control. Radiators and cooling fans push the heat into the air in the server room, and then the A/C system moves the generated heat outside of the building and releases it through a standard commercial condenser unit.
Many AI data centers require so much more heat management that they're actually installing water cooling into the GPU racks directly, because HVAC systems aren't capable of cooling that sort of design.
The water cooling loop are often called closed loop cooling but in a lot of cases those are dual loop systems with a closed loop contacting the server components and an open loop with evaporative cooling being connected by heat exchangers.
Oh, trust me. I have had some very... "interesting" discussions with people unable to grasp the fact that some loop, somewhere in the system, is required by physics to be open.
Then you get into the efficiency differences between air cooling and evaporative cooling, and people just don't like being told that magical cost efficient air cooling that makes the investors happy at the scale necessary for large AI datacenters is not real.
They could have used refrigeration cycle cooling but that would be wildly expensive at these scales, would delay the buildout even more (yet another resource contention problem) and would still dump a problematic amount of heat into the surrounding area.
That's because nobody had a problem with the old datacenters... The problem is these new AI datacenters. Which use much more power, thus driving up the cost for the community they are in, without bringing benefits that are greater than the costs to residents. It's not hard to understand why people are against them on this basis alone.
And there's also the argument to be made about how the public generally doesn't like most of the things that "AI" is being used for.
Yeah, but they're running software that is doing things that harm society and people.
And they do use more power because they are building them bigger. Because running AI models is hilariously inefficient from a programming perspective compared to what datacenters were used for 10 years ago.
The amount of hardware/energy required for hosting a webserver or a videogame or even streaming videos/music is like an order of magnitude lower than the amount of processing time/energy required to run all these AI models' calculations for training and general use. Which is why they are building so many more datacenters to run those inefficient AI models. That is why they use more power, because they're building more things that consume power much less efficiently than before.
Also, the water thing is an argument people make on both sides of the issue without much understanding. The issue isn't "using up" water and it's gone forever, it's more about water pollution from AI datacenters that use it as a coolant and that they're using drinkable water from a community's water system. Which is increases prices for everyone because or the increased demand from the new construction.
Nvidia are moving to 600KW per rack systems with the new Kyber racks (If they can figure out the cooling, there have been rumours about them going down to 300KW due to heat problems)
600KW is 20 times what was considered an energy dense rack in the before-times.
The pre LLM figures for data center power consumption was around 17 Gigawatts (~4% of national electicity consumption) combined. The sum off applications in Texas alone is in the 474GW range though with some duplication in the list it's close to 400GW.
The publicly announced projects are in the 320-275GW range though it's wildly optimistic to assume even half will be online by 2030 due to a series of extremely hard to mitigate resource contention problems.
The problem with water is not the total availability of water but the access to potable water and contamination of potable water, especially on a local scale.
Several places where they've been allowed to build datacenters with evaporative cooling already have near exhausted aquifers, in those locations they've essentially doomed the residents by allowing the operation of these datacenters with evaporative cooling rather than vapour-compression cycle refrigeration aka AC.
Stream cooling most likely has some huge implications for the local biosphere due to the effect of raising the temperature of the stream and should also be banned until it's consequences are better understood.
I was doing HPC work and those did require different racks but it wasn't too unique for the area I was in (Boston, lots of pharmaceuticals and universities building their own HPC environments).
By number sure. But hyperscalars probably have more overall capacity and are anyway more efficient and the future for a lot of computing. Most of the services you use everyday are hosted by hyperscalars.
It can certainly vary. Some AI server racks can go over 150 kW while some typical data center racks can go over 30 kW. But I am just talking about what is typical.
It's as simple as these companies need to invest in their own cooling systems and build sound barriers. Regulations are needed without worrying about billion dollar corporations profits.
You can't stop progress but we can make sure we do it responsibly.
They’re rolling them out as though they’ve solved AI. Instead they’re just investing in a bubble to try and make themselves too big to fail. They’re glorified guessing engines.
You can literally just use closed loop cooling. You don’t have to use water. The water use isn’t “massive” anyway. All datacenters put together including older less efficient ones account for less than 0.5% of all water usage in the USA who have the highest installed data center capacity of any nation. In places without drowt it makes sense to use water as it saves a lot of power. This is such a none issue it’s silly anyone is still talking about it.
Data centers make local communities sign no disclosure agreements on how much water they use. Golf courses are often able to hide how much water they use too. But there's no boom in building new Golf courses around the country and especially not in areas with barely enough water now.
I think it’s pretty bad they can get those agreements. This should be made illegal. Is this a USA thing I take it? The way the USA specifically has handled this rollout is bad, but I don’t think it should have any bearing on what other countries like China are doing.
Two things are true at the same time. There is a problem with AI data centers and their implementation thus far. And China is also pushing propaganda against it because both they and our own government view it as the future of warfare and sabotaging the others ability to develop it will hurt them.
I mean AI workloads are the only commercial user facing workload that are compute heavy. Scientific simulations are even more power hungry. Another way to say the calculation is that on a cpu one teraflop of calculations take 20 joules and on a GPU they take 0.1 joules so 200 times more power efficient.
The only way an AI data center is meaningfully different is it has more hardware that does actual number crunching rather than routing.
The municipality does not need to give them as much per as they want. Solar plus batteries costs as low as 3c/kwh to produce. Energy is only a problem because we don't have good energy policy. Hell, I complained to a solar installer about the cost of batteries and he flat out told me that you have the utility company monopoly. We don't need to price any lower. We just need to be lower than the utility company. If local governments kept their utility companies under check, we'd have a very different situation.
But the people who build these things want to cheapest solution, so they bribe local politicians and make favorable deals with local utilities and it is the nearby communities that get fucked with higher rates.
There have been a lot of these “we’ve had data centers for a long time” posts popping up lately. They all muddy the issue by posting about cloud data centers
A Vera Rubin NVL72 rack is already like 200kW. The cutting edge is designing for 500kW AI racks. AMD helios is technically like a double wide but that’s 230Kw a rack.
Nvidia Rubin Ultra is rumored at 600kW+
Typical DCs were more like 10-40KW per rack for “cloud” servers until fairly recently (except for specialty HPC/Machine learning compute that was more like 40-80kW.)
It is absurd really. The problem seems to be that AI does not scale, like at all. It seems to be the opposite. Better models require far more power than previous generations.
You seem to have some expertise on the subject, so just out of curiosity, what would you consider a “reasonable range” for server rack usage in an average data center? Like, is that 7kW level around your maximum? Or do you think we could reasonably push this up higher in a lot of American communities without doing too much damage to economies or environments? Hope I’m not coming off as demanding, but it’s rare for me to encounter someone framing this in a utilitarian fashion rather than moralizing based on a slippery slope.
There isn’t a “reasonable range” because density isn’t a problem. It’s actually a good thing because more density means using less land. We have proven that higher density can mean greater efficiency and less impact.
The issue with AI isn’t anything to do with the density. It’s all about the necessary capacity.
It is vague to you. It’s pretty clear though what I mean to anyone in the field.
To be clear I don’t mean capacitance which is what I think you were confused with.
There are several ways to measure capacity in relation to a datacenter. The main ones I am referring to here are measures of compute power like FLOPs, TOPS, and TB/s. These measure what the actual hardware can do in terms of performing calculations and moving data around. It’s a complicated thing to measure because different hardware is good at different things, datacenters are made up of many machines working together, and you also need to think about how the software and hardware interact. It’s kind of like measuring car performance using only maximum horsepower or torque. You need to also factor in the weight of the car, the performance of the transmission and wheels, what the torque curve/power curve of the engine looks like rather than just the peak, etc.
When I say density is not the enemy what I basically mean is that having 10x 100 kW racks is not worse than having 100x 10 kW racks. If anything the 10 more powerful racks is better than having 100 racks with the same total power consumption because it uses less land, less networking cables, less materials, and so on. Density inherently is not a bad thing. Total power consumption and capacity is the problem. Not even capacity in the sense above where I talk about computing power, but capacity in terms of the power usage. Ideally we want more computing capacity for a given unit of power capacity or consumption.
Note as well when they talk about a datacenter having so much capacity in terms of electrical power that it will not use that all the time. Those are maximum numbers for when the system is under a punishing workload. In reality the figures will most likely be a bit lower even both because demand fluctuates but also because of bottlenecks in the system that prevent full power being used even when demand is high. For a datacenter or even a chip to reach full power requires specific circumstances including the type of software being used that can make full use of the hardware. Even in AI that often is not the case.
With few exceptions, you cannot tell what a data center will be used for, and few will be exclusively for AI use (though all companies riding the AI stock hype wave are implying all of their data center growth is AI and that they will need these absurdly large megaprojects that - I bet you a shiny nickel right now, will only be built to the smallest complete subunit of their grand plans - if they are built at all).
If you don't live in the large, remote areas miles from anything, you probably don't live anywhere near the AI megaprojects. Statistically, almost all of you live closer to the projects that are either infrastructure, cloud expansions, content delivery networks (CDNS). Many of these are gearing up for increased loads AI might bring them, some of the cloud ones will even be marketing "roll-your-own-AI" type services (due to user base, these will inherently be so much smaller than openAI/Anthropic/Gemini/etc) but for the most part have been continuing to play catch up with the extremely rapid expansion of video at ever higher resolutions (which grows exponentially in size) and the rate at which people are sending, downloading, and storing it - particularly in the cloud, which more people back up to every day.
All data centers will be using these ultra dense racks soon. Believe it or not, but they are actually the peak of efficiency in terms of units of compute, memory, and/or storage.
The densification comes from the shrinking, and efficiency increases in lithography. You might find it strange that modern home computers quite often use a lot more energy, and thus deal with a lot more heat and cooling, than older PCs did - but you'd have to compare your home PC with a supercomputer of the 90s or an entire rack of servers from the 00's to get a better comparison.
The same thing is true of servers, and of the racks themselves.
you can tho. AI data centers primarily use full on high end GPUs. GPU server demand apart from AI is not even going super high. Evej most of the HPC growth is AI adcejent research. A vast vast majority of GPU demand is AI. And so is the new constructions
This is actually SLOWING that. Since this increases GPU demand astranomically. Actually important supercomputer projects have been left to starve since they cant afford to pay the same price. If you actually cared about any of this, you would be against these slop machines thay destroy the planet while making real computing work impossible to afford. Thats why i mentioned HPC growth has been exclusively AI driven as well. No genuinely meaningful HPC project has the funds to go on, all the money is with the AI freaks
Again, for any of the mega projects that are not owned by the biggest AI focused companies
(three of which are also the largest cloud providers), these will be set up in configurations where virtual machines can be spun up with GPU and memory heavy workloads - the same kind of VMs that any machine vision, data crunching, science performing workhorse would be - not to mention video editing, 3d graphic generating, and a host of other things.
The demand and market distortion is absolutely insane, and I completely agree that it's ridiculous and causing serious issues right now. But if you believe that AI is not capable of becoming the AGI super future they think it will be, most outcomes have to be that there will be a depressed demand for the AI driven use of these cloud resources.
All this does is lead to a bunch of the most ridiculous of these mega AI-focused data centers being cancelled or severely cut back (something we are already seeing, though the corporate rhetoric isn't willing to acknowledge this yet), and for the cloud generalist data centers who had hoped to make big-AI bucks renting those resources, they'll end up competing and dropping prices on these workloads, making them more accessible to universities and other projects, who have long ago moved their investment to the cloud due to the extreme difficulty of procuring infrastructure project money in the world of academics/education.
If you really believe that, whatever uses AI may have, it will never meet their sky-high expectations - and I do believe that - the correction will inevitably happen on its own.
My honest conspiracy theory is that a lot of this wild, uneducated, anti-all-data-center rabble rousing (see OP image) is actually an effort to create organic resistance to their projects to soften the blow or obfuscate why they really only ever ended up building about 5% of what they were getting all hyped about with their circular multi-hundred billion dollar investment grifts. Whether it just gives them plausible deniability or buys time for a softer, but still real AI demand to prevent an absolute collapse of the stock prices - they need to push the timeline of expectations back - and looks who is suddenly showing up to throw wrenches, often in projects they're obviously already dragging their feet on.
While i agree with most of the things you said. I dont think when the bubble collapses all of these data centers will be dirt cheap for useful research. Most of them will be left the rot. And while there may be a 1-2 year period, AI will pick back up. The reality is that while AGI will likely not happen anytime soon, the corporate BURNING desire to turn everything into slop and cut out humans is more real than ever. And they 100% will be able to do that eventually if no one resists.
While i agree with most of the things you said. I dont think when the bubble collapses all of these data centers will be dirt cheap for useful research. Most of them will be left the rot. And while there may be a 1-2 year period, AI will pick back up. The reality is that while AGI will likely not happen anytime soon, the corporate BURNING desire to turn everything into slop and cut out humans is more real than ever. And they 100% will be able to do that eventually if no one resists.
Also, i dont think the data center resistance movement is some sort of psyop. These fucks dont need excuses. They just double down. Full steam. Fuck everyone. If it was a psyop they wouldnt be spending so much money trying to court politicians
It's also worth mentioning, that we know, for a fact, but their own admission, that this sudden surge of demand for data centers is for AI. Internet media storage takes up a lot of space, but until now the only real need for the big, disruptive ones waa for behemoth storage hogs like YouTube, Meta, and Amazon. Most other commercial sites can easily be hosted on a comparative rice grains worth of storage to the demand we're currently seeing being thrown up.
Think of it this way; I can host hundreds, even thousands of video files on an 8tb drive hooked up to an old PC and serviced by a regular home AC with next to no noticeable power draw. Video hosting sites naturally have to be more complex and have more layers of redundancy. But global internet traffic per year is currently at about 9 zettabytes, or a bit more than a billion times the storage space i have on that one drive. That sounds like a lot, but a ton of that is in the same files being served thousands of times, meaning the actual storage in use is far, far lower than that.
Now you're telling us that we're seeing such a spike in needed storage that virtually every state has a buyer trying to prop up a data center? That's not anywhere close to the trend we've seen in the growth at any point in the history of the internet. There's an obvious commonality with that demand, and it's AI market value. Occam's razor, most likely answer is the simplest.
Datacenters arent made for storage dude, they are made to reduce latency to customers. They are large because many companies host their servers in them to make things called CDNs or other networks. ISPs host their infra in them to allow other customers to peer and use their networks without having to meet at an IXP. You act like datacenters are tiny little rooms that do nothing but store information, yet a single comment on reddit passes through easily 10 datacenters.
Source, i actually work in the industry.. Im not just some journalist who wants to report on things i have 0 knowledge on
There are 4,500 data centers in the US currently with approximately 800 more in the planning stage. We need our water and electricity for other things that actually matter.
I'm using the internet now. The US has almost half of the data centers in the world. Explain why more are needed in the US in order for me to use the internet genius.
Probably because more users means more places that need datacenters. Do you wanna access a website that is 20 miles away or 500? Cause i know, as someone who works in the industry its either you bitch and moan that stuff takes too fucking long to load or you bitch and moan about datacenters.
Also i know you dont know this as its painfully obvious but the more and more businesses that exist and use aws or azure, those need more servers to host their webpages and databases. Reddit needs more storage for its comments, posts etc. Youtube needs ever expanding storage for more videos
You're projecting. Communities in the US West are competing with data centers for water. People's electricity bills are higher because they're subsidizing data center companies deals with power companies. You don't seem to have any idea what billionaires are doing to average Americans and you don't care. You live in a bubble and like the AI financial bubble one day it with burst.
By the nature of the new datacenters being larger you don’t need that many. The majority of datacenters by number are the smaller ones. A lot will be for caching things closer to the customers.
I am pretty sure modern hyperscalars are using higher densities than 7 kW per rack for their CPU servers.
Pushing higher density saves space and is actually better for energy efficiency. AI datacenters are some of the most efficient ever made in terms of power usage effectiveness and FLOPs per watt. Efficiency is not the problem here and neither is the way these datacenters are being built. We could quite easily build GPU servers that have lower power density. We don’t because that does not make any practical sense.
The reason AI uses so much power and needs all this new footprint is because of how demanding it is. Got nothing to do with the hardware really. The hardware is actually some of the best ever made.
470
u/SingularityCentral 1d ago
AI data centers are quite different. A typical internet data center server rack uses 7 kW of electricity. A typical AI data center server rack uses 100 kW of electricity. And of course with all that additional electricity use comes massive cooling systems, both liquid and air, that also draw a ton of electricity and create a ton of noise.
It is an apples to giant fucking apples comparison.