p.enthalabs

45°C cooling design cuts data center water use to near zero

blogs.nvidia.com · Read Story HN original

Comments

Heat exchange is used instead of refrigerating the coolant. Makes sense. How do they manage the indoor climate for the humans working there though? Eventually everything will be at 45C in the building, will it not?
The heat exchange between that fluid and the ambient air isn't infinitely fast, if it's low enough they can just run "normal" A/C at low power for the humans. They just need to keep the heat in the fluid until it reaches… well… whatever heat dump there is. (cf. top-level post)
> Heat exchange is used instead of refrigerating the coolant.

There are some systems that pipe refrigerant around the building, but they’re relatively uncommon (VRF or variable refrigerant flow if you want more details).

Glycol and water is cheaper than refrigerant so there’s usually a chilled water loop that passes thru a heat exchanger that interfaces with a chiller (vapor compression refrigeration) to reject the heat from the chilled water loop.

This eliminates the need for evaporative cooling towers.

On one hand: great!

On the other hand: the heat has to go somewhere. So… where? Datacenters already create a warm microclimate in their vicinity, is that getting even worse?

This approach appears to directly reduce energy use (that's what the articles says). The heat would still be going into the local environment, but if there is a reduction in energy use, there should be less of it.
The temperature is independent of the actual heat flux. Also - a quick search suggests that at best the data center coolers run at COP of little more than 10. The inverse of that is the amount of heat wasted just on cooling. Having a system not relying on heat pumps would only make it better. A back of the envelope calculation based on PC AIOs suggests they would achieve a COP of 20 or more. A scaled up system would be more efficient than that, if not just for wider tubes.
Actual heating due to human energy use is not really a big deal except perhaps locally. Climate change is caused by changing how much heat the earth retains from the sun. Maybe if we stopped using fossil fuels and used immense amounts of nuclear power, we would care about the waste heat. But solar and wind power largely redirect energy flows.

It’s kind of like how brine from desalination is not a global problem for the oceans at all — all that matters is diluting it enough that it doesn’t poison the local ecosystem.

Indeed. If the datacenter uses less total power, it produces less waste heat.

If you manage to use the waste heat to avoid generating heat somewhere else (that the article calls heat recovery) then there’s a further reduction in total heat output.

This is kinda debunked / obviously false. It's almost entirely a land use issue: a building will create a heat island. Data centers aren't using enough energy to make a significant difference.

More on it at [0], but it doesn't take anything beyond a basic energy calculation to know that 1GW of energy is not going to have a significant effect "6.2 miles away".

[0] https://andymasley.com/writing/data-centers-heat-exhaust-is-...

Claude write good.

Nvidia has so much money and they can’t afford to pay a human for a day of their time to write a blog post?

Their valuation is based on their software stacks’s ability to displace human labor, this is just them eating their dogfood.
Oh I understand funny money

We are all fucked.

And it’s sad because Jensen seems like one of the rare good CEOs when I listen to him speak.

But even Dario says he doesn’t let Claude actually write his blog.

>And it’s sad because Jensen seems like one of the rare good CEOs when I listen to him speak.

Have we been listening to the same person speak for the last few years? Jensen rarely even sounds sane anymore.

I feel that the sad reality is that most blogs in the future will be addressed to AI and not humans, it's gonna be quite rare to read directly something as we will have built-in tools within browser and phone and OS and so-on that always rewrite on-demand based on current expertise, wanted tone and so-on. There is a recent study I believe that demonstrated that AIs digest better articles made by AI, which means that it might be just better to let AI write the articles so others AI have a better accuracy in digesting it (and incorporating it in their training data as well).

The same as technical docs for any codebase, humans will not read them anymore, only AIs which then translate it to human on-demand, it's already happening, I've worked recently with many new frameworks/codebases without even opening the doc (not even the Github page) and solely asking the agent to gather info for me about it.

PS: The reason I feel it will be this way is that it will allow to legitimatize mass data collection indirectly, instead of doing telemetry on page and software level, we will just send all the content automatically to some inference providers (probably provided for free by Google, MS and so-on)

We're there. The recent HN article about the Fender Stratocaster had some content from a Fender press release, which was regurgitating text from a legal aggregation site. It was, overall, bad coverage of the area of copyright on decorative but useful objects.[1]

Watch for cases where content has been through two layers of LLMs. It's not good.

[1] https://news.ycombinator.com/item?id=48665916

> Claude write good.

> Nvidia has so much money and they can’t afford to pay a human for a day of their time to write a blog post?

The shareholders desperately need that money.

You make it sound silly, but yes, as a (very small) shareholder of NVDA, I would indeed prefer that they use Claude to write blog posts rather than hiring a dedicated person to do that with "marketing flair". Better yet, I would prefer that they just publish the prompt with the actual update details and I'll take it from there.
This opens up an interesting synergy: district heating. 45C is low but not unworkable for a district heating loop, and a data center might be able to make a nice pitch to a community if the data center offers to provide heat to a district heating system for free. This brings the value to the local community of a nearby datacenter up from near zero to potentially a few million dollars per year.

Summer is still an issue, but fun solutions are possible. With the right geology, I think it’s possible to heat an underground volume in the summer and recapture (some of) that heat in the winter. In many, many climates, annual heating costs are far higher than cooling costs, at least if people aren’t stupid with skylights. [0]

[0] As a back-of-the-envelope heuristic, heating or cooling load due to conduction and air exchange is proportional to the difference between indoor and outdoor temperature. Outdoor temperatures of -10F to 30F are not unusual in the winter and are 40-80F away from an indoor temp of 70F. But outdoor temperatures in these climates rarely exceed 95F and are mostly lower in the summer, so that’s 15-25F of cooling. And heat pumps are more efficient at smaller temperature differences.

Radiative heating is an entirely different story.

Do you live near a datacenter? Property value goes down, constant humming.. the way we heat up the earth right now, i don't think you have to worry about heating
Couldn't imagine living with the ~55dBA noise literally all the time
It sounds like with this liquid cooling, they won’t need the fans?
Some systems use liquid cooling for the GPU and CPU, but air cooling for the PSU, RAM and SSDs.

With that said, by the standards of industrial sites data centres are quiet, low traffic and smell free. An industrial area that can’t build a data centre certainly can’t build a steelworks or oil refinery or leather tannery.

They almost certainly need fans on the outside of the building to cool the 55C water back down to 45C. But correct, no fans on the servers themselves or even in the building. Except perhaps for the humans, so they can stand to work inside the building, when needed.
The humming are the gas turbines which also damages your health.
If the outdoor temperature is cool enough (maybe 30C?), you just pipe the liquid outside through a large enough loop or heat exchanger to get it back down to under 45C. Even better if you can put the loop in a lake and dump the heat there (maybe not better from an ecological POV though). The pumps moving all that liquid becomes the noisiest component.
Noise is a design choice and could likely be legislated away. Reject heat is different than heating from greenhouse gas effects that are “heating the planet”.

No one bats an eye when an air conditioner runs.

> No one bats an eye when an air conditioner runs.

In the US

Now do heat pumps in the winter.
Data centres aren’t loud it’s the gas power generating stations next to them that are loud.
> No one bats an eye when an air conditioner runs.

I find their noise pretty obnoxious. Out of respect for my neighbours, when I get around to installing one, I'll be getting the absolute quietest model available.

I’ve been to datacenters, but not the huuuge ones people seem to talk about in the context of AI. They are noisy inside (due to air cooling, which is largely avoided by the tech in the OP), but they’re entirely unremarkable outside compared to any other commercial or industrial building. Computers are not inherently loud, nor is power conversion.

Power plants are all over, even in populated areas. They’re not so bad either (except perhaps coal).

There is no fundamental reason that datacenters need to be especially unpleasant to their neighbors.

It depends a lot on things like geology and some people are a lot more sensitive. It is really an issue.

I don't have any datacenters near me but I can hear some heavy hums from the washing machine 3 floors up when it put my head on my pillow, for some reason it just propagates through the building physically. When I walk around I don't hear it. Datacenter noise can be the same.

IMO they should be put away from habitation, there's no reason for them to be near there anyway

> IMO they should be put away from habitation, there's no reason for them to be near there anyway

I agree, but that's a hard problem (in the US anyway). Unless you're plopping data centers in the middle of national parks, or in the middle of the desert where water is going to be a problem, you are nearly always going to be within some small mile radius of civilization. Plus the cost of trenching new fiber out in the middle of nowhere.

The same reasons humans want to concentrate in a particular area (access to jobs, infrastructure) are the same things that data centers need.

Once water-less cooling tech like this improves then yeah, just plopping them in the middle of the unpopulated desert becomes viable (assuming you can get the fiber out there and latency is tolerable), so long as they generate their own power.

The climate requirements to run at this hotter temperature still probably means it'll require more active cooling in the desert during daytime /summers. Assuming we're talking about hotter desert environments like US southwest. That might make your proposal not as economical.

Imo we should just solve the problems with data centers being near cities. Manage/regulate the noise and any waste (heat included, it shouldn't drastically impact the neighbors) and make them pay for any utility capacity/reliability upgrades needed. If this article is right and water usage can be nearly eliminated then it seems like the rest should be solvable? Especially if we can take the extra heat and use it for local power or heating needs.

You might (but probably not) be able to do district heating with this, but electricity generation is not going to be efficient. Heat is a very low grade form of energy, and you need a large differential to drive a turbine efficiently.

If you cycle between 45 C and 55 C water temperature (as mentioned by the press release), you are only getting a 10 C delta. That isn't even enough for district heating, probably not even with heat pumps.

Now if you have something like a steel foundry, that have much hotter cooling water, you can absolutely use the heat for district heating, but even then it usually isn't enough for cost effective electricity generation. Even when it is waste heat, as the equipment to handle it still costs money and requires maintenance.

> If you cycle between 45 C and 55 C water temperature (as mentioned by the press release), you are only getting a 10 C delta.

You are calculating the wrong delta T. To heat a space, you need your working fluid to be warmer that the space you’re heating by an appropriate amount.

55°C is certainly on the cool side to heat a building, but it’s entirely workable with a high-area, highish-thermal-conductivity system. Here’s an actual chart:

https://www.warmboard.com/wp-content/uploads/2022/04/WaterTe...

You don’t actually want an absurdly warm floor.

Even for buildings that need warmer fluid, water at 45-55°C is a fantastic source for a heat pump.

55 C might be enough to heat buildings but it's right on the edge where domestic hot water needs to be to kill Legionella. So traditional DH systems need to run hotter.

There low-temp/cold DH systems out there that rely on heatpumps in the buildings to extract the heat. Less losses in the network and you can even use them for cooling, but needs heat pumps everywhere.

In comparison, a traditional heat exhanger is pretty simple technology; just a hunk of metal with a valve.

> I don't have any datacenters near me but I can hear some heavy hums from the washing machine 3 floors up when it put my head on my pillow, for some reason it just propagates through the building physically. When I walk around I don't hear it. Datacenter noise can be the same.

Right. Vibration propagates through solid (and liquid) materials.

But this can all be measured and controlled, and there's nothing special about datacenters. A building that is hundreds of feet away will couple to your pillow much less strongly than a washing machine in your building. And the washing machine often has a wildly unbalanced load and minimal decoupling between itself and the floor, whereas a big fan in a datacenter or other industrial building ought to be balanced and also ought to be installed on decoupling mounts.

If datacenter operators (cough xAI) are being lazy about properly selecting, installing and maintaining equipment, then you can have a problem. Otherwise you have a much smaller problem.

> It depends a lot on things like geology and some people are a lot more sensitive.

People said this about high voltage electric lines and wind turbines. Blind tests proved they were imagining things.

It's like anything else in this world. Corner cutting and being shitty leads to shitty outcomes
>There is no fundamental reason that datacenters need to be especially unpleasant to their neighbors.

Sure there is, being a good neighbor costs more than being a bad neighbor

On-site natural gas turbines at a handful of DCs are genuinely loud. In general I agree that DCs are mostly fine neighbors, but maybe louder power plants aren't.
Yeah this is it. You can make really nice datacenters that are basically quiet and environmentally perfect. This was never in dispute.

But that is not how corporations roll. They want the cheapest shit that they can get away with. No regulations only corruption. Which is middle of nowhere America.

And if they put them in the middle of nowhere, I don't see why there's a problem.

What I don't understand is putting these things in populated areas.

Would you like to work in the middle of nowhere?
After construction, not all that many people work at a data center. Some ops staff, maybe a small security team.
So let's spit on those 'not so many people' from our ivory tower?

Besides, it's not about people, it's about power. Pulling a few MW into middle of nowhere is prohibitively expensive.

If they can be deployed in low-earth orbit with nobody working on them, they can be deployed 20 miles east of Bumfuck, Nebraska with nobody working on them.
The physical threat model of the Nebraska option does not rely on the tyranny of the rocket to keep intruders away.
The point is that if you don't put them in the middle of somebody's neighborhood, nobody will care to attack them.

The idea of building robust, lightly-staffed technical facilities in obscure places all over the country is nothing even remotely new. It's how the long-distance telephone network was constructed and successfully operated in the decades before satellites took over: https://telephoneworld.org/long-distance-companies/att-long-...

30 minutes outside of the suburbs doesn't sound like an awful commute.
they just want data centers now. most companies would rather use solar, but they can't on short timelines due to land use regulations (and import tariffs)
When I was building data enters in populated areas there were regulations for noise, visual signature, power usage, and etc. It looks like a lot of these newer sites are in low regulation areas. Which is great for profit margin, and not so great for neighbors.
Sounds like the “don’t tread on me” crowd is about to find out the value of regulation.
Look up Benn Jordan's video on datacenter infrasound. Just because you can't perceive the noise doesn't mean it's not there and it doesn't have an effect on the human body, especially over very long periods of time.
That's not a counterargument: it's an ad hominem.
How? He carefully examines and actually reads all the papers referenced in the video and shows that they either don’t support Benn’s claims or completely contradict them? Drawing conclusions about Benn Jordan on this basis is not ad hominem. Given the contents of these papers (i read a few of the abstracts), Jordan either didn’t read the papers, didn’t be understand them or else is being disingenuous.
You are arguing that he came to the "wrong" conclusions without actually arguing against those conclusions, or how he got to them. Instead, you are arguing that a reasonable person can come to your conclusions, and therefore Benn is not a reasonable person.

At no point in his video does Benn claim to be representing the consensus. The whole premise of his videos is that he is introducing something for which there is very little consensus! Each paper has its own conclusions, which don't have to be the same as Benn's. This whole thing boils down to an attack on Benn's character ability to draw meaningful conclusions on his own.

> You are arguing that he came to the "wrong" conclusions without actually arguing against those conclusions

I am not.

Benn claims that the papers say one thing when in fact, if you read them (even the abstracts), they mostly say the exact opposite. It's that simple.

I was a regular viewer of Benn Jordan until seeing one of his videos about how Apple sucked because the iPad was incompatible with so many MIDI and DAW controllers. He went through his devices one by one, plugging them into the iPad and then showing that the iPad didn't recognize the device.

I thought it was a joke of some kind and eventually he would give the "reveal" but no, he finished the video without ever considered that USB is a host/device connection - and although USB-C confusingly introduced a symmetric cable, you cannot connect two devices back-to-back (one end has to be a host) and expect anything useful to happen. And that's why Apple sells a "camera" kit to allow an iPad to act as the host to USB devices.

Seeing him use an oscilloscope to diagnose an Ethernet issue in an earlier video had given me the expectation that he was technically knowledgeable and interesting. So it was shocking that he could present conclusions with such a confident and knowledgeable air, having missed such a basic and fundamental fact about USB.

And there was no push back in the comments that I could see at the time - so seemingly half a million viewers would have finished the video feeling they had learnt something.

> the way we heat up the earth right now, i don't think you have to worry about heating

So what, winters would be no more? Snow will disappear, no more ice-men and christmas trees, and subzero conditions in general, too?

And no more food. Or at least not enough of it to feed very many people.

You do eat, don't you?

As the earth has warmed over the last century, has food production increased or decreased?

If production increased (dramatically) while temperatures increased in the past, could this trend continue?

Christmas food?
Coldest month average temperature where I live is around -7C, with peaks of -35C. Climate change is not going to increase that average, more like decrease. Typically, of course, electricity price is the highest during that month too.

I think we are going to need heating.

>Do you live near a datacenter? Property value goes down, constant humming

I don't live next to one but I'd take constant humming over the constant stop/go traffic noise, honking, squeaky brakes, slamming doors and revving engines I now have on my western side of the apartment, thanks to the unemployment office the city opened on my street not too long ago.

So how come constant humming is somehow an illegal nuisance, but we've been expected to put up with the much more annoying urban traffic noise for decades just fine?

My parents apartment have constant humming anyway thanks to the HVAC system on the roof of the nearby supermarket and white/brown noise is far more tolerable and easy to tune out than traffic noises.

More noise categories should be illegal or fined in dense areas, not less
Agree, but data centers are no inside dense areas though.
To me that only makes clear that gas turbines are the noise issues, no data centers. Surely there exists data centers that work off the main power grid and don't have a natural gas turbine as their own power source.
> we've been expected to put up with the much more annoying urban traffic noise for decades just fine?

For one, there tends to be little traffic at night when most people want quiet in order to sleep. Driving is also something (nearly) everyone does and benefits directly from, so negative externalities are easier to accept. It is much harder to accept a new source of noise near your home you haven't asked for and don't directly benefit from.

I live nearby a road going down a coulee that dickheads love to speed down in warmer weather at night. I'd trade that for a hum any day.
> Driving is also something (nearly) everyone does and benefits directly from, so negative externalities are easier to accept.

This reads a little too close to driving being an inherently good thing or some sort of objective requirement, but it's only that way in certain urban places because the built environment makes it as arduous as possible to do those things without.

Something that pisses me off about many urban places that don't even otherwise require people to drive, is that many who do use their cars the most often have their neighborhoods protected from the noise they contribute to everywhere else. This whole thing of putting apartments only where there's already the most disgusting car-infested thoroughfares; "sorry, can't have an apartment one street in off the main drag, that's only for bungalows! Don't like it? Get richer. Excuse me while I drive through your bedroom and park for free in front."

>s that many who do use their cars the most often have their neighborhoods protected from the noise they contribute to everywhere else.

This, so much this. All the noise producing infrastructure in cities is dumped in the highly dense poor areas, and the rich people living in the quiet suburbs in single family home who need to drive in front of your home, are protected by this externality.

Politicians also like to pat themselves on the back for things like zoning density "particularly along transit corridors" as if only the immediately stroad-adjacent parcels are viable for accessing the bus and anyone who makes less than $300k a year should be grateful.

Like thanks, I guess, for doing the absolute minimum that should have already been the default policy for 50 years

How dare those nasty, dirty, unemployed live their lives under likely desperate circumstance. They are so much worse than corrupt oligarchs pumping and dumping their way into the greed hall of fame.
Do you? I live at 4th and Brannan and there was one just off 3rd and Brannan in San Francisco. It was shut down when hosting.com sold it off but I didn't notice it while walking by then and I don't notice it while walking by now.

My GPUs at Hurricane Electric in Fremont are also completely unnoticeable outside the building. Inside, when I'm working at the cabinets it's obviously deafening. Outside you wouldn't even know. Realistically, the predominant sounds at my home are from the traffic on the Bay Bridge so it's nice when there's congestion because it's quiet.

Honestly, I wish there were more urban datacenters. It's getting quite annoying having to make a 1 hr trek to Fremont every time I want to rack a new server.

> Honestly, I wish there were more urban datacenters.

There's a lot of them in high rise buildings... but they come with high rise rent.

Imagine if one of the amenities in a high-end residential building could be a cabinet haha! That would be amazing. Doesn't make economic sense, but I'd love it. Would love to explain that to my wife. "I know that one has the pool, but we can rack my servers in this one if we live there".
Something here about water usage, a small data center, and a swimming pool. Combining the three may either be awesome or tragic!

I like this idea. A small cage in an apartment complex would be a huge selling point.

Or even a La La Land on the corner, urban DC in the back. Winning across the board.

> water usage, a small data center, and a swimming pool

Swimming pool? The cooling can be for a hot tub!

>the way we heat up the earth right now, i don't think you have to worry about heating

Nearly 10x more people die from the cold than from the heat.

"...9.43% of global deaths were attributable to non-optimal temperatures, with 8.52% from cold and 0.91% from heat."

https://www.thelancet.com/journals/lanplh/article/PIIS2542-5...

This study doesn't factor in droughts, floods, crop death (and starvation), and other non-direct effects. It also doesn't consider wet-bulb events, because it's looking at average ambient temperatures.

I don't think this is climate change propaganda, but your application of this study by evoking it in a discussion about climate change feels like it.

Deaths from all natural disasters have dramatically decreased over the last century.

I won't bother sharing the source because you'll find some reason to dismiss it I'm sure.

It's interesting you claim my comment is propaganda when I cite a scientific source and yet have nothing to say about the parent comment which claims we no longer need to worry about heating due to climate change. Which of those comments seems more propagandistic?

> no longer need to worry about heating due to climate change

I'm nearly positive that was sarcasm.

> Deaths from all natural disasters have dramatically decreased over the last century.

Because of our ability to predict them, and due to advances in scientific knowledge that allows us to more safely handle them. It's not because they've become less deadly. They're also becoming more frequent, and deaths alone isn't their only problem. They also drive loss of shelter, migrations, economic damage, etc.

> you claim my comment is propaganda when I cite a scientific source

Yes, plenty of science is propaganda. The study you cited wasn't propaganda, but the way in which you used it was, because it's not applicable to the way you're using it.

Direct exposure to ambient heat is only one of the many issues related to climate change, and one of the least deadly. As I pointed out, the study didn't include wet bulb events, so even direct heat exposure wasn't properly covered in the study in terms of climate change.

I grew up in Northern Virginia, (of AWS US-East-1, MAE-East, and Equinix fame), where there are more data centers than anywhere else in the world, and I never heard organized opposition to them until the last couple of years. They were mainly viewed as a way for Loudoun County to build their industrial tax base without the downsides of having industrial workers, and allowed them to consistently lower property taxes while having excellent schools. Data centers are unsightly and use electricity and water, but so does literally any kind of industrial facility. They are also pretty quiet, if you exclude the ones using on site gas turbines for electricity.

Property values have consistently gone up in that region for decades, and are up to $6 million an acre if there's enough contiguous land to put another data center on.

Many of the people complaining about datacenters would also complain about literally any kind of development.

The new gen of AI-boom-construction datacenters seem to be substantially different than the ones I used to go when working at companies whose software ran inside them. Those were running air-cooled rack servers, which is a whole different world from these things. The couple of the new gen I've seen have been much larger and much more turbine-happy.
i'm confused on the humming part, i've driven by them many times and they're just...large, sorta ugly, buildings. but that's really it. i wonder if i walked up to a building i'd hear the humming. quite surprised to learn that they emit a humming that bothers nearby folks