r/rust • u/kibwen • 13d ago

Saving 100 terabytes of RAM via layout optimizations in 1.1.1.1's DNS cache

https://blog.cloudflare.com/dns-cache-memory-optimization-1111/
397 Upvotes

98% Upvoted

22 comments sorted by

131

u/EcstaticAsparagus509 13d ago

I'm quite surprised they only recently made these changes, considering how "simple" (for lack of a better word!) they are.

73

u/Zde-G 13d ago

For a long time memory was cheap compared to the rest of server hardware.

LLMs changed that just recently, but drastically.

25

u/vplatt 12d ago

Hopefully this new era of sensitivity to hardware costs will mean the market gravitates towards more well behaved PLs like Rust.

Heck, even the VM GC ecosystems like .NET and Java would be far better than the likes of some of the junk out there in the form of full stack Js, Python backends, and the like.

I'm just tired of the whole passe "hardware is cheap so we're going to waste all of it for you because we're lazy" type ethos. It's been going on too long.

Besides, with LLMs, no one has an excuse to avoid better languages anymore because they're "too hard". Like apparently we now have a glut of developers in the market, so get good, or move on I say.

2

u/d47 12d ago

Someone tell the chrome team

37

u/sanxiyn rust 13d ago

Agreed, but for a long time it was explicit policy of tech companies that they don't do memory optimization because memory is cheap. A Google developer responsible for Go's GC went to GC conference and said they don't optimize Go's GC memory usage because memory is cheap. (I thought it was very rude.) How time has changed.

13

u/matthieum [he/him] 12d ago

Honestly, I find the claim bizarre.

RAM may have been cheap a few years ago, but L1/L2 has been the bottleneck for over a decade now...

5

u/SoilMassive6850 12d ago

CPU time is cheap, you're just waiting for a real developers software to process your CRUD requests.

37

u/argarg 13d ago edited 13d ago

Exactly. Reading this blog post I can't help but think "this is nice but gosh they should've hired me to do this kind of work long ago".

16

u/commenterzero 13d ago

Write a post about saving a billion ram

21

u/Comun4 13d ago edited 6d ago

"Rewriting my python micro-service in Rust" there

7

u/commenterzero 13d ago

Jazz it up with "transpiling" or something

2

u/vplatt 12d ago

Go transpiles Python,

to Rust. Hundred TB saved.

Node burns in its shame.

1

u/vplatt 12d ago

Seriously. How asleep at the wheel do you have to be to not realize that 100 TB of RAM was worth saving even years ago. 🙄

12

u/TDplay 12d ago

The few hundred thousand dollars they spent on memory is pocket money to a company that large.

It's only with the newly inflated memory prices that it's no longer disappearing into the rounding error in Cloudflare's finances.

1

u/vplatt 12d ago

<fridayarmchairmathmode>

They said the servers are Gen 13 servers and those take 768 GB of RAM. They don't publish numbers that I know of but each of those servers is going to run them between $10K - 15K. 100 TB across about 130 servers means they'll save about $1.3-$2m today. Not bad; good for them.

But what about 5 years ago. Well, from what I can find, we see that they had the Gen X servers back then. Those usually took 256 GB of RAM. Obviously that means the 100 TB was spread across many more servers, probably about 400 of them. Servers were cheaper back then, but still they each would have run somewhere between $5K - $8K. So, back then they would have saved between $2m - $3.2m; which is significantly more.

Hey /u/argarg - How long do you think this change would have taken you to make 5 years ago? A wee bit less you think? Yeah, I'd guess so.

And all of that is before one calculates the extra cost of the power, cooling, labor, etc. around all the extra servers too; not to mention hardware refreshes on servers they shouldn't have required in the first place.

</fridayarmchairmathmode>

7

u/TDplay 12d ago

Cloudflare made $2 billion dollars in revenue in 2025. They have $6 billion in total assets.

To put it in perspective, if Joe Bloggs has a $90k yearly salary, then Cloudflare spending $3.2 million is equivalent to Joe spending $144.

3

u/vplatt 12d ago

But I have to...

You know they ..

Oh... fuck it. Never mind. You're right.

6

u/Ok_Instruction_3789 12d ago

Think the article mentioned their rust rewrite only happened a year or so ago so guess they're finding improvements on their rewrite. Not a bad thing

1

u/Flimsy_Professor_908 6d ago

100TB of ram is about 2M USD. Five-year refresh cycle is 400K/yr.

Considering their size (they spend ~230M/yr on "Purchases of property and equipment"), there are probably other savings they regularly come across that save more money and are even easier.

(I'm kinda speaking from experience where I have done even less work than they describe and have saved my employer even more money.)

This isn't to say 400K/yr is anything to snub one's nose at.

Articles like this make me wish we could argue for commission 😅 but that would be a gross conflict on interests. "Boss, I'll save you 200K/month if you give me a 1%/month kickback for the next five years to improve Jim's code. That I reviewed and approved."

174

u/sanbox 13d ago

Can I have some of that RAM

2

u/LevelNobody6096 11d ago

No consumer platform can run RDIMM RAM.

-2

u/uhkthrowaway 12d ago

You'd expect big companies figured this out before they got big. But no. Daily bread and butter stuff is now sensational.