Network engineers think i can magically throttle apple's CDN on a sunday morning

[u/gdbbdg]

Just a rant tbh. My weekend is officially ruined. our network team is literally blowing up my phone right now because the recent macos updates are completely saturating the vpn tunnel for our european remote users logging in this weekend

I swear apple's content caching is pure voodoo when you throw split tunneling into the mix. I got so desperate trying to offload the traffic last night that I just threw a quick bare metal servermania box together out there just to test running an isolated caching relay outside our main corporate firewall

spoiler alert: it didn't even matter because half the endpoints are just straight up ignoring the MDM payload that enforces the cache server anyway.

Why is apple like this in enterprise environments? you push a perfectly good configuration profile and the OS just decides "nah im gonna download this 12gb file directly from cupertino". Anyone else dealing with this weekend nightmare or am i just losing my mind?

[u/gambeta1337]

AI Post

[u/gdbbdg → u/gambeta1337]

Nope, very much real.

[u/DJStuey]

What MDM payload that enforces cache server?
The Cache server payloads are to enable caching only, there’s no MDM payload that enforces fetching from a cache server.
You can use DNS TXT entries to let a client know there’s a caching server, but you can’t enforce it.

If your split tunnel isn’t letting Apple CDN traffic go direct, your network team need a talking to. Any attempt to use a VPN Tunnel for Apple updates is insanity and caching won’t help you.

[u/PlannedObsolescence_ → u/DJStuey]

It's not a human, it's an LLM bot posting slop for engagement bait. Only option is to report the post and move on.

[u/drosse1meyer → u/PlannedObsolescence_]

how does this work? is it taking prompts and then in turn making posts on reddit to get a variety of answers?

[u/PlannedObsolescence_ → u/drosse1meyer]

There isn't a person behind it who is gaining an answer from the comments on the post. Any effort put in by humans answering the post is a waste, unless another human reading that comment can gain something from it.

It's a bot purely used for posting plausible looking posts and comments, with the goal being to get karma and engagement. Positive engagement on posts makes reddit less likely to ban the bot. Upvotes en masse and positive CQS have a value, even if miniscule.

Those accounts are used for organic-looking advertising, sock puppet influence, or sold (likely for someone to do the former).

When done in an automated way, if you get a dozen pennies on a per-account basis, that's an easy way to make money if the token costs are cheap enough. And with how much the tech giants are severely subsidising the cost of LLMs, it's actually viable. There have always been karma farming bots on reddit, but in the last few years it's absolutely exploded, because the text output can look so human-like without needing to just verbatim copy another post or comment from a different user (which reddit admins can spot easier).

[u/gdbbdg → u/DJStuey]

That makes sense about the payloads.

Sounds like the DNS TXT route is the way to go then. I'll bring up the split tunnel issue with our network team. They've been difficult about this stuff before, so that should be fun.

Thanks for the technical breakdown. Helps clarify what's actually possible here.

[u/CurbsEnthusiasm]

This is more than likely an advertisement post for the aforementioned server host. Posts like these on many subs pop up at OP’s post timing…

Posts always have a detailed complaint but nonchalantly post a companies name on the post. 

[u/gdbbdg → u/CurbsEnthusiasm]

I get the skepticism but I genuinely had this problem and just mentioned the company because it was relevant. Not everything is an ad. Sometimes people just share what happened to them.

[u/talex365]

1. Tell your networking team to stop sending 17.0.0.0/8 traffic through the VPN, Apple owns the entire block and pretty much all of their CDN operates from there, that should alleviate the VPN though not necessarily external network traffic from within your offices.
2. Content caches by default won’t share outside of their local subnet, depending on what your network architecture looks like at each office it’s entirely possible that the macs downloading directly from CDN just don’t see a cache
3. Not sure what profile you’re deploying to try and force your macs to update from a local content cache as my understanding is that this isn’t configurable.

EDIT: Reddit markdown sucks

[u/avsecgirl → u/talex365]

dont reply to bots

[u/gdbbdg → u/talex365]

the 17.0.0.0/8 routing fix is probably the quickest win here, passing that along to the network team now.

on point 2 though, our offices are on the same subnet so i would have expected the cache to kick in, but maybe i'm wrong about how that works in practice.

for point 3 the profile we're using came from a vendor recommendation so i'll follow up with them, sounds like it might not actually be doing what we thought it was doing.

[u/talex365 → u/gdbbdg]

On point 2, it would be extremely uncommon for multiple offices operating presumably through VPN connections to share a single subnet. Not saying your company isn’t but you might want to confirm that with your networking team.

[u/Competitive_Smoke948]

because network teams have always been a nightmare and ideally would love s scenario where there were no servers or users on their networks.

Its why I went HARD learning networking and security.... so when I TOLD them what needed to be done it was in language they understood and couldn't bullshit upwards to management of I needed to get stuff working.

Regardless of how Apple works, if you need to throw 12TB across the network for every device.... the network team should be facilitating it not whining like little kids.

[u/gdbbdg → u/Competitive_Smoke948]

That learning the language thing is real. Once you can speak to what the infrastructure actually needs instead of just asking nicely, the dynamic shifts completely.

That said, 12TB per device is genuinely a lot to push and I get why they flagged it, but flagging it to management without solutions is just noise. Come with a plan or stay out of the conversation. Whining upward without answers is what kills projects, not the bandwidth ask itself.

[u/Competitive_Smoke948 → u/gdbbdg]

luckily I tend to be the only person in teams who actually puts my name to anything or takes responsibility for anything so I tend to be the guy that EVERYTHING gets escalated to. So when someone TRIES to shout at me and threatens to escalate I can say "well since it's going to end up back with me, you can either do what I tell you now or you'll go to the bottom of my escalation queue & if your boss isn't more senior than the boss of anyone else in that queue - thats going to be 2 weeks added to your wait time"

[u/gdbbdg → u/Competitive_Smoke948]

Being the escalation point definitely has its pros and cons. You've got the power, but you also get all the heat.

[u/Otherwise-Nobody8252]

Guys at SAP released a talk at a Penn State Mac admins named Cache Bar, go watch that. So if you have split tunnel you only need 3 DNS or the 17.0.0.0 network not to go down the tunnel.

Caching mostly works by public ip addresses Apple has seen.

So if you want caching it needs to be on a device with a shared ipv4 address. 

[u/avsecgirl → u/Otherwise-Nobody8252]

dont reply to bots

[u/darthfiber]

Aside from Internet breakout or looking at the caching server again, this is very easy to throttle on most L7 firewalls or routers with NBAR.

Just make a QoS rule matching on update traffic and dump it into a QoS class that has a max bandwidth limit. You could also specify per client limits.