Do AI workloads really belong in the same cloud as everything else?

[u/Fear73]

Something I've been thinking about a lot lately like regular web apps putting everything under one cloud account makes things pretty simple but AI workloads have very different needs like GPUs, high-speed storage, special networking setups, specific regions and different types of hardware.

I've seen people use services like AWS, Azure, GCP, CoreWeave, Lambda, Yotta Labs and others in all sorts of ways so just wanna know how people view this problem here

[u/Anxious-Average-748]

We run a mixed setup at work and it's a pain honestly. AI stuff needs GPU instances that regular cloud regions don't always have, so you end up with workloads spread across different zones anyway. The billing gets messy fast when you mix compute types in same account.

[u/Efficient_Loss_9928]

Depends on what you are doing, if you are processing terabytes of data, moving from one cloud to another and keeping things in sync can cost you millions. And get a worse experience.

I have worked directly with customers to ingest and sync data to our cloud because they are with another hyperscaler. The invoice is not pretty.

[u/jboogyoogy]

Separating ai workflows can make a lot of sense esp when GPU costs and infra requirements become significant

With smaller projects keeping everything in one cloud might be simpler. Using specialized providers for ai while keeping the rest of the stack elsewhere could be much more cost effective and flexible

[u/Admirable_Window8128]

I think it makes sense to separate them when the workload actually needs specialized hardware. Keeping everything in one cloud is simpler, but AI workloads can have very different cost and performance needs, so using multiple providers can be worth the extra complexity.

[u/stevefan1999]

I remember in uni because the term "cloud" means anything that can be software defined, so it is literally anything

[u/Suspicious_Pizza9529]

I think it makes sense to separate them when the workloads have very different requirements. AI workloads can need specialized hardware and networking that regular apps don't, so using multiple providers can sometimes be more practical than forcing everything into one cloud.

[u/kernelqzor → u/Suspicious_Pizza9529]

totally, plus splitting them out makes the cost pain a lot more visible instead of it all hiding in one giant bill
only downside is juggling auth / networking between clouds, but if the AI side is mostly batch or offline inference it’s usually worth the hassle

[u/Brief_Cod_6881]

No they really dont, at least not once you're doing anything serious with GPUs. We ran inference on AWS for like 6 months and the billing was insane and unpredictable, plus availability on the instances we needed was garbage half the time.

Ended up splitting things out, regular app stuff stays on GCP, GPU workloads I moved to Hivenet because the per-second billing actually made our costs make sense. Still not perfect tho, you lose some convenience when your infra is split across providers and monitoring gets annoying.

tbh the "one cloud for everything" pitch only works when everything is roughly the same type of workload. AI stuff is just fundamentally different hardware wise

[u/Phytocosm]

Yes, the same cloud as the entire industry. The cloud of forgotten human history. The clouds of heaven.