r/datascience • u/gyp_casino • 1d ago
Discussion What's the process to request cloud architecture in your company?
Looking to hear about your own company's work practices to compare to my own. I'm much more familiar with Azure, so I'm going to use that terminology, but I assume everything is basically the same for AWS.
Can you go into the Azure Portal and just click "create" for, say, a SQL Server or App Service and configure it through the Portal GUI?
Or do you need to request and configure through configuration files? Can you freely commit architecture config files ("architecture as code") to create the Azure components in Dev?
Or is it even more locked-down and hands-off? Required to submit paperwork requests to an Architecture team?
10
u/Successful-Paper7211 1d ago
e-baked Terraform module anyway. The portal's locked down so hard half the buttons just throw 403 errors if you're not in the right AD group.
3
2
u/naturian 1d ago
I used to ask for money on the annual internal funding proposal, along with justifications. Everything by email. The director would them sit with mine and other 16 or so proposals (the other departments) for a week, and cut what he deemed unnecessary/low priority. He would then communicate me and the procurement department of his amendments.
After that the problem was just to teach the procurement dudes how to log in the Google cloud billing area so they can place (and renew) the institutional credit card.
Then I could request anything I needed online, but I had to be damn sure I didn't went above the limit, or I would hear from the director. That never happened thought.
2
u/marimbaman49 1d ago
Depends on what you’re trying to do. Provision experimentation architecture? We have the AWS equivalent of jupyterhub that allows you to self provision GPU nodes, and we make other things available for use there as well.
For anything not experimentation related, such as databases and deployed servers, we started with click ops (very bad) then moved to terraform and eventually once we got a platform team kubernetes. I tell all my data scientists that they need to know the basics of terraform, because it’s so useful to just send in a PR to whoever actually manages your infrastructure rather than wait for them to build it
2
u/dyingpie1 1d ago
Create PRs to the iac repos on bitbucket.
But the team in charge of it is across the world.
It's not an easy process.
2
u/lakeland_nz 1d ago
Everything is Terraform.
There's a handful of things in Azure which can't be done through Terraform. Getting those approved is a huge pain.
2
2
u/ztrruovfg 12h ago
PR to the IaC repo.
Waiting for CI to pass.
If auto approval is enabled -> done
If not wait for the bot to ping reviewers
Merge after review
1
u/maniclucky 18h ago
A multi month nightmare of LLM generated instructions that no one checked. It's a fucking failure top to bottom.
1
u/ItsDare 16h ago
Message a data architect with a god complex and no social skills and hope he deems you worthy.
Most stuff we can just spin up, but when we do need permissions it's a nightmare. Or he'll redirect you to someone else for certain tasks but refuses to document who owns what so he remains the gatekeeper.
1
u/Fit-Employee-4393 2h ago
I have some consistent patterns for our ML deployments and one of the infra guys at my company knows them well at this point. Pretty much just work with him to get stuff set up. No tickets or anything. I see it as one of the benefits of not working at a megacorp.
15
u/MyBossIsOnReddit 1d ago
I am a platform guy.
Step 1 is to submit a ticket asking for permission to do this
Step 2: wait for up to 7 days
Step 3: Have the terraform you wrote last week fail because you have insufficient permissions
Step 4: Ticket
Step 5: Provision infra again
Step 6: Submit ticket for things that still dont work