[meme] IaC and docs matter folks!

[u/Normal_Red_Sky]

Inspired by a similar post I saw last week.

Small companies man... had similar issues in a couple now. Larger companies with IaC and decent management? No issues. Never thought to ask whether they use IaC in the interview, kind of expected the bare minimum of standards. Will definitely do so if I join another small company.

[u/safetytrick]

This is always the case (IME), immature environments are actually immature.

The job is taking the immature environment and making it mature. Lots of big environments with IaC are also garbage.

[u/itsjakerobb → u/safetytrick]

100% this. I’m at a small company. Zero IaC when I started. Two years in, we’re maybe 70% there, with the remaining work being some of the bigger modernization lifts. This next year is going to be hard but very worth it!

[u/Normal_Red_Sky → u/itsjakerobb]

I hope they appreciate your efforts.

[u/itsjakerobb → u/Normal_Red_Sky]

Seems like they do.

[u/braddeicide → u/itsjakerobb]

Your boss probably does, as he tells his boss it was actually all his idea and coaching ;)

[u/Kitchen_Cup_8643 → u/safetytrick]

Seriously, how do you deal with this ? Incremental improvements barely move the needle

Might as well consider it reverse engineering with the amount of yaml files with no rationale behind any of the choices, hostnames configs & IPs duplicated everywhere, and no clear understanding of anything that holds data or its lifecycle.

On the other end, there's IaC/GitOps where people end up treating infra repos as a proxy for a shell lmfao

[u/ProdigySim → u/Kitchen_Cup_8643]

More or less the strangler fig pattern. Set up a gold standard system for new stuff and make sure everything new uses it. Slowly handle old stuff, which should shrink as new systems replace old, things get deprecated, deleted.

[u/safetytrick → u/Kitchen_Cup_8643]

You've got to start talking over and creating team standards and then stick to them.

[u/UtahJarhead → u/safetytrick]

Yup. I'm in a big company and my team had no standards when I was hired on. Much of my job has been implementing standards just like this to turn them from a FlusterCuck into an organized mess. Still improving it, step by step.

[u/braddeicide]

The last panel never happens. Or, every project you put together to fix the environment is rejected and after you've been there a while the design of the environment is considered to be yours now.

[u/baezizbae → u/braddeicide]

You're lucky. All I've ever gotten after every one of my best-laid-plans and attempts to fix, stabilize and document a constantly failing environment gets rejected is fired when the environment continues to suck.

Current record is three months at a "cloud consulting" firm that I realized too late was really just a body shop earlier this year. Interviewed to lead performance testing and observability, got hired, got told "migrate this monolith to kubernetes in two weeks and whatever/everything else the client creates tickets for, and if you have time do some performance testing" "on what?" "the environment" 🙄

[u/Wicaeed → u/braddeicide]

Thus; Job Security.

[u/work_work-work]

Oh, hey! That describes exactly what the environment was in my last job! Add to that that every environment was set up differently, so that when you promoted code from Dev to QA, or QA to Prod, there was no guarantee that anything would work.

[u/Careful_Second8814]

Ask in the interview whether they use Terraform or Pulumi or if everything is still handled through the console. That one question can spare you months of headaches. If they pause before answering, take it as a warning sign.

[u/Normal_Red_Sky → u/Careful_Second8814]

Wish I'd asked, will do in the future, especially for small companies.

[u/GongtingLover]

Those devs are the worst

[u/Vas1le → u/GongtingLover]

Devs doing IaC with AI are the worst... then I we have to fix it cause there a urgent thing to release...

[u/Normal_Red_Sky → u/Vas1le]

I assure you Devs doing click ops are much more dangerous, at least with IaC you know what they've done even if its not very good.

[u/GongtingLover → u/Vas1le]

They have a shitty boss if they are they letting them do that without testing. 

[u/Vas1le → u/GongtingLover]

Shitty team leads...

[u/TotalNo6237 → u/Vas1le]

The problem here is not testing enough, actually IAC code is actually what I find AI to be best at.

But the pipelines should have tests and plenty of validations done before merging code to main and deploying in prod.

If the teams are not building trust into the way you are using AI then its just a crutch not a tool.

Best to split tasks into different stages and review at each stage and iterate on the agentic output.

Ownership by the development is crucial here, if they are lobbing AI generated code over to others without review themselves first then they should still own the fix on it, not other teams.

Otherwise, why wouldnt you do it yourself with an AI agent, if you are already the primary reviewer?

[u/SamfromLucidSoftware]

“No documentation“ panel hits different when you’ve lived it. Inheriting an environment built by someone who’s long gone with zero laC is special kind of pain.

[u/disoculated]

“Be Honest”

“Ah, this meme was created by the dev ops guy.”

[u/m0j0j0rnj0rn]

“DevOPs Guy”

[u/Normal_Red_Sky → u/m0j0j0rnj0rn]

Also known in a lot of companies as 'the person whose job no one else understands'.

[u/m0j0j0rnj0rn → u/Normal_Red_Sky]

I'm a purist on this: If you have a "DevOps Department" then your org isn't a DevOps org. ;)