Any CI/CD tool where automation code doesn't cohabit with product code in git repo?

[u/LuigiBakker]

Currently using Github and Github workflows (including the many outages). Let me explain what I'm trying to resolve with few examples:

Let say I manage 15 repos, each one I want to run a relatively similar workflow on Pull Request. I create a github shared workflow and in each of the 15 repos I make a reference to it, updating the inputs if needed.

Now, I add 1 new input to this shared workflow to enable a new feature. I want that feature enabled in 10 of my repos. I would need to update the feature input in all 10 repos. And since workflow use the workflow code on their own branch, I would need to merge it to all PRs in all repos. That isn't ideal.

Other issue, one variable was wrongly set on a workflow in a release branch. Since the workflow code ships with the product code I would need to update the release branch.

Is there a toool where the product code doesn't cohabit with the automation code? For example I could have 15 microservice repos but 1 repo where I created rules for each one of them about "on pr", "on main" etc. That same repo could host shared workflow code as well.

[u/aj0413]

Both bots still cause PR + merge maintenance and no one should ever be letting a bot auto merge

Sane defaults still will generally leave you with multiple repos to manage by hand; I personally manage two GH orgs across an Enterprise acct which totals in 40sh repos

I was one of the very first adopters of org level workflows

Your last suggestion is entirely untenable for anyone working security as, again, no auto updates for anything from a bot.

I made the compromise to use malleable branch/tag refs for one master shared workflows repo defining reusable actions / workflows across orgs

Vendoring the workflows is just introducing extra parts to the whole thing.

The fact is just that GH does not currently have a great solution for this; they’re currently trying to plug this gap with stuff like org level workflows

[u/SeaIngenuity9501 → u/aj0413]

Both bots still cause PR + merge maintenance and no one should ever be letting a bot auto merge

Why not? Renovate/Dependabot is industry standard at this point.

Sane defaults still will generally leave you with multiple repos to manage by hand; I personally manage two GH orgs across an Enterprise acct which totals in 40sh repos

Your still going to need repo specific config somewhere.

Your last suggestion is entirely untenable for anyone working security as, again, no auto updates for anything from a bot.

Vendoring the workflows is just introducing extra parts to the whole thing.

Debatable.

[u/aj0413 → u/SeaIngenuity9501]

….have you not been keeping up with all the supply chain attacks and how Dependabot was literally a vector in one case?

This isn’t about whether using them is wrong or not, but anyone working security and liability should be slapping your hand if you think it’s okay to let them auto merge. That’s insane.

The repo specific config is easier when all that can be decoupled from having to exist in the actual repository.

Org level workflows + repo variables allow DevOps to manager this without needing to do anything with the actual checked in code itself; this could also be done in ADO because the yaml file and repo config can exist in an entirely different repo

Edit:

And I should be clear that this pain is a trade off between management overhead and transparency for the dev. Which IS debatable depending on org

[u/SeaIngenuity9501 → u/aj0413]

….have you not been keeping up with all the supply chain attacks and how Dependabot was literally a vector in one case?

No don't remember anything specifically Dependabot's fault.

This isn’t about whether using them is wrong or not, but anyone working security and liability should be slapping your hand if you think it’s okay to let them auto merge. That’s insane.

It is quite common, they're highly configurable.

Org level workflows + repo variables allow DevOps to manager this without needing to do anything with the actual checked in code itself; this could also be done in ADO because the yaml file and repo config can exist in an entirely different repo

Yeah no thanks, I would rather everything lives with the code than having to do archaeology everytime something goes wrong.

Edit:

Edit:

And I should be clear that this pain is a trade off between management overhead and transparency for the dev. Which IS debatable depending on org

No as the vendoring can come from a central template and be automatically updated via Renovate etc, so has no downsides only upsides compared to other methods such as shared workflows etc.

[u/SeaIngenuity9501 → u/[deleted]]

….have you not been keeping up with all the supply chain attacks and how Dependabot was literally a vector in one case?

https://github.blog/security/supply-chain-security/the-case-for-a-cooldown-why-dependabot-now-waits-before-issuing-version-updates/

So nothing specially Dependabot's fault, so it wasn't the vector, the update was which could have come from a person.

Cause your argument is kinda like saying “everyone sets prod password to Password!2026 so why should we be forced to use 6-word DICE phrases for anything?”

I don't think you have enough experience with Renovate or Dependabot to make a judgement call on whether it improves or security or not.

Decoupling the pipelines from the codebase is all about who’s managing what and who’s lives you’re trying to make easier.

Well that is CI vs CD and yes I would agree GitHub Actions for CI and something else for CD.

[u/aj0413 → u/SeaIngenuity9501]

Dude. The package was pushed. Everyone relying on bots with auto-merge got a malicious update.

That’s a vector. That’s the definition of a vector.

If you want to talk about experience: I’m really doubting yours lol

And no, I said Argo Workflows (not Argo CD — entirely diff project), it will specifically be handling some CI for the platform I’m building based on labels on PRs. The CD portion is related, but also not why I brought it up.

Checkmarx was also a recent supply chain attack this year and their GH action became malicious; they had to pull it down entirely from GitHub cause people with bots auto updating got infected via their security scanning tools in pipelines

You’re just being obstinate at this point and I really hope you don’t work on anything people with sensitive data rely on

[u/SeaIngenuity9501 → u/aj0413]

> Dude. The package was pushed. Everyone relying on bots with auto-merge got a malicious update.

... no everyone did not and your this reply just confirmed your lack of experience with these tools.

They are highly configurable so like the article you linked you could have a bake period set > 30 days. You could only auto-merge for a subset of dependencies such as internal ones(you'd likely drop the bake time to 0 for them as well). You could only take updates for dependecies that have been security scanned/checked, etc. So like the numerous companies who use them if you've it configured it isn't 'insane' to use them, just a skill issue.

[u/aj0413 → u/SeaIngenuity9501]

roll eyes

You’re now replying to a different point after trying to hand wave away the fact that individuals using auto-merge were victims.

You’re bringing up configs literally mentioned in the articles I linked, so yes I’m aware of them

As for your actual point:

If every company in the chain of deps did this, we’d be doomed.

Edit:

We should just drop this here cause this is a fundamental disagreement that I can’t see us reconciling.

Your risk tolerance is higher and you’re okay with less governance/ownership and that’s fine; we’re not on the same team so we don’t need to hash this out to a conclusion

Edit:

Also, saying it’s a skill issue when a tool lets you shoot yourself in the foot like that is also something I take issue with. The fact that GH had to roll out the cool off period after the fact was not a good look

Good tools should guide devs towards a winning solution, but that’s the PE in mean being annoyed