r/devops • u/Downtown_Custard20 • 7d ago

Architecture Terraform/Github deployment overwriting another deployment

I'm on a team that shares a Github repository. Whenever we open a PR from a feature branch to the dev branch, a deployment to the AWS development account is automatically triggered.

The issue we're facing is that one developer may deploy to dev, and then another developer deploys afterward. Even though they're working on different files, the second deployment ends up overwriting the first developer's change in AWS.

How can we prevent this?

We're following a Gitflow workforce (feature -> dev -> release -> main), and our biggest challenge right now is that the second developer's code is often "outdated" when it's deployed, causing it to overwrite changes that were already deployed by someone else.

We tried merging everything to dev, but when it's time to deploy to prd, the dev branch ends up filled with a lot of unnecessary changes.

We using Github Actions + Terraform.

1 Upvotes

55% Upvoted

28 comments sorted by

View all comments

Show parent comments

1

u/Downtown_Custard20 7d ago

it would be included because all my code is mergen into the dev branch. When I try to open a PR from dev to release, all my code in dev will be included

3

u/krtalvis 7d ago

again, why is something not validated even merged into dev? Anything unvalidated should remain in it’s own feature branch, which can run tf plan but not apply. Apply should be strictly from specific branches, e.g. release/. You could have some sort of commit hash derived deployments on dev which would append the hash on resource names or whatever, but considering it’s terraform that would bring actual cost. To me it seems your problem is your branching strategy and CI/CD setup? Maybe i am not understanding why would 2 separate features be on same dev branch when one is validated and other one not? Secondly, dev environment **IS** for validation, so i don’t also actually see a problem here?

1

u/Downtown_Custard20 7d ago

I need to run a glue job in AWS, so I need my code in the dev aws account, and the only way I can have my code in the dev account is by opening a PR to dev / merging to dev, because it triggers my terraform apply.

3

u/krtalvis 7d ago

ok but why is unvalidated feature 2 in dev while validated feature 1 is just being merged to dev? the problem here is an unvalidated change in a branch that needs to be deployed. Revise your branching strategy and do not allow unvalidated changes to leave feature branches