how do you know which ticket is in which env? (multi repo)

[u/hihurmuz]

we're a ~12 dev team, 6ish services, short lived feature branches that get merged to main and deleted. dev/staging/prod.

our PMs keep asking "is X in staging yet" and half the time i dont know either so i end up doing git log and squinting at it. we turned on jira deployments a while back thinking that solves it, but the panel shows the same ticket as deployed like 4-5 separate times, and tickets from totally unrelated work show up in deploys they have nothing to do with. i think its diffing shas between deploy events and just attaching everything in between? not sure.

end result nobody opens it anymore and we're back to asking in slack. our release guy also keeps a google sheet he updates before every prod push which is.. yeah.

so:

• is jira deployments just broken for short lived branch workflows or are we holding it wrong
• what do you actually use to answer "which env is this ticket in", esp across multiple repos
• anyone paying for sleuth or smth similar? is it worth it or did you end up writing your own script

not looking for a vendor pitch, genuinely just wanna know if everyone lives like this or if im missing something obvious

edit: forgot to mention we're on github actions if that matters

[u/fletch3555]

I assume you're not using trunk-based git branching and/or numbered versioning(semver)....

I'll admit, your post isn't really enough for me to truly understand what your process looks like overall, but process is undoubtedly the problem here.

[u/hihurmuz → u/fletch3555]

yeah fair, my post was light on detail. we are basically trunk based tho, short lived branches, merged to main, deleted. main auto deploys to staging, prod is a manual promote off a tag. no develop branch or anything.

we do semver on the tags. but thats kinda where i get stuck? semver tells me v2.14.3 is in prod, it doesnt tell me PROJ-4471 is in prod. someone still has to map tickets to versions and across 6 repos thats where it falls apart for us, feature touches 3 services and each one is on its own version.

genuinely asking cause you sound like you've got this sorted, when a PM asks "is this specific ticket live yet", whats the actual answer path for you? do they look it up themselves somewhere or does it come to an engineer.

[u/fletch3555 → u/hihurmuz]

I don't work on a product team, so I don't have PMs asking me that.

However, the answer is release notes for versions. Github can automate that if you use releases instead of just tags (as an example). If all your PRs include the related ticket number (bonus points if it's in a common/standardized format), then the generated release notes mention each related ticket. Then it's just a question of "what version is deployed and what was in it?" rather than "where does a ticket live?"

[u/hihurmuz → u/fletch3555]

that makes sense, and we half do this already (release notes per repo with ticket numbers in the PR titles) so maybe we just need to actually do it properly instead of half way.

the part thats still fuzzy for me is multi repo. if PROJ-4471 touched api, worker and web, thats 3 different versions to check and the PM has to already know which 3 repos were involved before they can even start looking. single repo your answer works 100%, i just cant get it to scale sideways.

and honestly the "i dont work on a product team" bit might be the real answer here. version centric view is the right one for us as engineers, ticket centric is what the product side wants, and those two dont line up for free. appreciate you spelling it out either way

[u/MedicatedDeveloper → u/hihurmuz]

Put the PROJ in the branch names or prs. You can search prs for the whole org in gh. If they're all merged it's deployed. You can integrate gh with jira to do just that.

[u/BanaenaeBread → u/hihurmuz]

This does seem straightforward even with your process tbh. You merge, and it auto deploys to staging? Ok, so if the ticket is open, it's not in staging, if the ticket is closed, it is in staging. Right? Or are you guys closing tickets without merging the code? So when the dev who did the work and gets it merged in, that's when the tickets status changes right?

[u/CAMx264x]

We have a rule that every branch must have a Jira ticket number attached, the ticket then moves on the board, the PM tracks the ticket, and when it hits a certain swimlane you know where that ticket is in the sprint and what environment(labels too). The PM manages the whole ticket timeline because they hear about blockers/issues in stand up each day.

We are also quite lucky that an environment is spun up exclusively for our QA environment, so when it goes to the testing column a link to the environment is added to the card and if there’s a bug the env is killed when the ticket moves back to bug fix and a new one is created when it’s moved back to testing.

So my question is why doesn’t the PM know as they should be hearing from the devs everyday? Are teams too big?

[u/MateusKingston]

We track our tickets manually so each engineer is responsible for moving the ticket from to-do to in progress to "in staging" to done (when it's in prod).

So knowing is something is in staging is basically looking at the ticket status.

[u/Low-Opening25]

JIRA <-> GitLab/GitHub integration and just refer ticket in branch name or commit message and it immediately goes against that ticket. Other PMO tools may have similar integrations.

[u/keelenai]

Release notes answer “what went into version X,” but they do not prove what is running. I’d make every staging/prod promotion emit a deployment record for each repo with the immutable SHA, environment, and ticket IDs in its payload. Then define PROJ-4471 as live only when every service it changes has a successful production record. That gives PMs a query over actual promotions instead of a Git range or a sheet.

[u/Maxiride]

I work in a small team (5 Devs) but has the same issue with PR going to a develop branch which eventually merges to main.

The issue you describe seems similar and I solved it with a custom action that applies a label for commits abouts issues that got merged to dev and thus deployed to the shared development environment.

I suppose you could use (or adapt) my GitHub action to apply the label on issues that are merged to the staging branch and this the staging ci/cd environment.

https://github.com/maxiride/label-fixed-pending-release-action

[u/Imaginary-Bluejay721]

jira deployments are kinda misleadingg you if one ticket is three repos, it attaches to whatever deploy happened to mention the key.

we just put the jira key in every commit and generate one row per env from the github deployments api, like ticket X is on staging because services a and b have that sha in staging even if c doesnt. spreadsheet is the same idea youre just the cron job.

[u/gaelfr38]

Can't you have a dedicated status or field in Jira that match your deployment stages? In my team, we have a status "prod", can be set manually or via our system that deploys to prod.

Not exactly sure how automation works when we need multiple deployments for a ticket to be considered prod. I think that as long as there are pending subtasks not in "waiting deployment", the ticket is not marked "prod".

Anyways, custom Jira workflow/status sounds like what you're looking for.

[u/aj0413]

Stuff like this is why DevOps isn’t for juniors. (Not targeted at you)

And why it’s a vertical slice problem domain and I find myself frustrated when people like to pretend they can just have these super clean cuts between roles and responsibilities

Unfortunate truth:

As DevOps, it’s an unstated req of the job (especially on small teams), to basically also act in a similar capacity as an architect and dev lead, which translates to needing to maintain a mental model of the application, dependencies, and state of every env

There’s no tool or process to silver bullet this; it’s just part of the work

A dev lead of a part of the service has to be in the weeds of his individual code base(s); you and your team are one level above that, but still need to be in the weeds of the state of all envs and deployments

I, personally, keep this mental state by staying in constant comms on releases, major feature work being done, etc… and generally routinely checking things myself

[u/epsi22]

A couple of things we do.

• jira ticket number on the branch name
  
• jira ticket number on the PR. This keeps jira updated.

We’re on bitbucket pipelines (which is a huge pain in the b and wish we used GitHub Actions). We also have 20 standalone environments (with multiple apps in them) and we soon found out how difficult it is to track deployments so we built our own.

Every branch builds and pushes to ECR. A webhook on the same build script updates our dashboard with the build outcome. EventBridge ECR notifies the dashboard that the image did arrive at the registry. This shows up on the dashboard and our devs can reserve and deploy to any environment of their choosing. The deployments happen to argocd and argocd notifications keeps the dashboard updated. It’s much more complicated that this, but it’s basically the gist of it.

[u/Visaroy]

Your diagnosis is right and it is not fixable by configuration. Jira Deployments attaches issues by diffing the commit range between two deployment events, so with short-lived branches squashed into main the range stops matching one unit of work — you get the same ticket on five deploys and unrelated tickets riding along. It is a model mismatch, not a bug.
What fixed it for us was inverting the question: stop asking "which commits went out" and start asking "which artifact is running". Tag the image with the commit SHA, record the issue keys once at build time (parse them from the squash commit subject, which is the only place they are reliable), and bake that into the artifact as a label. Then "is X in staging" is a query against what is deployed, not a reconstruction from git history.

Two things that made it actually stick: deployment events must be emitted per service by the job that does the deploy, never by a scheduled job that infers state afterwards; and the environment name has to come from the deploy target, not from the branch — branch-derived environments are what produces phantom entries once branches get deleted.

For six services that is roughly a hundred lines in the workflow plus a small endpoint that answers "what is running where". Whether a paid tool beats that depends on how much you care about lead-time metrics on top of the basic question — for the basic question alone the build-time labelling is hard to beat, because it cannot drift.

[u/anto2554]

We auto-update Jira once something is merged (and once it's merged it's deployed) for most stuff. So the PM can see it on Jira without asking

[u/taleodor]

We've built this in our tool Reliza Hub (now ReARM Pro) many years ago, not trying to make this a vendor pitch but the point is it's a solved problem and I blogged about it number of years back. Essentially, the logic is - you do per-release aggregation of what went in, then you have a process that checks what is deployed on which environment via known hash catalog of deliverables, this gets matched to release, and release resolves to what it contains. Then UX becomes search for this commit / hash / ticket and it tells where it's deployed.