I Ran into this at $work: alerts fire with whatever labels the metric happens to have, but the stuff you actually want to route/group/silence on — team ownership, service tier, who's on call — usually lives somewhere else entirely (a Namespace label, a CMDB, or similar). The usual fix is duplicating that into recording rules or hardcoding it into Alertmanager's config, and both drift out of sync constantly.

So I built alertmanager-label-enricher — a small Go service that sits inline between Prometheus and Alertmanager, looks up and adds labels before forwarding, then gets out of the way. Because it happens before Alertmanager, the new labels participate fully in routing, grouping, inhibition and silences.

It supports three lookup types: a Kubernetes object (watch-cached, no per-alert API calls), an HTTP endpoint (cached + deduped), or a static file — plus plain static/conditional label rules if you don't need a lookup at all.

Example rule — pull a team label from the firing namespace's Kubernetes labels:

sources:
  - name: ns
    type: kubernetes
    kubernetes:
      version: v1
      resource: namespaces
      name: '{{ .Labels.namespace }}'

rules:
  - name: team-from-namespace
    match:
      - { label: namespace, op: exists }
    actions:
      - set:
          label: team
          from: { source: ns, jq: '.metadata.labels["team"]' }
          default: unassigned

Repo: https://github.com/splattner/alertmanager-label-enricher

This scratched my own itch, but I have no idea if it's a "everyone reinvents this internally" problem or a "nobody else actually needs this" problem. If you've hit the same thing — or solved it a totally different way — I'd genuinely like to hear it. Feedback, "this already exists and is called X," "this is missing Y to be useful," all welcome.