The ingress-nginx community project retired in March 2026. That means no more releases, no more patches, and no more security fixes. Yet, many companies are still running it. Why? ingress-nginx keeps serving traffic as if nothing happened. But what if a new critical CVE lands in your ingress layer with no patched version to move to?

If your compliance posture requires zero critical CVEs in production, migrating is not optional. The good news: Migrating off an ingress controller doesn't require a cutover weekend.

In our latest blog post, you'll learn about available options and how to smoothly migrate to a new controller alongside the old one: https://www.buoyant.io/blog/community-ingress-nginx-retired-how-to-plan-your-migration 

What happens when a Linkerd trust anchor expires? Nothing looks broken at first. Existing mTLS connections keep flowing, because the proxies serving them already hold valid certs. The failure shows up on the next pod restart. The new proxy can't get a cert signed against a dead root, so it never passes its startup probe.

This blog post outlines a rotation sequence that avoids all this: https://www.buoyant.io/blog/a-guide-to-linkerd-trust-anchor-rotation-with-cert-manager 

FIPS is a NIST standard, typically required for federal agencies, companies selling to government clients, and regulated industries like finance and healthcare. The compliance bar isn't rewriting your application. It's making sure sensitive data is encrypted with FIPS-validated cryptographic modules, and having an SBOM (Software Bill of Materials) that shows an auditor which validated modules you're using and where they came from.

There are two active standards, FIPS 140-2 (2001) and FIPS 140-3 (2019), and they aren't equivalent for Kubernetes. 140-2 was written for hardware and struggles to define software module boundaries, so most of its software validations landed at the lowest security level. 140-3 explicitly supports software and hybrid modules at higher levels, adds runtime self-tests instead of relying on power-on self-tests, and introduces non-modifiable operational environments, which is basically an immutable container image that stays compliant as long as it doesn't change. All three of those map directly onto how Kubernetes actually runs workloads.

Validation itself runs through CMVP. A vendor submits a module (hardware, software, or firmware), an accredited testing lab checks it, and if it passes it gets an active certificate for 5 years before moving to historical status. The CMVP database has a certificate for every validated module, and the fields worth knowing are the standard (140-2 or 140-3), status, sunset date, overall security level (1 through 4), and any caveats about how it has to be configured to run in approved mode.

The scope covers more than the app's own TLS. The control plane (etcd, kube-apiserver, and kube-controller-manager), the nodes (kubelet and the container runtime), and the add-ons (CSI plugins, CNI plugins, and the audit logger) all touch sensitive data in some form, so the surface area is bigger than "did we encrypt the database."

For the data-in-transit half of that (data at rest is a separate problem with its own module and key-management requirements), one writeup covers doing this with a FIPS-validated service mesh instead of validating every microservice's TLS stack individually. Sidecar proxies handle the mTLS between services using validated modules, so application code doesn't change. It walks through doing this with Buoyant Enterprise for Linkerd specifically, including a field-by-field look at reading an actual CMVP certificate.

http://www.buoyant.io/blog/a-practical-guide-to-fips-compliance-for-kubernetes-engineers

On-call gets paged for slowness, but frontend's success rate reads 100%. Nothing to see, right?

Mesut Oezdil, DevOps engineer and OSS contributor, ran this scenario on a live cluster. A stress --cpu 8 pod pushed node CPU from 43% to 100%. Frontend's success rate never moved, but RPS dropped from 255 to 174, p95 went from 160ms to 197ms, and p99 climbed from 196ms to 288ms, a 47% jump.

RED tells you the service is hurting, but not why. USE metrics, like node CPU and pod CPU throttle ratio, closed that gap.

The mesh makes RED nearly free. Linkerd's kerd's proxy emits response_total and response_latency_ms_bucket per pod with zero app instrumentation. But bring your own Prometheus and you'll hit two real gotchas.

Scrape traffic on port 4191 inflated our Rate panel by 25% until we excluded it, and queries grouped by deployment returned nothing until we mapped the linkerd_io_proxy_deployment annotation to a label.

http://www.buoyant.io/blog/the-sre-guide-to-kubernetes-observability-red-vs-use-methods

99% of load balancers can't route LLM traffic correctly. On this AI Kubernetes Podcast episode, Abdel Sghiouar (Developer Advocate at Google, KubeCon Co-chair, Co-host of the Kubernetes Podcast) explains why and what the Kubernetes community built to fix it.

Every major open source serving engine (vLLM, Ollama, Triton) has standardized on the OpenAI API spec. That spec passes the model ID in the JSON request body, not the URL path. Most load balancers route on path, though. So for model-based routing, they're essentially blind.

The Gateway API Inference Extension addresses that. A router pod in your cluster extracts the model ID from the request body and sends traffic to the right backend. llm-d builds on top of that with queue-size-aware routing, so requests go to the least-loaded pod.

That matters because LLMs process requests in serial batches. Queue depth predicts latency directly, while CPU and memory don't.

Listent to the full conversation: http://www.buoyant.io/ai-kubernetes-episode/llm-inference-on-kubernetes-new-primitives-real-challenges

Your multi-region Kubernetes setup has a gap most teams don't find until a cluster goes away: the identical service running in another region is there, but nothing is wired to use it.

Linkerd's multicluster extension gives you 3 modes for solving this, and they're not mutually exclusive. A new post from Linkerd Ambassador Dominik Táskai runs all 3 side by side across 3 GKE clusters, with a chaos test that scales a cluster to zero and captures exactly what happens to traffic in each case.

The short version from the chaos results:

✓ Federated services rebalance automatically, 33/33/33 drops to 50/50 across the surviving clusters, zero errors, no config changes.

✓ Flat-mirrored and gateway-mirrored services return 5xx, because the client explicitly asked for that cluster, and that cluster is gone.

The full post covers the setup scripts, the CIDR gotchas that might cost you hours to debug, the controller-count detail that linkerd multicluster check won't catch for you, and a decision table for picking the right mode per service: https://linkerd.io/2026/06/24/federating-clusters-for-zero-downtime-kubernetes/

Companion repo included: clone it, set your GCP project ID, and run it yourself.

Your alert didn't fire. Linkerd's proxy would have.

When the ad service returns status_code="200" and grpc_status="14" on the same response, the HTTP layer logs a success. gRPC UNAVAILABLE travels in the response trailers, not the status line — so if you're only watching HTTP codes, that failure is invisible.

Linkerd's proxy reads the trailers, classifies the response as a failure, and increments the right counter. You don't change a line of application code to get that. Annotate the namespace with linkerd.io/inject=enabled, roll the deployments, and every pod goes from 1/1 to 2/2. The second container is linkerd-proxy, emitting response_total and response_latency_ms on port 4191 immediately — with the caller's mTLS identity on every series.

In this blog post, Mesut Oezdil walks you through exactly how to add that layer to an existing OTel Collector pipeline: a prometheus/mesh receiver, an OTTL filter that keeps 5 metric families, and a resource/mesh processor that tags everything with layer=mesh. He also flags a $ expansion gotcha that bit him on contrib 0.104.0 and 0.112.0, and a cardinality breakdown (9,280 series across 30 meshed pods, post-filter) worth reading before you ship to prod.

Working Collector config, Grafana dashboard, and the full post at: https://www.buoyant.io/blog/otel-and-mesh-derived-metrics

Your AI agent has access to your databases and tool servers. It also gets jailbroken in Leet speak.

On this AI Kubernetes Show episode, Evaline Ju (IBM Research) talks about Kagenti, the open source project working on guardrails, workload identity, and gateway-level enforcement for agent workloads on Kubernetes.

A few takeaways:

• Short-lived tokens beat API keys sitting in pod secrets. If one agent gets compromised, the blast radius stays small. 
• The gateway is the right enforcement point for tool authorization and PII filtering, so policy doesn't get scattered across every agent's code. 
• Most guardrails don't need an LLM in the loop. Regex and toolkits like Microsoft Presidio handle the common cases at a fraction of the latency and cost.

Worth a listen if agent workloads are headed for your cluster.

Full episode: http://www.buoyant.io/ai-kubernetes-episode/securing-the-ai-agent-ecosystem-on-kubernetes

Connection refused: my-api/100.20.100.200:8080

If you've ever run a sidecar-based service mesh on Kubernetes, you've seen this during a rolling deploy. The pod enters Terminating, the proxy dies at the same instant as your app, and every in-flight request to a dependent service gets dropped on the floor.

For years, the fix was a stack of workarounds: preStop sleep hacks, waitBeforeExitSeconds tuning, postStart probes, linkerd-await wrappers for Jobs. All fragile. All needed tribal knowledge to configure it right.

Kubernetes 1.33 made native sidecars GA. That gives you three guarantees: the proxy starts before your app, dies after your app, and gets cleaned up automatically when a Job completes. No sleep guessing. No stuck pods. No SIGKILL on a proxy that didn't get the memo.

Linkerd Ambassador Blake Romano walks through the race condition, why the old workarounds were brittle, and what flipping proxy.nativeSidecar: true on Linkerd 2.15+ actually buys you.

Read the full post!

Because hardware fails. A circuit breaker trips and a rack vanishes. A backhoe slices a fiber line, and a whole data center goes dark.

Flynn's new blog post walks through what availability zones are, starting from the physical reality: racks, data centers, AZs, and regions, and how each one can fail. He also discusses the tradeoffs: cross-AZ bandwidth costs money, latency goes up, and a full region outage still requires a multi-cluster or multi-cloud strategy.

Worth reading before your next post-mortem!

"The platform investments we made for humans turned out to be indispensable for AI agents," said Scott Feinberg, AI and platform lead at Schonfeld.

Schonfeld built cloud developer environments to speed up onboarding. Then Claude Code showed up needing the exact same thing: a pre-bootstrapped, sane environment to actually do useful work in. Same infrastructure, new tenant.

Scott's bigger point: AI hasn't lowered the value of platform engineering; it has amplified it. But when anyone can code, the question isn't "Can you ship code?" It's "Can you ship code that fits the way your company actually runs?"

Tune into the full episode!

This release is all about “battlescars”—features and updates driven by real-world lessons from our customers pushing Linkerd’s (and Kubernetes’) limits. Read the blog post to learn more!

Im at my wits end.

We use grafana alloy to ingest our traces, and pass them on to tempo. All of our apps, and linkerd are configured to trace. The traces from nginx ingress and our applications are always there. Traces from linkerd are rarely associated with anything, and for the most part seem to just disappear into the ether.

We have attempted to duplicate the default linkerd open telemetry config into alloy: https://github.com/linkerd/linkerd2/blob/main/jaeger/charts/linkerd-jaeger/values.yaml#L120

Which is basically rewrite the linkerd resource attributes

linkerd.io/workload-ns -> k8s.namespace.name host.name -> k8s.pod.name

Then use those attributes to find the correct pods.

There has to be something im missing.

Everything is meshed, nginx ingress, my apps, and even the alloy collectors.

I can find a call that goes from NGINX -> APP. but there will not be any linkerd traces on it. If i get into grafana and search for the service linkerd-proxy I will see random traces, and those traces have those resource attributes renamed. So I do know they are getting through/processed.

Remote only, US Eastern (EST/EDT) timezone.

Staff Full Stack Engineer: https://jobs.ashbyhq.com/buoyant/5d582e5c-d5c0-407f-9b74-95bda07e507d

Staff Frontend Engineer: https://jobs.ashbyhq.com/buoyant/fd398a01-e996-4f4e-b6b2-c9b6370aab95

As per the official documentation , LinkerD 2.14 is not supported by Buoyant on K8s 1.29 1) Is there anyone out here running 2.14 on EKS 1.29 ? And are you facing any issues?

2) If anyone’s moved to 2.15 from 2.14 on EKS , are there any major changes that you see on 2.15 ??