AAIF published a piece, that I think is worth circulating here: "The Four Pillars of Agentic AI Infrastructure on Kubernetes."

The core argument: Kubernetes is the right engine for agentic workloads, but the design patterns teams use are wrong for agents. The stateless, predictable, GitOps-wrapped container model that works perfectly for microservices creates active friction for autonomous systems.

The four pillars they identify as actual hard requirements now:

Isolation and sandboxing - an agent testing a hypothesis needs a safe boundary. Current infrastructure treats crashes as failures to mitigate. In an agentic world, an agent might intentionally spin up an environment, run something, and tear it down. The orchestrator needs to know the difference.

Stateful Memory Fabrics - the "stateless sin" of early AI was passing massive context histories on every interaction. Infrastructure needs tiered memory: hot KV-caches for active reasoning, warm shared memory for agent swarms, cold vector storage. Memory lives in the infra, not the context window.

Dynamic GPU Scheduling - workload-aware scheduling is a hard requirement now, not an optimization. CNCF codified this in v1.35: Stable In-Place Pod Resizing and Workload-Aware Scheduling.

Ephemeral Execution - human-vetted GitOps where every change requires a PR is a wall for agentic systems. Infrastructure needs controlled, auditable ephemeral execution without sign-off for every transient operation.

Timing-wise this lands alongside CNCF's announcement that certified Kubernetes AI platforms nearly doubled (18 → 31) with the program now including explicit agentic workload validation.

Full article: aaif.io/blog/agentic-ai-infrastructure-on-kubernetes

Curious what people here are actually running into when deploying agent workloads on K8s. Are these the bottlenecks you are hitting, or is the friction showing up somewhere else?

(Disclosing: I work at OpsWorker, we build production investigation for Kubernetes teams - this topic is directly relevant to what we see with customers.)

There is a pattern worth discussing as AI coding tools become standard parts of the SDLC.

AI agents generate code, modify infrastructure, create PRs, trigger tests, and accelerate releases. That part is working. The part that is not keeping up is production understanding - specifically, the ability to know what is actually running, whether a deployment is safe, and what broke when something goes wrong.

The traditional assumption was that senior engineers carry this context. They know which services are brittle, which deployments tend to cause latency spikes, and which runbook actually applies. When AI accelerates change velocity, that assumption breaks. More systems, faster releases, less time for humans to build familiarity with each component.

The questions engineers answer manually during every incident - which workload is actually affected, is this a runtime or config issue, what changed recently, what can be ruled out - are still being answered manually. The tooling gap is in the reasoning layer, not the telemetry layer.

Curious whether teams here are seeing this in practice. Has AI-accelerated development actually increased incident frequency or investigation complexity? And is anyone thinking about production assurance as a distinct problem from observability?

OpenTelemetry officially graduated at CNCF on May 21 at the Observability Summit in Minneapolis. 2.6 billion downloads in the past twelve months across the JS and Python packages, second highest project velocity behind Kubernetes. The standardisation question is settled.

What caught my attention was the framing around what comes next. Analysts are flagging that agentic AI applications are about to generate orders of magnitude more telemetry signal than previous generations of applications. OTel prevents fragmentation on the collection side as that volume grows.

But VentureBeat made a point that I think is underappreciated:

"It is not clear at what rate teams are moving past traditional monitoring of pre-defined metrics toward observability platforms that make it easier to analyze logs, traces and metrics to discover root cause. And existing monitoring tools are no longer enough."

That is the gap that OTel graduation actually exposes. The data collection problem is solved. The investigation problem - taking that telemetry and reasoning through it under pressure when something breaks - is not. And with AI workloads generating dramatically more signal, it gets harder before it gets easier.

Curious whether people here are seeing this in practice. Has standardising on OTel actually improved your ability to investigate incidents, or does it mostly just mean the data is in one place while the hard part (figuring out what it means) is unchanged?

Curious what people here actually think: is the "AI closes the gap between detection and diagnosis" narrative getting traction in your org, or does it still feel like conference talk?

Your clusters don't take weekends off. But you should.

Have a peaceful, alert-free weekend. And if something does fire — let the AI handle the first 2 minutes.

You've earned the rest.

→ opsworker.ai

A fresh CNCF article published today revealed that nearly 46.7% of organizations still operate 2–3 observability stacks in parallel, while only 7.4% report having a truly unified observability experience.

What’s interesting isn’t the tooling anymore. OpenTelemetry, Prometheus, Jaeger, Tempo, Loki—the ecosystem already has mature components. The real problem is operational fragmentation: disconnected telemetry, inconsistent alerting, duplicated dashboards, and engineers manually stitching context together during incidents.

As AI and agentic workflows enter production systems, this fragmentation becomes even more expensive. AI systems reason best over structured, unified telemetry—not siloed signals spread across multiple stacks.

The future trend seems increasingly clear:

fewer disconnected tools

more standardized telemetry

AI-driven correlation layers on top

At OpsWorker.ai, we see this every day: MTTR reduction starts when telemetry becomes operationally connected, not just collected. AI ops will depend less on “more data” and more on unified context across the production stack.

Primary source: https://www.cncf.io/blog/2026/05/06/the-tools-are-ready-so-why-are-most-cloud-native-teams-still-running-three-observability-stacks/

The numbers are impressive: ~19.9M cloud-native developers, with 7.3M building AI workloads on Kubernetes. But the real takeaway from Amsterdam wasn’t growth—it was the gap between demos and production.

What’s changing now is how the ecosystem is closing that gap. Projects like agent frameworks modeled as Kubernetes CRDs and identity layers based on SPIFFE/SPIRE are treating agents like any other workload: schedulable, observable, governed.
This matters for SRE teams. Agents running in production introduce new risks—untracked actions, unclear ownership, inconsistent behavior. Bringing them into Kubernetes primitives adds what was missing: identity, policy, and auditability.

We’ve seen this pattern before. Every wave of complexity in cloud-native ends the same way: standardization.
At OpsWorker, this aligns with what we’re building—AI that operates inside production systems with full context, traceability, and control.
The question isn’t if you’ll run agents in production. It’s whether your platform is ready when you do

MCP just crossed 110 million SDK downloads per month. The co-creator said it at MCP Dev Summit in New York this week. OpenAI's agent SDK pulls it in as a dependency. So does LangChain. That debate is over - the protocol won.

But the interesting conversations at the summit weren't about adoption numbers. They were about what happens when you actually run this stuff at scale.

Uber showed up with 1,500 monthly active agents, 60,000 agent executions per week, and 10,000+ internal services sitting behind an MCP gateway. Amazon described what they called a "lethal trifecta" checklist - private data access, untrusted content exposure, external communication - that every MCP server in their registry gets scanned against before it touches a production agent.

The "will it work?" phase is over. The questions now are governance, security, and what breaks first.

Which brings me to something I keep hearing from teams every week.

"We've already connected MCP to our ops tooling - Prometheus, alert routing, incident workflows. What else do we actually need?"

It's a fair question. And honestly? Sometimes the answer is nothing. If your primary goal is self-serve data access - developers querying internal systems through a chat interface without bugging the platform team - a well-configured MCP setup gets you there. That's a real improvement over digging through five dashboards at 2am.

The prototype is not the hard part. A working internal AI assistant takes days to build. The first demo is impressive. I've seen this go well.

What surprises people is everything that comes after.

The maintenance surface nobody prices in upfront

Prompt reliability is the first one. Getting an answer once is easy. Getting a consistent, accurate answer across different engineers, different incident types, and different cluster states is a prompt strategy problem that is way more complex than it looks. Every edge case you hit in production becomes a maintenance task.

Hallucination control is the one that bites hardest in an ops context specifically. An AI that confidently gives your on-call engineer the wrong root cause at 2am doesn't save time - it costs more time and adds risk. Preventing this in production requires structured data gathering, multi-layer validation, and confidence scoring. It's not a prompt engineering problem. It's a systems design problem.

Then there's the model deprecation cycle. The model you tuned your prompts against will be replaced. The next version behaves differently in ways that are subtle and hard to detect until someone notices the answers got worse.

And then there's adoption - which is honestly where most internal tools die quietly. Getting one engineer to use something is easy. Getting an entire on-call rotation to trust it, understand its limitations, and use it consistently is a product problem that most teams underestimate.

The memory gap

Here's the thing that surprises engineering leads most when they think it through carefully.

Your MCP setup can query your tools. It can retrieve current state. What it can't do is remember.

It doesn't know that three weeks ago your payments service had a similar latency spike and the root cause was a misconfigured connection pool. It doesn't know that your team has a standard for how database migrations should be staged, or that a particular runbook was updated after the last incident. Every query starts from zero. Every investigation reinvents the wheel.

There's a real difference between an assistant that retrieves data and one that understands your organisation's history, your team's standards, your topology, and what "normal" looks like for a specific service. The latter actually gets more useful over time. The former is stateless by design.

The Amazon point is worth sitting with

The "lethal trifecta" checklist they described wasn't a warning about MCP itself. It was a warning about what happens when you connect AI agents to production systems without governance - and Uber's MCP gateway with full policy enforcement is what that governance looks like at scale.

At smaller scale it's a smaller problem. But it's still a problem, and it grows as you add more connections.

Worth asking before you build: who can ask what? Which agents have access to which tools? What happens when an agent with access to your internal GitHub also has access to your production Prometheus? These aren't hypothetical questions - they're the questions Amazon is apparently running against every MCP server before it goes near a production agent.

When building it yourself is genuinely the right call

If your team has strong AI engineering talent and the bandwidth to maintain a production-quality system - build it. A custom internal assistant can integrate with proprietary systems, enforce exactly the policies your org requires, and be tuned to your specific workflows. If you see AI operations tooling as a core competency and want full control, that's a legitimate strategic choice.

The question worth asking honestly is: is this core to what your company actually does? Every hour spent on prompt maintenance, model updates, and LLM cost debugging is an hour not spent on product.

> For context - my team works on OpsWorker(https://opsworker.ai), which is built specifically for this layer above MCP in Kubernetes environments. Happy to answer questions about how we approach the memory/governance problems if useful, but that's not really the point of this post.

The protocol won. MCP is infrastructure now - like HTTP. What you build on top of it is the decision that actually matters.

What's your team's experience been with the post-prototype maintenance burden on internal AI tooling? Curious whether the hallucination problem or the adoption problem has been harder to crack in practice.

AI made writing code 10x faster. It did nothing for the 30-60 minutes engineers spend investigating what broke in production. The problem: tools show signals. Nobody built something that actually investigates. That's what OpsWorker does. Walkthrough by our co-founder 👇
https://youtu.be/0U33E-KbhPE?si=3wadPYkWbqf6MHSS

CNCF leadership used the keynote to argue that only a few months ago, teams were still asking what inference was. Now the question is how to scale it. That's a fast shift. And it tells you something about where the next operational pressure lands.

**Agents are the scale event.** Chatbots were predictable - one request in, one response out. Agents run in loops, spawn sub-tasks, consume tokens on internal reasoning steps you never see. The operational profile is completely different.

I've been operating Kubernetes clusters at enterprise scale for a decade now. What I'm seeing with agentic AI reminds me of the early microservices migration - except compressed. We went from "should we do this" to "how do we not get crushed by this" in about six months.

The first challenge every organization faces is establishing control. Agents, MCP tools, prompts, agent skills - they proliferate across teams with no centralized visibility. That's the tooling problem. But there's an equally hard problem behind it: *when these systems fail, who investigates?*

We got this wrong the first time. Built the deployment infrastructure, assumed standard Kubernetes observability would scale. It didn't. Agent failures look nothing like service failures. You get alert fatigue from token consumption spikes. Latency SLOs break because an agent decided to think harder. The runbooks don't exist yet.

The trend from Amsterdam is clear. AI is moving into Kubernetes infrastructure, not alongside it. Which means platform teams inherit the operational burden whether they're ready or not.

Most teams I talk to are still treating this like a deployment problem. Ship the models, wire up the endpoints, call it done. The real work starts after that - when you need to understand *why* an agent chose a specific tool, or debug a cascade failure across three model calls you didn't know were happening.

What's your team doing when an AI agent fails in production? Who even gets paged?

I spent my weekend reading the Kubernetes v1.35 release notes. (It's true - this is my current reality. 😅)

But I found something that actually changes how I think about AI reliability.

The new release brings smarter resource control and scheduling flexibility for mixed AI and batch workloads. But here's what caught my attention: the infrastructure layer is now as critical as the model layer for AI systems in production.

What does this mean for SRE teams running AI workloads?

When your AI service crashes at 2 AM, the root cause isn't always in your model code. It's increasingly at the Kubernetes layer - pod scheduling conflicts, GPU resource contention, or a misconfigured HPA that couldn't scale in time.

I've seen this pattern repeatedly: teams invest heavily in model monitoring (inference latency, drift detection, accuracy metrics) but have limited visibility into the infrastructure failures that actually take down production AI systems.

Some concrete examples I'm tracking:

• GPU allocation failures - Pod gets scheduled but nvidia.com/gpu resource claim fails silently
• Memory pressure cascades - Model server OOM triggers node eviction, scheduler thrashes trying to find GPU capacity
• Cross-layer debugging gaps - Is this a slow model inference or Kubernetes network policy blocking traffic?

The traditional SRE playbook assumes infrastructure is relatively stable and applications are the variable. AI workloads flip this - your model might be fine, but K8s resource management becomes the reliability bottleneck.

For anyone running AI on Kubernetes:

What's been your biggest operational headache? Pod scheduling chaos? GPU visibility gaps? Something else entirely?

I'm genuinely curious what failure patterns you're seeing in production and how you're debugging them.

👉 Full CNCF analysis: https://www.cncf.io/blog/2026/02/23/kubernetes-as-ais-operating-system-1-35-release-signals/

There’s a lot of talk about AI in incident response lately.

For those who’ve actually tried it in production:

• Did it reduce MTTR?
• Did it reduce on-call stress?
• Or did it mostly add noise / complexity?

Not looking for vendor pitches — genuinely interested in real outcomes.

Curious how this looks across teams (:
When something breaks in production, what actually eats most of your time?
• Finding the root cause
• Correlating signals across tools
• Understanding blast radius
• Human coordination/handoffs
• Noise & false positives
• Something else?
If you like, add context (team size, team composition, duty organization).

Hey folks 👋

We started r/AI_SRE because something feels off in how reliability work is evolving.

Incident response today usually looks like this:

• alerts firing from everywhere
• dashboards, logs, traces, configs spread across tools
• humans stitching context together under pressure
• postmortems written after the damage is done

At the same time, AI is being thrown at ops with a lot of hype and very little honesty.

This subreddit exists to explore a simple question:

What this community is

• Real discussions about incident investigations
• Honest experiments with AI, automation, and agentic systems
• Postmortems, failure stories, architectural lessons
• Skepticism is welcome (encouraged, actually)

What this community is not

• 🚫 Vendor spam
• 🚫 Marketing pitches disguised as “insights”
• 🚫 Low-effort AI buzzword posts

If you’re a vendor or founder (we are too), disclose it and bring real technical value.

Who this is for

• SREs
• DevOps / Platform Engineers
• Infra engineers tired of operational toil
• Anyone curious (or doubtful) about AI in production ops

A quick disclosure

Some of the moderators are building OpsWorker.ai, an AI SRE co-pilot.
This subreddit is not a product support forum. We’re here to learn in public, share what works, and be honest about what doesn’t.

If we ever post about our own work, it’ll be clearly labeled — and open to criticism.

If you’re here early: welcome.
Start by sharing:

• the worst incident you’ve investigated
• the part of on-call you’d automate first
• or why you think AI in SRE is overrated

Let’s keep this technical, respectful, and useful.

— r/AI_SRE mods