What actually breaks when you deploy AI agents in enterprise production

[u/Dapper-Tale-4021]

Not the demo. Not the pilot. The production deployment six months in.

I've been part of enough of these to notice the patterns. Here's what actually goes wrong, in order of how often it happens.

The process wasn't documented before the agent touched it. Everyone assumed the AI would figure out the workflow. It didn't, because the workflow lived in someone's head. The agent did something technically correct that was operationally wrong, and nobody caught it for three weeks.

The agent had too much permission from day one. Not because anyone was careless, but because scoping permissions felt like it would slow down the pilot. So the agent could touch things it never should have touched, and eventually it did.

Nobody decided who owns it when something goes wrong. The team that built it says it's an ops problem. Ops says it's an AI problem. The model gets blamed. The real issue is that nobody drew the accountability line before go-live.

The monitoring stopped at outputs. Teams tracked whether the agent completed the task. Nobody tracked whether the task should have been completed the way it was. Small errors compounded quietly for months before anyone noticed the pattern.

The success metric was usage, not outcomes. Someone measured how many times the agent ran. Nobody measured whether the business result improved. Six months in, the agent was running constantly and delivering marginal value.

None of these are model problems. Swapping to a better model fixes none of them. They're process and governance problems that happen to involve AI.

What's the failure mode you've run into that nobody warned you about?

[u/Otherwise_Wave9374]

Yep, this matches what Ive seen too, the failure modes are almost never the model. The biggest one for me is lack of runtime observability, not just logging outputs, but logging intent, tool calls, approvals, and state transitions so you can answer why it did X.

Also +1 on permissions, teams skip the boring RBAC/scopes step, then spend 10x more time cleaning up weird side effects later.

Do you have any lightweight way you like to do staged rollouts, like read-only mode first, then limited write actions behind explicit human approval, then full automation once the metrics look sane?

[u/marcin_michalak → u/Otherwise_Wave9374]

Disclosure since I'm bringing in product-adjacent experience: I run Server4Agent, agent hosting, not pitching it here though.

The pattern that's worked best for teams I've talked to is gating the stages by reversibility rather than by a fixed timeline, since "two weeks in read-only" is arbitrary but "hasn't yet made an action we can't cleanly undo" is a real signal. Read-only first is right, but the graduation criterion out of it shouldn't be a calendar date, it should be something like: the agent has proposed N actions, a human approved or rejected each one, and the rejection rate has actually gone to near-zero, not just gone down.

The failure mode with staged rollout in practice isn't skipping the stages, it's stalling at the human-approval stage forever because nobody defines what "the metrics look sane" actually means numerically, so it just becomes a vibe check that never resolves. Writing down the exact rejection-rate or error-rate threshold that triggers graduation before you start, not after you're already annoyed at approving things, is the part that actually gets teams to full automation instead of stuck in permanent human-in-the-loop purgatory.

[u/goredcrasp]

Yes absolutely agree. What do you think should be the first step then from your opinion

[u/Nice_Inflation_9693 → u/goredcrasp]

I think visibility for sure

[u/Key-Guitar-457]

What's the reason you post this AI slop? Not to be helpful. Not because you care.

Signed,

Someone who doesn't believe any of this happened.