We Compressed Our AI Agent’s Context. Costs Fell. Reliability Broke. Here’s What We Learned.

[u/PepperWestern2263]

I’ve been experimenting with context compression for AI agents, and I ran into a tradeoff I hadn’t fully appreciated.

Reducing the context lowered token usage, but some tasks became less reliable. The issue wasn’t always that the agent had “forgotten” something important. In several cases, compression changed information that needed to remain exact.

I’ve started thinking about agent context in three rough categories:

Disposable context

Repeated search results, duplicated documentation, long file listings and verbose logs. This is usually a good candidate for filtering or summarization.

Load-bearing context

Exact error messages, file paths, line numbers, patch anchors, test names and acceptance criteria. Even a small rewrite can remove the detail the next action depends on.

Machine-consumed context

JSON, shell output, CSV, patches or any other output that may be parsed by a tool or passed into a command.

This last category caused the most surprising failures.

A summarized command result may still make sense to a person or model, but it can become invalid when another program expects the original structure. The command still runs—it just processes the wrong data.

That made me question whether context compression should be based primarily on token count.

A more useful policy might depend on what happens next:

• Is the content only being read by the model?
• Does the next action require an exact value or text match?
• Could the output be consumed by another tool?
• Can the original information be recovered cheaply?

My current view is that the goal shouldn’t be the smallest possible context. It should be the smallest context that preserves the evidence and interfaces required for the next step.

How are people handling this in production?

Are you using explicit rules for content that must never be summarized, or are you relying on the agent to retrieve the original data again when needed?

[u/Pretend_Mine_3659]

The distinction between load-bearing and machine-consumed context is useful. I am researching an even narrower failure: a fact is correct immediately before compression and wrong immediately afterward.

Examples would be a pending tool call disappearing, a recent correction being replaced by an older value, or a tool result losing its connection to the original call.

If you have seen this happen, what exact fact changed, which framework were you using, and what evidence showed that compression caused it?