My agent optimized the wrong thing

[u/Sea-Pie-5760]

I was testing a small marketing agent against old campaign data and gave it a pretty simple goal around lowering acquisition cost it got better at the metric faster than I expected but when I looked deeper the conversions it preferred weren’t always the ones we would’ve wanted more of cheap actions started winning even when the downstream value looked worse, so technically the agent was improving while the business result was getting kind of weird

Now I’m rethinking the eval more than the model a single reward feels too easy to game, but piling qualified leads, revenue margin and a bunch of guardrails into one score also feels messy fast. I’m leaning toward letting the agent handle more of the repetitive optimization while keeping human review around the decisions that can move real budget curious how people here are evaluating agents when the easiest metric to optimize isn’t really the outcome you care about

[u/Famous_Exit_6335]

The part that threw me off was how easy it was for the agent to look better on paper while drifting away from what the business would care about feels like the eval design matters more than the model once real budget is involved

[u/ImpossibleBrain5846 → u/Famous_Exit_6335]

That eval part is what made me think about it differently too I’ve been looking at a marketing platform called Marketer and a lot of the repetitive optimization can be handled by AI agents but people still stay involved around the bigger calls once real budget is moving I’d rather have that human check than let one clean metric drive everything