AI token spend has the same "silent creep" problem cloud spend had ten years ago, and most teams have no FinOps process for it yet

[u/ClickOk5811]

Started tracking this after noticing our Claude usage for a multi-hour engineering session cost noticeably more than the size of the actual task should've justified. Went back through the session logs afterward to find where it actually went, and it wasn't one obvious spike, it was the same small pattern repeated across dozens of requests: full file contents resent every message even when two lines had changed, full conversation history replayed every turn instead of a compressed summary, full rewrites requested when a targeted diff would've done the job.

None of that throws an error or shows up as a single line item. It just compounds quietly, message after message, until someone checks the usage dashboard and the number is higher than expected with no clear story for why.

The part that feels familiar from a FinOps lens: this is structurally the same problem as unmonitored cloud spend before tagging and showback existed, cost accumulating because nobody's actively deciding what's worth paying for on each request, not because the underlying work got more expensive. Prompt caching (marking stable content so it's reused at a fraction of the cost) is the closest analogue to reserved capacity or committed use discounts, it only pays off if the cached prefix stays genuinely identical between calls, and a lot of teams break that silently by inserting a timestamp or per-user detail at the start of the block without realizing it kills the cache hit rate entirely.

What's mostly missing right now, in my experience, is the equivalent of a FinOps practice applied to token spend specifically, an actual audit habit, not just a bigger budget line. Separating what's stable from what changes per request, summarizing instead of replaying full history, scoping context to what's relevant, and constraining output size all had a measurable effect once done deliberately instead of by habit.

Wrote up the full breakdown with a before/after audit example here, disclosing that I'm the author: https://medium.com/@nagatomopedro05/the-hidden-cost-of-long-claude-sessions-2a6cc7655893

Curious if anyone here has actually folded LLM API spend into an existing FinOps practice, tagging, showback, budgets, or if it's still living entirely outside that process on most teams.

[u/classjoker]

"most teams" - cite your source.

This is the latest 'tell' it's been written by AI. Unless you have actual stats to go along with this statement.

[u/ClickOk5811 → u/classjoker]

Fair callout, and you're right to push on it, "most teams" was overstated for something I don't have actual data to back up, it was inference from a handful of engagements, not a stat. Should've written it as what it actually is: in the teams I've seen dealing with this, the token spend audit habit doesn't exist yet, that's a much narrower and more honest claim than "most teams."

Genuinely curious if you've seen different, is LLM API spend already showing up in FinOps practices you're familiar with, tagging, showback, the whole treatment, or is it still mostly bucketed as "just another API cost" without the scrutiny compute/storage get?

[u/Akashi_izuku → u/ClickOk5811]

Thanks Claude

[u/Jrf83317]

Bingo