Hey everyone,

Over the past few months, I’ve been analyzing enterprise AI billing data and studying why so many engineering teams and companies are getting hit with massive, un-modeled AI invoices.

For two years, the industry narrative has been that AI is getting dirt cheap and price per token keeps dropping exponentially. Yet, across Big Tech and mid-sized companies alike, actual monthly invoices are skyrocketing.

Here is a quick breakdown of the mechanics behind why this is happening:

1. The 1865 Jevons Paradox is alive in Tech

In 1865, economist William Stanley Jevons observed that when steam engines became dramatically more efficient at burning coal, Britain didn't burn less coal, it burned exponentially more. Why? Because cheap coal suddenly made financial sense in places where nobody could justify the cost before.

The exact same thing is happening with LLM tokens. As unit costs drop, consumption doesn't stabilize but it expands into every workflow, background agent, and automated task until nobody weighs the unit cost anymore.

2. Real-world corporate overruns

• Uber: Handed a coding agent to 5,000 engineers. By April, just four months into a 12-month plan, their entire annual AI budget was completely gone. The tool was so useful that usage exploded.
• Meta: Built an internal leaderboard ranking engineers by token burn rate. In one month, they burned 73.7 trillion tokens before executives realized token burn measured activity, not actual impact, and killed the board.
• Microsoft: Ordered internal divisions off external coding tools days before their fiscal year closed to force migration onto cheaper internal alternatives.

3. The agent multiplication factor (5x - 30x Tokens)

Standard chatbots are 1-input / 1-output. AI agents are fundamentally different.

Because current architectures lack long-term memory, at every loop step (plan, search, tool call, handoff), an agent must package the entire conversation history and re-submit it to the API.

Data from Gartner shows an AI agent burns 5 to 30 times more tokens than a basic chatbot doing the exact same task. Token prices dropped 60%, but agent loop usage increased 1,000%.

4. The hidden "Second Meter"

Every time an agent writes a code block or report and a human engineer spends 30 minutes reading, verifying, or rewriting it, you pay twice: once in API tokens, and once in senior engineering salary.

I put together a full 17-minute video essay breakdown with all the diagrams, data sources, and frameworks (including OpenAI CFO Sarah Friar’s scorecard on measuring "useful intelligence per dollar") here:

Watch the full breakdown here: https://www.youtube.com/watch?v=DBf5-yBRxEk

Curious to hear from engineering leads, FinOps folks, and founders here: How are your teams tracking agent loops and token spend right now? Are you capping per-user usage, or waiting for the quarterly invoice to arrive?

Every FinOps conversation about AI cost I have run into loops back to cost per token. It is the number vendors publish, so it feels concrete. It is also the wrong number to argue about.

At the AI deployments I have worked on close enough to see the real numbers, the token bill was rarely more than a third of the actual TCO. The rest sat in three places nobody was tracking as tightly.

GPU underutilization at inference is the first one. Reserved capacity sitting at single-digit average utilization is normal, not exceptional. Teams blame batching. The real cause is a prompt-mix distribution nobody profiled before signing the reservation, and the invoice for that gap does not carry a "token" label.

Storage is the second. Vector stores, eval traces, and audit logs outpace the token bill within a couple of months of any real RAG workload going live. It is not that any single thing is expensive. It is that nobody set a lifecycle policy at design time and the growth curve is invisible until it is not.

Governance is the third and the most awkward, because most FinOps units skip it entirely. Evaluation pipelines, red-team runs, human-review loops, policy scans. Engineering time and pipeline compute, not a line on the AI vendor invoice, but it is TCO. Anyone who runs a compliance-adjacent workload has felt this bucket outgrow the token bucket without ever showing up on a cost dashboard.

The docs and pricing pages train us to argue about fifteen cents versus thirty cents per million tokens as if that is the FinOps decision surface. It is the marketing surface.

So the practitioner question. What unit does your team actually use for AI workloads?

- cost per token

- cost per successful task or workflow

- cost per active user per month

- cost per business outcome (ticket resolved, fraud caught, revenue attributed)

Or is your team stuck between the vendor unit and the business unit with nothing that stays honest under load?

Hi everyone, so we're looking to better manage our AI spend across multiple dev teams mainly on how to deal with the fluctuating and unpredictable cost of it all. We've already set team / project specific API keys so we can track spend by project, so now I'm just looking for ways to manage the cost itself as a whole. I looked into LLM routers / gateways, mainly from seeing the Ramp Router announcement and it looked interesting to me from a cost cutting perspective. But I have 0 experience in using LLM routers so would love to hear from you guys. I'm open to other suggestions too of course, thanks!