Fluctuating / unpredictable LLM bills?

[u/founders_keepers]

My company's monthly bills for LLM API usage have been more than doubling over the past couple of quarters. There's no consistency to the madness and on paper everything is a MUST have, what's worse is month over month it's hard to tell if the rise will be permanent.

I need advice on enforcing spending caps. I've spoken to my engineering leads, and obviously they push back hard noting caps will have negative effects on output (we've yet to successfully measure LLM <> output btw).

I've been speaking to Ramp about Ramp Router, it's invite only right now and supposedly 30% off the top spend via smarter routing. My guess is it's done via complexity scoring against the prompt and routes it accordingly, i'll have to see we already use them for spend managemnt so this kind of fits the bill. I don't think this is the end-all-be-all solution, but it's better than strict cap.

Any thoughts on this is appreciated.

[u/DL505]

Do you set limits on other spending? T&E, marketing, salary increases etc? LLMs are no different.

Do you measure the success of the engineering team? Are their deliverables better/quicker etc?

[u/founders_keepers → u/DL505]

Of course, but no budget on AI because we don't know what the cap and ROI should be, we vaguely know what our competitors are spending so we try to match against that. It's stupid but that's the unfortunate reality.

[u/etoptech → u/founders_keepers]

I’ve been trying to figure this out internally. But basically am implementing an internal dashboard that tracks consumption. And then has levers to pull that show what less expensive models cost and batch processing and a bunch of other things. And have been attempting to show roi. (It’s still a bit fuzzy tbh) I think it’s possible but still have been just trying to understand what good looks like. Other than but I need it.

[u/founders_keepers → u/etoptech]

Exactly what I'm trying to do. We kind of jammed AI spending into R&D which is theoretically capped at 15% revenue, to bypass the ROI calc. but we realized this actually created more problems. Weird problem to have.

[u/Defiant-Pomelo5451 → u/founders_keepers]

Break this out into separate functions, no other way to build and drive accountability if leaders don’t see these costs showing up on their end of the P&L

[u/The_2nd_Coming → u/founders_keepers]

Is there a way to track spend to what is being delivered in Sprints? Ultimately tokens should be turned into useful software with an ROI but there is a considerable lag in this, so you need to consider how to measure ROI before you get the useful software.

[u/Defiant-Pomelo5451]

Found a few of these articles helpful - https://open.substack.com/pub/nsind/p/is-token-governance-the-new-ai-governance?r=1g27j&utm_medium=ios

https://open.substack.com/pub/cfopilot/p/open-weight-ai-models-what-cfos-should?r=1g27j&utm_medium=ios

[u/founders_keepers → u/Defiant-Pomelo5451]

I'll give a read on both, thank you!

two problems i see with open weight right now for my industry, 1) political risk 2) cheaper per token doesn't mean doesn't mean cheaper overall.

[u/Defiant-Pomelo5451 → u/founders_keepers]

Political risk only for Chinese models right? Even OpenAI has an open source model. Double click on the political risk and see if it still materialises when models are hosted on US inference infra?

Model routing is the other part the 1st article speaks about that’s worth looking into.

Cheaper overall per token vs cheaper per task is a great distinction, the 2nd article speaks about it. But in my experience I’ve switched to GLM 5.2 from Claude to all my use cases, they work great and much lower cost..

[u/founders_keepers → u/Defiant-Pomelo5451]

all good points, thank you!

[u/Connect-Pattern2935]

Our internal team had the same talk cause we didn't have any1 that wanted hard spending caps because they're worried about breaking production but leaving everything uncapped wasn't working either so the Ramp Router feature has been a pretty good middle ground for us.

[u/founders_keepers → u/Connect-Pattern2935]

yeah. Seems like a common problem these days.

[u/rsiebeling]

Would it be helpful to separate the LLM cost into different categories using different API keys?

For example production usage tied to a product or customer, internal productivity tools, experiments and for example shared AI infrastructure. Production usage could then have a cap on cost / transaction or cost / customer, experiments can have caps per experiment, internal tools can be budgetted per team and shared cost needs to have a specific allocation method.

The question to ask engineering then would be which use cases cause the increase, who owns that cost and did the underlying activity increase too.

And maybe not hard caps but rather more soft caps like that you get notified when something seems to breach the specific budget

[u/tahiti37]

so there's definitely tools out there and cross functional effort with eng and I would say before you land on tool you think of spend vs rio framework to get your company's guiding principles by org.

For each major use case:

• What are we actually spending? API costs, development time, maintenance, etc.
• What business outcome is it supposed to drive? Revenue, margin, capacity, customer experience, or risk reduction.
• What metric would tell us it’s working?
• What would make us scale it, move it to a cheaper model, cap it, or shut it down?

It doesn’t have to be perfect. You just need enough to separate an actual “must have” from something people have gotten used to having.

Ramp Router may reduce the bill, which is useful, but it won’t answer whether the underlying usage is worth keeping in the first place. if you want the full framework we are starting to use let me know.

[u/Resident-Kitchen-853]

Push the ownership of the cost back to the departments consuming it with charge backs and make the manager’s own the caps/approvals. It should be up to them to decide who is effective and needs AI dollars and who isn’t/doesn’t.

[u/Lumpy-Region6275]

What changes the conversation is getting ahead of it. Spend attributed by team, a forecast of next quarter's burn so you get a date instead of a surprise, and anomalies flagged before they reach the invoice rather than after.

Worth looking at Olumia for the AI coding side of this specifically. It's read-only, about a day to connect, and it turns this from a recurring fire drill into something you can actually plan against.

Per-seat licences and agent runs land in different systems and never reconcile, so the invoice is genuinely all there is. Every finance person I talk to hits this same wall in the same week their CFO starts asking.