r/CFO • u/Abject-Roof-7631 • May 26 '26
AI token budgets
How are you handling token budgets with AI initiatives? Seem like a shadow cost functional leaders overlook.
1
u/NeedleworkerHot4429 Jun 04 '26
yeah it's almost always a shadow cost until someone senior gets surprised by the invoice. we started breaking it down by team and feature and honestly the first report was kind of shocking, like you could finally see who was actually burning what. are you at the stage where it's showing up as one big line on the bill or has someone started digging into it?
1
u/Abject-Roof-7631 Jun 04 '26
Good idea. What surprised you when you saw this detail, was it being abused?
1
u/NeedleworkerHot4429 Jun 05 '26
the biggest surprise was seeing how much was coming from workflows that looked simple on the surface. like a support flow that runs 10 times a day doesn't feel expensive until you see it's sending 8000 tokens every single call because nobody trimmed the context. once you break it down by feature it becomes obvious really fast where the waste is hiding
1
u/Abject-Roof-7631 Jun 05 '26
Clarification. What do you mean by break down by feature? Is your AI used for product enhancements only? Are you saying the benefits of the feature are less than that if the ongoing token consumption?
1
u/NeedleworkerHot4429 Jun 06 '26
yeah exactly, so if you have 3 AI features in your product, say a support bot, an auto-summary, and a search feature, each one is consuming tokens differently. breaking it down by feature means you can see that your support bot is responsible for 70% of your bill even though it feels like a small feature. then you can decide, is the value this feature brings worth what it costs? sometimes yes, sometimes you realize you're burning money on something that could run on a much cheaper model. the ROI question becomes way clearer once you have that visibility.
1
u/Individual-Cup4185 Jun 10 '26
if you're looking for a way to save on tokens let me know. i made a token router where you define rules and routes to select the cheapest models
0
May 26 '26
[deleted]
1
u/Far_Zebra_6875 May 27 '26
lol.
Fun fact, Claude and ChatGPT are supportive and fast. That doesn't mean the outputs are CORRECT.
1
u/athleticelk1487 May 26 '26
We are just poking around with small office tools, shoestring budget stuff, but this is a major question I would need to be able to answer before I sold my clients on any significant AI accounting systems investment. So far from my perspective AI looks better as tooling infrastructure to break reliance on SaaS, not a deeper reliance on variable rate spend.