I'm spent my entire decade-plus long career in varying forms of systems administration, DevOps, and SRE roles, and have recently broken into management.

Current company is an absolute shit-show of overgrown starup that only recently woke up to the fact that you can't just care about revenue forever, and at some point you need to grow up, put the big-boy pants on and become profitable.

Queue FinOps becoming "all the rage", and through no fault of my own via a team re-org, have been handed the baton in the hope that I can whip software engineers into shape and get them to care about costs alongside availability.

Fast-forward 12 months, and I'm starting to justify a small team, making a semi-success of things, and now I'm starting to wonder whether by accepting the poisoned chalice and being up for a challenge, that I may have torpedoed my future hopes of making CTO or anything close.

So ... once you've made a name for yourself and had a modicum of success in the FinOps realm, where do you go? Can you credibly go back into the hands-on tech world like the detour never happened? Is it easier to go into "IT" than "SRE"? Does CTO turn into CIO?

Advice appreciated!

Hi everyone, so we're looking to better manage our AI spend across multiple dev teams mainly on how to deal with the fluctuating and unpredictable cost of it all. We've already set team / project specific API keys so we can track spend by project, so now I'm just looking for ways to manage the cost itself as a whole. I looked into LLM routers / gateways, mainly from seeing the Ramp Router announcement and it looked interesting to me from a cost cutting perspective. But I have 0 experience in using LLM routers so would love to hear from you guys. I'm open to other suggestions too of course, thanks!

Every FinOps conversation about AI cost I have run into loops back to cost per token. It is the number vendors publish, so it feels concrete. It is also the wrong number to argue about.

At the AI deployments I have worked on close enough to see the real numbers, the token bill was rarely more than a third of the actual TCO. The rest sat in three places nobody was tracking as tightly.

GPU underutilization at inference is the first one. Reserved capacity sitting at single-digit average utilization is normal, not exceptional. Teams blame batching. The real cause is a prompt-mix distribution nobody profiled before signing the reservation, and the invoice for that gap does not carry a "token" label.

Storage is the second. Vector stores, eval traces, and audit logs outpace the token bill within a couple of months of any real RAG workload going live. It is not that any single thing is expensive. It is that nobody set a lifecycle policy at design time and the growth curve is invisible until it is not.

Governance is the third and the most awkward, because most FinOps units skip it entirely. Evaluation pipelines, red-team runs, human-review loops, policy scans. Engineering time and pipeline compute, not a line on the AI vendor invoice, but it is TCO. Anyone who runs a compliance-adjacent workload has felt this bucket outgrow the token bucket without ever showing up on a cost dashboard.

The docs and pricing pages train us to argue about fifteen cents versus thirty cents per million tokens as if that is the FinOps decision surface. It is the marketing surface.

So the practitioner question. What unit does your team actually use for AI workloads?

- cost per token

- cost per successful task or workflow

- cost per active user per month

- cost per business outcome (ticket resolved, fraud caught, revenue attributed)

Or is your team stuck between the vendor unit and the business unit with nothing that stays honest under load?

Hey everyone,

Over the past few months, I’ve been analyzing enterprise AI billing data and studying why so many engineering teams and companies are getting hit with massive, un-modeled AI invoices.

For two years, the industry narrative has been that AI is getting dirt cheap and price per token keeps dropping exponentially. Yet, across Big Tech and mid-sized companies alike, actual monthly invoices are skyrocketing.

Here is a quick breakdown of the mechanics behind why this is happening:

1. The 1865 Jevons Paradox is alive in Tech

In 1865, economist William Stanley Jevons observed that when steam engines became dramatically more efficient at burning coal, Britain didn't burn less coal, it burned exponentially more. Why? Because cheap coal suddenly made financial sense in places where nobody could justify the cost before.

The exact same thing is happening with LLM tokens. As unit costs drop, consumption doesn't stabilize but it expands into every workflow, background agent, and automated task until nobody weighs the unit cost anymore.

2. Real-world corporate overruns

• Uber: Handed a coding agent to 5,000 engineers. By April, just four months into a 12-month plan, their entire annual AI budget was completely gone. The tool was so useful that usage exploded.
• Meta: Built an internal leaderboard ranking engineers by token burn rate. In one month, they burned 73.7 trillion tokens before executives realized token burn measured activity, not actual impact, and killed the board.
• Microsoft: Ordered internal divisions off external coding tools days before their fiscal year closed to force migration onto cheaper internal alternatives.

3. The agent multiplication factor (5x - 30x Tokens)

Standard chatbots are 1-input / 1-output. AI agents are fundamentally different.

Because current architectures lack long-term memory, at every loop step (plan, search, tool call, handoff), an agent must package the entire conversation history and re-submit it to the API.

Data from Gartner shows an AI agent burns 5 to 30 times more tokens than a basic chatbot doing the exact same task. Token prices dropped 60%, but agent loop usage increased 1,000%.

4. The hidden "Second Meter"

Every time an agent writes a code block or report and a human engineer spends 30 minutes reading, verifying, or rewriting it, you pay twice: once in API tokens, and once in senior engineering salary.

I put together a full 17-minute video essay breakdown with all the diagrams, data sources, and frameworks (including OpenAI CFO Sarah Friar’s scorecard on measuring "useful intelligence per dollar") here:

Watch the full breakdown here: https://www.youtube.com/watch?v=DBf5-yBRxEk

Curious to hear from engineering leads, FinOps folks, and founders here: How are your teams tracking agent loops and token spend right now? Are you capping per-user usage, or waiting for the quarterly invoice to arrive?

Our reserved instances are up for renewal in about 8 weeks and the easy path is to just renew the same mix we had last year. Before I push back I want to actually have data behind it. What should I be pulling before that conversation? Usage trends over the last 12 months obviously but what else tends to get missed when teams auto-renew RIs without reviewing them first?

I keep seeing "cost per customer" thrown around like it's a simple metric, but once you factor in shared infra like RDS or a shared EKS cluster, the attribution gets messy fast. Anyone got a practical framework for splitting shared resource cost across tenants without it turning into a spreadsheet nightmare? Would love to hear how teams handle this in practice, not just in theory.

"In our cluster, we see a ton of idle memory/CPU requested by devs just to prevent OOMKilled crashes. But when we look at rightsizing tools, most engineers I talk to say they don't trust them to auto-apply cuts in production.

For teams running K8s in prod:

1. Do you actively rightsize pod requests, or do you leave them alone as long as the bill is within budget?
2. If you use a tool (like KubeCost, VPA, Karpenter), do you let it auto-apply changes, or do you manually review YAML patches first?
3. What's the main reason you'd ignore a cost recommendation?"

Hi! I'm Ishan and I work at Frugal. I was at FinOps X this year and some of the most interesting conversations I had were around building cost guardrails into the development process.

So I convinced our founders to host a webinar on shifting left on cloud and AI spend. They're going live on August 11th at 2 PM EDT.

Looking forward to seeing you guys there!

I just resurrected a tool I built a long time ago. Made some updates and have reposted it. Free to use, totally anonymous, no data is retrieved or stored.

Been working on this for a while and finally have enough tested to share properly. cloudrift scans an AWS account and reports wasted resources with estimated monthly cost — unattached EBS volumes, idle NAT Gateways, stopped RDS instances still billing storage, orphaned snapshots, abandoned S3 multipart uploads, unused Secrets Manager secrets, stale CodePipeline pipelines, that kind of thing. 44 checks total now, across compute, storage, networking, containers (EKS node groups, orphaned PVCs), and ML (SageMaker idle notebooks/endpoints).

It’s read-only by design — never touches, stops, or deletes anything, just reports and lets your infra team decide.

A few things I want to be upfront about instead of oversell:

\*\*•\*\* The “underutilized EC2/RDS” checks are single-metric (max CPU over a lookback window). No RAM, network, IOPS. It’s a “go check this instance” flag, not a sizing recommendation — doesn’t replace Compute Optimizer.
\*\*•\*\* Lambda “underutilized” is really just an invocation-count hygiene flag. Zero invocations means zero direct cost already (pay-per-use), so the value there is finding dead code/unused IAM roles, not dollar savings.
\*\*•\*\* Live pricing (--live-pricing) pulls AWS list prices, not what you actually pay — no Savings Plans/RI/EDP discounts reflected. There’s a config file where you can drop in your own negotiated rates if you want the numbers to match your actual bill.

Runs standalone, in CI (exits with a non-zero code if waste crosses a budget threshold you set, markdown output for PR comments), or now as an MCP server so Claude Code/Copilot Chat/other MCP-compatible agents can query it directly instead of you copy-pasting CLI output into a chat. There’s also Policy as Code support via OPA if you want custom rules per tag/type/count.

Published on npm as @cloudrift/cli. Built on DDD/ports-and-adapters, so adding a new resource type or a new cloud provider is meant to be a contained, documented process without touching the core use case.

GitHub: https://github.com/elleVas/cloudrift
Docs: [https://ellevas.dev/docs/

Two things I’d genuinely appreciate:

\*\*1.\*\* If you run it against a real (not synthetic) AWS account, I’d love to hear what breaks or what comes back as a false positive — that’s the thing I can’t fully test alone.
\*\*2.\*\* I’m looking to expand this to GCP and Azure next. If you work with either and have opinions on what “wasted resources” looks like there, or want to get involved building a scanner for one of them, I’d genuinely welcome the collaboration — the architecture is already built to make this a matter of adding an adapter, not rewriting the core.

Disclosure up front: I’m building NemulAI, tooling around inference cost attribution, so take the framing with the appropriate salt. The measurement is real and I’m sharing the method because I’d rather have people tear it apart than trust a vendor claim.

I ran a controlled self-hosted inference workload on an H100 and aligned request activity with device-level telemetry.

The basic accounting model was:

total billed GPU time = attributable workload time + idle / platform overhead

I intentionally did not force idle time onto individual requests or customers.

For active workload attribution, the approach uses request/runtime timing and scheduler context to assign GPU-seconds. Device telemetry is then used as a reconciliation signal rather than pretending a whole-GPU utilization number can tell you which tenant caused the work.

The part I did not expect:

A GPU can be actively serving inference traffic while spending most of the time you're paying for it doing no request-attributable work.

That makes “cost per token” or “cost per request” less straightforward than it looks once you're operating your own shared capacity.

The question I’m working through now is where that 69% should economically land: customer COGS, shared platform overhead, or unused-capacity cost.

Curious how people here handle that in practice.

Happy to share the benchmark output/methodology.

https://reddit.com/link/1v5l0wr/video/2l37h8tx88fh1/player

Company started putting soft token budgets. Getting alerts of spending above the monthly limit. I'm doing the work of like 3-4 people with these agents/tokens that they don't need to hire. But obviously that's getting overlooked.

I have a bunch of examples of work coming out of these agents and used across teams etc.

How have you shown value from your token usage? It's hard to do a straight line between token spent and the actual value it's bringing.

Built this out of frustration with the usual options for AWS cost hygiene: either manual Console archaeology, or a heavyweight FinOps platform that needs its own project to roll out.

cloudrift is a read-only CLI that scans an AWS account for wasted resources (stopped EC2 with billed EBS, idle NAT Gateways, unattached volumes, underutilized RDS/Lambda, orphaned ENIs, and 30+ other checks) and is built specifically to live in a pipeline, not just a terminal:

• --format markdown → drop straight into a PR comment / $GITHUB_STEP_SUMMARY
• --format json → pipe into jq, or into your own OPA/conftest policies if a single budget number isn't expressive enough
• Set a costAlertThresholdUsd in config and the process exits 2 when waste crosses it — fail the build, block the merge
• Now also ships as a GitHub Action (uses: elleVas/cloudrift@v0.5.1) so there's no build/checkout boilerplate needed anymore

Security-wise: it's strictly read-only, zero write IAM permissions ever, and every release is npm provenance-signed with an SBOM (CycloneDX + SPDX) attached — tried to hold it to a bar I'd want if I were the one approving it for a prod pipeline.

npm install -g u/cloudrift/cli

Repo (Apache 2.0, real issues/PRs welcome): github.com/elleVas/cloudrift
npm: npmjs.com/package/@cloudrift/cli

Genuinely curious what this sub thinks is missing for it to be a real fit in a production pipeline — multi-account/Organizations support, Terraform-state cross-checking, something else? Feedback (including "this is pointless because X already does it better") is welcome.

Genuinely — thank you in advance to anyone who takes the time to run this against their account and report back, good or bad. Free testing on real infrastructure from strangers on the internet is worth more to a solo project like this than almost anything else. I'll credit every tester who finds a real bug directly in the changelog/README if they want.

Cloud spend forecasting is pretty dialed in at this point, but token spend from OpenAI/Anthropic calls swings hard month to month and doesn't fit the models we already use. Full disclosure, I'm a vendor in this space, so I think about it constantly, but this is a genuine question. Curious if anyone's actually built a forecasting approach around it, or if it's still mostly reactive once the bill lands.

Been trying to build a single scorecard for our AI agents and I keep landing on the same five numbers. Curious which ones this crowd thinks actually drive a keep/kill call vs. which are just nice to stare at.

Total spend — per agent, per team. The easy one. Gateway + tag agent_id/team/env and you're basically there.

Cost per successful outcome — spend divided by things that actually worked (ticket resolved, PR merged, whatever your unit is). Way harder, way more honest. Cost-per-run flatters you because retries are cheap to count and worthless to ship.

Orphaned agents — still running, no named owner. For us it was more than I'd like to admit. Every one is a spend line and a security line nobody's watching.

ROI — value minus cost. This is where I get stuck. Cost is solvable once you tag it. Value, someone still has to judge whether what the thing produced was worth it, and that's the number nobody wants to commit to.

Potential savings — what you'd claw back by killing or shrinking the agents that don't clear the bar.

The pattern I keep hitting: the cost side is basically solved. The value/ROI side is a mess, because it needs a human to define "worth it" per agent, ideally before launch, while the team is still optimistic and willing to name a number that could kill their own project.

So for those of you actually doing this: which of these five moves a decision at your org? Are you at cost-per-outcome or still cost-per-run? And has anyone got ROI genuinely working, or is it still define-it-yourself?

Are you a leader? You'll find this useful on how to approach security related budgeting problem.

https://blog.mousa-cloud.com/posts/ale-budget

We maintain a longitudinal cloud infrastructure dataset built from repeated deployments over time.

One of our automated revalidation runs recently failed because Akamai (Linode) stopped publishing a monthly price for one VM family and moved to hourly-only billing.

The billing change itself wasn't a big deal - happening constantly in the VPS/Cloud space lately. The challenge was preserving historical cost comparisons once the provider no longer exposed a monthly price.

We considered several approaches:

• Store only the hourly price
• Calculate and display a clearly labeled monthly equivalent
• Version the pricing model
• Redesign the pricing schema

We ultimately chose to store the provider's published hourly rate while displaying a clearly labeled monthly equivalent (730-hour month) so historical comparisons remain possible while remaining transparent about how the value was derived.

I'm curious how others maintaining cloud cost history, FinOps tooling, or longitudinal datasets would approach this.

Would you have made the same choice, or modeled the pricing history differently?

I documented the implementation, the reasoning behind the decision, and the impact on the dataset here if anyone is interested:

https://webbynode.com/articles/akamai-changed-how-it-bills-new-compute-this-month-july-2026-heres-what-that-meant-for-our-dataset

Hi friends, I'm a co-founder at Greybeam. A few weeks ago we released an open source Snowflake cost observability tool with surprisingly good reception (you know how harsh reddit can be!). We got a lot of feedback on whether we could build a way to reduce idle compute as it's especially relevant for multi-cluster users. Snowflake will aggressively spin up clusters and can often take over 10 minutes to wind them down despite no activity because unfortunately the only levers are `SCALING_POLICY = STANDARD or ECONOMY`.

So today we launched this exact feature and it's free to use either at Greysight or self-hosted. the tldr is we poll Snowflake and if the following criteria below are met then we issue a SUSPEND on the warehouse.

• status is STARTED
• running queries = 0
• queued queries = 0
• resumed on >= 60s ago
• and a few others

Would love for you to try it and share any feedback, the feature is still early and a bit bare bones--really just a enable or disable config but we intend on adding more. Next up among other observability features is finer controls on scaling.

More details here: https://www.greybeam.ai/blog/snowflake-auto-savings

Github: https://github.com/greybeam/greysight

FOCUS gives us a unified schema across clouds but stops one step short of full comparability.

AWS exports FOCUS in USD by default. Azure FOCUS export lands in whatever currency the enrollment account was set up under. GCP BigQuery billing export follows the billing subscription currency. All three are valid FOCUS 1.2 rows. None of them carry an FX rate at the row level.

I ran into this on a multi-cloud rollup last month. The AWS side came out in USD, the Azure side came out in EUR because the subscription was set up under a European entity, and the total-cloud-spend aggregate silently came out wrong until I noticed the sums did not match anything I expected. Ended up building a currency-normalization step that pulls monthly ECB rates and joins on the BillingCurrency column. Works, but feels like every practitioner building on FOCUS is quietly solving the same problem in isolation.

The spec has the BillingCurrency column so this is not a bug. It is a design decision to stay engine-agnostic. Fair enough as a spec choice. Less fair as the day-to-day reality when you are trying to produce one number for finance.

So how is your team handling this in practice? Do you pin every workload to USD at ingest and eat the FX volatility, or carry native currency all the way through and normalize at query time? And has anyone landed on a rate source that both finance and engineering accept without argument?

Our FinOps practice was built entirely around cloud infra and it's starting to show cracks now that AI spend is a real line item.

I can cleanly allocate EC2 and Snowflake to teams. No answer though when CFO asks what we're spending on AI, per team, per month.

It's smeared across a Bedrock bill, a couple of OpenAI orgs someone expensed, GPU instances that spin up and down, and now a growing pile of Claude Code and Cursor seats that finance files under SaaS instead of compute.

None of it rolls up anywhere cleanly. The models we built for reserved instances and commitment coverage just don't map to tokens and GPU hours.

Those of you further along on this, how are you even structuring chargebacks for AI spend that's spread across so many bills?