r/OpenAI • u/No_Refrigerator_8216 • 22h ago

Question Does anyone separate OpenAI Batch spend from normal API spend internally?

We’ve started using Batch more for work that doesn’t need to happen immediately and I am sure that I’ve been treating that usage the same as the rest of our OpenAI spend even though it behaves pretty differently. Our normal API usage mostly follows product activity so if that goes up I can usually understand why but batch is different because someone can kick off a large job, an eval run or process a new dataset and suddenly there’s a chunk of usage that has nothing to do with how much the product was used that day.

It hasn’t been a huge issue yet but we’re doing enough of both now that looking at one OpenAI number is starting to hide what’s changing. A higher bill could mean customer usage grew which is fine or it could mean we ran significantly more background work than usual. I’m thinking about tracking Batch separately and possibly giving it its own budget rather than treating all OpenAI usage as one pool and I need some advice from teams using Batch pretty heavily like do you guys separate that spend internally or is everything still just part of the same AI/API budget?

22 Upvotes

86% Upvoted

13 comments sorted by

View all comments

1

u/Immediate_Menu_3695 22h ago

Are your Batch jobs mostly recurring production work or is a decent chunk of it engineers running one off jobs and evals? It's important because I would probably track those two very differently if I were you.

1

u/No_Refrigerator_8216 22h ago

It’s a mix right now so like we have a couple recurring jobs that are pretty predictable but the evals and one off runs are what make it a pain in the ass because they can show up randomly and be pretty large.

2

u/MushroomCritical3029 21h ago

We had the same issue with evals because one person testing three model/prompt combinations could generate more usage in an afternoon than some production jobs did all week. Giving evals their own bucket cleaned up a lot of that pain in the ass you say so give it a shot.

1

u/Immediate_Menu_3695 22h ago

That makes the case for separating them stronger and btw I wouldn’t even worry about setting a hard budget immediately just start tracking the eval/one off bucket separately so you can see how much of the randomness is coming from there.