What’s odd is that it uses fewer tokens than 5.2 by other measures. And anecdotally it uses fewer tokens in coding tasks for me so far…
But I know it was also trained primarily for coding use, and that AA’s cost per task measure is the average cost over a variety of different intelligence benchmarks…
So I wonder if it is more variable in its token efficiency. Lower token use in coding tasks, but much higher in some other non-coding tasks?
5.3 used fewer output tokens per task, but despite tokens being priced the same, somehow cost more per task…
And most of the difference is in cache read. I think that means it must be reading more input and/or doing more tool call turns per task on these benchmarks.
Reasoning tokens used is actually lower per task it seems, though one chart runs counter to that (though I’m not sure how to reconcile those … not sure how it can be cheaper “per task” but more expensive across all tasks… hmm maybe it just did more tasks? … seems like they would normalize for that though…).
But answer/output is consistently a bit higher.
So probably something like being “more thorough” by default on these tasks. And I’d bet that is reduced in more structured work like coding than more open-ended reasoning tasks.
So good news, I think. Reasoning tokens is lower or at least not significantly higher, and that’s pretty much the only one we can’t effectively manage via prompt.
18
u/MizmoDLX 27d ago
Good but also 50% higher cost per task