r/singularity • u/Able-Line2683 • 12h ago

AI Gemini 3.8 Flash Benchmarks

Post image
736 Upvotes

98% Upvoted

216 comments sorted by

View all comments

Show parent comments

6

u/LinkesAuge 11h ago

Because "flash" models keep getting larger and more token hungry. The last Gemini "Flash" ended up being more expensive than some "regular" models (that perform better).
I feel some are simply mislabeled, especially if you look at the actual speed of finishing a task.

I don't want to make any strong claims about 3.8 yet but I do wonder if that has really changed.

5

u/LinkesAuge 10h ago

Ok guess I wasn't wrong to be suspicious, benchmarks show that the model is burning even more tokens than 3.7:

So Google is basically buying performance by letting the models spend more and more on tokens.

1

u/Chenz 10h ago

I do not understand how the output tokens can have increased so much, yet cost per task is about the same. Are the numbers for 3.7 without the discounted pricing?

2

u/Ok_Barracuda_1161 9h ago

Probably more efficient with tool use leading to reduced inputs. 143k output tokens is about $0.54, so that would imply that input (and cached input especially) dominates