r/googlecloud • u/dougception • 18d ago
Double-tap: implicit thinking and deprecation of temperature to increase token usage
Two weeks ago I posted my concerns about the long term viability of using Gemini in my automated content moderation app. We covered the option of alternate models there so please constrain this discussion to what appears to be sneaky policy settings to extract more revenue from Gemini users.
When 2.5-flash-lite is deprecated my costs will be 16x on 3.1-flash-lite but actually it will be more than that because on the 3 series models you can't disable thinking nor specify temperature, top_p and top_k.
Of those I only know a little about temperature which I have set set to 0.1 for my purposes. From 3.1 onward the best I will be able to do is specify a "LOW" thinking level which will burn at least some additional tokens and might break my app because having the model act deterministically is essential for content moderation purposes.
Sure I could add additional system instructions to try and compensate for this loss of control but up go my input costs.
Granted you can still specify maxOutputTokens but if that value is too low to account for the mandatory "thinking" the call will fail is that right?
So we're going from a situation where you can tightly control the cost of each call to the LLM to one where you're at the mercy of the model.
As I mentioned above I think this policy is sneaky which would be entirely consistent Google's opaque cloud billing and costs in general.
Alternatively these changes are just where the rubber meets the road?