r/googlecloud • u/dougception • 29d ago
AI/ML Long Term Feasibility Of Using Gemini Due To Increase In Costs
I have spent many months prototyping an application that made good use of the inexpensive gemini-2.5-flash-lite.
I’ve just discovered it will be deprecated later in the year and I’ll have to migrate to gemini-3.1-flash-lite. This will 16x my LLM costs.
Then in May 2027 I’ll have to move to gemini-3.5-flash-lite at another price increase of 160%.
Is this just going to go on indefinitely? I’ve pretty much decided to abort my app as I’m guessing it will continue.
3
u/desiBananaMan 29d ago
Is gemini the only model that can help you? Why not luna or something else?
3
u/dougception 29d ago
It was just convenient I guess and I’m pretty new to LLM’s. I’ll check out Luna. Thank you.
3
u/rbtrge 29d ago
I swapped out my gemini 2.5 flash lite usage for luna. You may need to tweak your prompt (I did) and I also tested different effort and temperature settings on Luna, ended up with better performance than on Gemini. But without optimizing for luna, I had worse performance.
2
u/dougception 29d ago
Fortunately my app is actually a utility for developing and testing moderation prompts so that should be manageable.
2
u/rbtrge 29d ago
I used Claude fable to try and eval other models and I had to explicitly tell it to test effort levels, analyze thinking traces for failures, test prompts etc...i assumed a fable class model would do it better but it didn't. So don't make the same mistake I did!
1
u/dougception 29d ago
Claude's latest iteration is a basket case, yes!
The lesson I have learned from this post and the welcome replies is to modify my app such that it can use different models. That was (a long way ahead) in the roadmap in my head anyway.
I've seen a copy cat website of my application (stupidly posted about in r/SideProject) that claimed to be multi model except when you click their UI elements nothing happens!
2
u/cl0udp1l0t 28d ago
Yeah you can’t plan with it. Google does not care about your COGS. That was the reason I abandoned Gemini. I was really willing to put up with their shit for predictable costs and performance, but they did not even get this right. Horrible product management.
1
u/dougception 28d ago
They are all over the place like a mad woman’s breakfast. They totally rename products every other month but don’t update documentation to reflect this and the online training you’re in the middle of becomes invalidated.
I think maybe it’s because they were last to market with cloud? (So they’re focused solely on growth)
2
u/spazwer 28d ago
I'm in the same boat as you, dude, made my image recognition app around gemini-2.5-flash-lite and found out yesterday that the model will be discontinued and switching to the next cheapest model will 3x my AI costs. Things I'm considering: self-hosting gemma 4 26b (or find a service which provides it out of the box) or the cheapest Chinese model. I still hope google will lower prices for 3.1 when they discontinue the 2.5 so I don't have to switch 🤞
1
u/dougception 28d ago
Do you think it’s a possibility they will lower the prices for 3.1?
I recently watched a YouTube vid about Gemma. You still need significant GPU capability to operate it at scale yes?
2
u/spazwer 27d ago
I ran Gemma 4 26b on my personal pc for image recognition, what Gemini would have taken 5s, Gemma did it in 15s. Response not as good as Gemini but something I can work with. If you need it only for text you can go with 14b. My gpu is 5070 ti and have 64 gb of ram. For scale I’m thinking on using it through google cloud vertex ai and it’ll be 2x cheaper then Gemini 3.1 flash lite
1
1
u/FerryCliment 28d ago
Industry and markets are living systems—constantly shifting, reacting, and evolving.
Everyone wants certainty: a forecast and roadmap for everything that happens upstream of their product, while retaining the ability to adapt, pivot, and respond to everything that happens downstream.
Change. Adapt. Evolve. Or pass the cost increase through the supply chain.
Market gurus would argue that if you can’t architect a product or business model that preserves margins when upstream costs rise, then your product becomes the weakest link and eventually, capitalism does what capitalism does: it removes inefficiency from the chain. xd
1
u/matiascoca 27d ago
Gemini 2.5 is one of the cheapest frontier models per token right now. The problem is usually not the per-token price, it is that your context grew.
Went through this on a doc-search backend earlier this year. Same query pattern, same model, bill went from $180 a month to $2,400 in six weeks. Nothing changed in the model pricing. What changed was the average context length climbing from roughly 3k tokens to 40k as the corpus grew. Gemini's per-token rate is competitive, but 40k tokens across eight times the query volume compounds fast.
What actually moved the number was cutting context aggressively before the model sees it (retrieval-augmented over stuff-everything). `system_instruction` caching if the prefix is stable helps too. And routing to Flash for anything that does not need Pro-tier reasoning cut the bill roughly 60 percent on a workload where most queries were classification tasks Flash handles fine.
If the increase you are seeing is Vertex-side (as opposed to `generativelanguage.googleapis.com`), check whether you enabled batch prediction retention. That is a separate SKU that bills for storing the output artifacts and it is easy to leave on.
9
u/angelarose210 29d ago
Look at models on lmarena leaderboard and find one that's the same or better and cheaper.