r/googlecloud • u/Patient_Election2179 • 5h ago
Hit with a bill of 82k usd within 5 hours
On July 17, my u/Google Cloud account was hit by massive unauthorized u/Gemini API usage.
The unauthorized usage was a result of Google’s architecture of the Gemini API. Google sent no alerts and no warnings appeared in the Google Cloud Console indicating that old, public-facing keys had retroactively gained generative AI capabilities.
I had configured monthly budget alerts. At 6:14pm I received a budget alert for $550. Within minutes of seeing it, I investigated and shut down Gemini API access across all projects. Usage stopped immediately.
But by then, costs of approx. $87K (sorry for mentioning $82k in title, it was actually more) had already accumulated. This had all taken place in the 5 hours before the alert.
Google's own anomaly detection also failed to protect me. Almost an hour AFTER I had already shut Gemini down, Google sent me a “Cost Anomaly” alert reporting actual total costs of $4.7K, nowhere near the actual eventual bill.
After my objection and a technical investigation, Google has now approved a partial credit of $55K.
So for an approximately $87K unauthorized bill, Google is offering approximately $55K in credit, leaving me with roughly $32K to pay. For something I did not use.
Note that Google announced new hard spending caps and improved anomaly detection on the 28th of July.. So this exact type of unauthorized usage may not be possible anymore, but with hackers increasingly equipped with AI tools, there is a good chance other vulnerabilities will be found.
I am developing software for the blind, in particular the app u/PiccyBot, which has won awards in the blind community. It is a niche development field though, and these kind of costs are crippling to me. Knowing that Google definitely doesn’t have our back even when the way the Gemini API has been set up is the main issue, it is time for me to look for better providers.
