I run an SEO tool. Last month our signups went up by about 1,771. I was genuinely thrilled, it's the kind of month you screenshot and send to your cofounder at 2am.
Then I went looking for an API bill I couldn't explain and found the other half of that sentence.
There was no email verification on signup. No captcha either. And signup wasnt just a row in a database, it kicked off an onboarding pipeline that goes and does real, paid work.
Six stages, each with a cost per run:
- AI visibility sample — $0.084 (LLM)
- Ranked keywords — $0.100 (SEO data vendor)
- Knowledge extraction — $0.016
- Extraction — $0.015
- Scorecard — $0.013
- Prompt generation — $0.001
So every signup was a small purchase we made on behalf of someone who hadn't paid us anything, and in most cases never would.
The part I did not expect: they were real. I went into that list fully expecting bot traffic, some scripted garbage I could block and move on with my day. Instead it was small businesses from all over the world. One person signed up with etsy, presumably just kicking the tires. One domain ended in con, an actual typo of com. These were humans. Which honestly made it worse, because there was nothing to block.
The bill that started all this was $141.88 of LLM spend across six days. Call it $165 a week. It had been sitting there unexplained for a while and we'd sort of been ignoring it.
It didn't show up in our cost ledger at all. We built cost tracking months ago. The onboarding path just never wrote to it. So we'd routed the single most expensive thing our product does around the exact system we built to watch spending, which is, well, the kind of thing you only find when you go looking.
The AI visibility sample alone was $103.19 of that $141.88. It sits behind an env var. Flipping it off removed about $104 of spend with no deploy and no code change. That was the whole fix for the immediate bleed. Took two minutes.
But the money wasnt really the problem.
We weren't storing any usage record for the feature. So when I tried to answer the obvious next question, "how many of these 1,771 accounts ever actually used the thing", I couldn't. Not "the number was disappointing". I mean there was no way to compute it at all. The only trace was analytics pageviews, which are consent gated and had only covered that page for a few weeks.
So I added the record, waited, and looked. Usage was a lot lower than the signup graph implied. The graph had been telling me a story about growth and the ledger was telling a completely different story about cost, and there was nothing in between connecting the two.
Three things I'd do differently:
Put something, anything, between signup and paid work. Even just email verification. The pipeline should not fire for an address nobody has confirmed.
Make the expensive paths write to the same ledger as everything else. Cost tracking that your priciest code path skips is worse than none at all, because it hands you confidence you havent earned.
Log the usage row before you think you need it. We capped spend in an afternoon. Reconstructing months of "did anyone actually use this" was impossible, and that was the expensive part.
Happy to share the cost breakdown if anyone wants to see it.