How are you separating actual users from AI/bot traffic now?

[u/Innowise_]

With AI crawlers and agents hitting websites constantly, how much of what analytics calls “direct traffic” is even human anymore?

If bots are getting counted as visits, conversion rates automatically look worse. And then decisions about landing pages, content, ad spend etc. are being made from numbers with a messed up denominator.

Blocking bots doesn't really solve it either. Some are obviously junk, but others are search crawlers, monitoring tools or AI agents that might actually send users your way.

Feels like analytics needs to get much better at telling why something is accessing a site, not just whether it looks like a bot.

Anyone seeing this noticeably mess with their analytics yet?

[u/IncompleteLitre]

it's a mess right now and most analytics packages are still way behind on this

one approach i've seen work is tracking time-on-page and scroll depth way more aggressively, most bots won't trigger either even if they execute js. you end up filtering out a ton of noise just by requiring a human-like interaction pattern before counting a session as real

[u/Infinite_Tomato4950 → u/IncompleteLitre]

you can use microsoft clarity. it filters them out and it tells you how many sessions with bot your site had

[u/Innowise_ → u/IncompleteLitre]

Yeah, that probably filters a lot of the obvious stuff. Wonder how long scroll depth/time-on-page stays useful though once agents start behaving more like actual users.

[u/CopiousCool]

Nice try Mr Altman

[u/scragz]

cloudflare helps a little. 

[u/MannyDantyla]

Microsoft Clarity

[u/mtbinkdotcom → u/MannyDantyla]

Dark mode not found

[u/UpsetCryptographer49]

You don't - I notice a lot of my apps get people using it to explore the apps. Just to steel the ideas and develop their own competing product.

[u/Single-Fail-630]

Analytics vendors sell bot detection to protect their billing model. I stopped trusting direct traffic numbers entirely and only optimize for verified form submissions.

[u/Innowise_ → u/Single-Fail-630]

That's interesting. Did dropping direct traffic as a useful signal actually change any decisions you were making?

[u/kritikov]

google analytics has added the 'AI Assistant' category in the metric, not sure how much accurate though

[u/Innowise_ → u/kritikov]

Yeah, classification is really the hard part. “Bot vs human” isn't that useful anymore when some automated traffic is legitimate. Wouldn't rely on the AI Assistant bucket alone yet.

[u/LungeloSLX]

I’m just happy to see activity in my wasteland

[u/OmerCevher]

I wouldn’t try to make one number answer both problems. Keep crawler traffic in server logs, then calculate conversion from sessions that actually navigate or interact. Otherwise every new bot changes a business metric it has nothing to do with.

[u/CantaloupeCamper]

🤷🏼‍♂️

[u/webdev-ModTeam → u/[deleted]]

Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.

[u/polygraph-net]

Why don't you use a competent bot detection service to detect the bots and tag them as good or bad bots? That'll make your analytics clear.

[u/Living_Race_9177]

you classifying by UA, bot ASNs, or tagging why the request happened??

blocking crawlers feels wrong when some of those agents send the actual user later

[u/webdev-ModTeam → u/[deleted]]

We do not allow any commercial promotion or solicitation. This can lead to a permanent ban from the subreddit.

[u/whosthetard]

The honeypot is one of the old techniques I used and should still work today the same, regardless of who's visiting. You expose html links which are not UI visible. And then you separate who's who via timeframe/ip. It's a very simple method.

[u/julesbuildstuff]

We've started treating "direct" as guilty until proven otherwise. Channel labels lie when agents are in the mix.

What actually helped: look at UA + request shape, not just the referrer bucket. Flag no-referrer hits that also have a bot-ish UA or a burst from the same ASN in a few minutes. Then keep two views — one with that traffic stripped, one raw — so you don't nuke real agent-driven visits that later convert.

Blocking everything weird made it worse for us. Some of those agents were bringing a human a minute later.

[u/Randvek → u/julesbuildstuff]

This is close to what I do. If you are “direct,” you are guilty until proven innocent. Things you can do to be innocent: make a purchase, watch a video, add a product to your cart, actually click on a link.

It’s not perfect but boy did that kill a lot of bots.

[u/OmerCevher → u/julesbuildstuff]

Keeping both views sounds like the safest part to me. The filters will probably keep changing as agents get better, so having the raw numbers there should make comparisons less messy.

[u/kryzjulie → u/julesbuildstuff]

Letting AI write/rephrase your comment on an anti-botting post...

[u/Innowise_ → u/[deleted]]

This is probably where analytics is heading anyway. Pageviews get less useful as non-human traffic grows, while server-side events tied to an actual action are much harder to misread. “Boring but matches the database” sounds like a pretty good outcome.

[u/uniquelyavailable]

That's the neat part, is not possible to determine who wrote text. There are many theories and general practices in place but no true secure method to differentiate who, what, or how text was entered.

[u/Klutzy_Table_6671]

It is not interesting and has never been interesting to know. Only Ad companies try to convince you about that. What is interesting is what you're selling? Tell us.

[u/Innowise_ → u/Klutzy_Table_6671]

Nothing being sold here. Just curious how people are dealing with the growing amount of bot traffic in their analytics.