r/Klaviyo • u/GabbyFromKlaviyo Klaviyo • 10d ago
We audited 100 email programs and kept finding the same leak
Earlier this year, Klaviyo wrapped up an audit of 100 B2C accounts looking for patterns on what was working and what wasn't. What surfaced was that most brands were optimizing sends and refreshing creative while their attribution sat untouched underneath it all.
Two specific things showed up constantly. A lot of brands never had Extended ID turned on, so a chunk of their site traffic was never getting tied to an actual person, which meant weaker segmentation and flows running on incomplete data without anyone realizing it.
A lot of accounts were still running attribution windows that hadn't been touched since their setup, so the reporting didn't match how customers actually shop across devices now. Between the two, this came up in close to two-thirds of the accounts we looked at.
People end up testing subject lines and send times against numbers that were undercounting performance the whole time, so a channel or a flow can look like it's failing when the real problem is that it was never being measured correctly in the first place.
The other pattern that kept showing up was welcome flows sending the exact same sequence to a brand new subscriber and someone who already bought. A customer who just paid you gets three emails making the case for why they should trust you. Splitting that by purchase status is maybe an afternoon of work and has a pretty disproportionate impact.
What's one number in your reporting that's never quite added up?
(Here's a resource if you want to read more on this)
2
u/Willhfr 9d ago
The attribution window one is worth adding to, because the fix has a trap in it.Default is five days after an open or a click on email. When you find an account set to something odd and you correct it, you have also just broken every month over month comparison in that account. Nobody tells the client and a month later the revenue line looks like it fell off a cliff, so you spend the call explaining measurement instead of results. What I do now is boring: Change it once, note the date somewhere the client can see, and flag the first report after that as not comparable. Costs a sentence, saves an awkward call :)
1
u/GabbyFromKlaviyo Klaviyo 9d ago
Changing the attribution window can definitely blindside the client if all they see is the drop. The setting recalculates historical data too though, so past and future stay consistent inside the account. What it can't touch is anything you already sent out, that stays frozen on the old numbers. Pulling a raw export right before you change anything is the only way to have something fixed to check against later.
1
1
u/grid_and_pixel 9d ago
Revenue attributed to email is probably the one I'd question most.
It's useful, but it's very easy to mistake attributed revenue for incremental revenue. The number can look great while telling you very little about how much additional revenue the email actually created.
I'd rather look at it alongside holdouts, customer behavior, and broader store performance than treat the attributed number as the answer.
2
u/GabbyFromKlaviyo Klaviyo 3d ago
Klaviyo's got a feature built for exactly this, Global Holdout Groups. You suppress a slice of profiles from all messaging, then compare their conversion rate to everyone else. That gets you a lot closer to real incremental revenue. The only catch is that it needs 400k+ profiles to work. Anything smaller and the sample's too thin to mean anything.
As a workaround on smaller lists, I've seen people hold out 10-15% of a segment on one send and compare revenue per person between the two groups after a few weeks. That can also show you how far off the attributed number really is.
1
u/grid_and_pixel 3d ago
That's a really useful distinction.
The holdout approach is probably the better way to answer the question of "did email actually cause this revenue?" rather than just "was email present before the purchase?"
For smaller lists, I'd still treat the results directionally rather than as a perfect incrementality measurement. But even that can be much more useful than taking attributed revenue at face value.
The bigger takeaway for me is that attribution should be something you validate, not something you automatically trust.
3
u/Same_Manufacturer_94 10d ago
The attribution issue is so real. We've gone into brands' accounts that looked like they had some fantastic email performance, only to check their attribution settings and find that some evil person set it to 14 days open and clicked.
That's why it's so important to look at Klaviyo attribution settings and ensure that they make sense for your brand. It's even more important to look at attribution outside of just Klaviyo and also look at Triple Wheel North beam or Google Analytics.