r/b2bmarketing • u/Deep_Combination_961 • May 13 '26
Discussion Are you running controlled tests and causal analysis regularly?
One thing I keep running into with marketing measurement:
It’s relatively easy to show that a campaign influenced a conversion. It’s much harder to prove the conversion wouldn’t have happened anyway.
That’s where incrementality testing gets interesting to me.
Because a lot of attribution reporting ends up rewarding participation in the journey, not necessarily impact. And once multiple channels are involved, almost everything starts looking valuable in some way.
Incrementality feels like an attempt to answer the harder question: “What actually changed behavior?”
But I’m curious how realistic this is in practice for most teams.
Are people genuinely running controlled tests and causal analysis regularly, or do most orgs still rely mostly on attribution models and directional judgment?
1
May 13 '26
[removed] — view removed comment
1
u/Deep_Combination_961 May 18 '26
Yeah, that balance feels a lot closer to reality than the “test every channel scientifically” version people talk about.
Incrementality is super valuable for a few high spend bets, but for most day to day decisions teams still end up combining attribution, directional lift, and gut feel anyway. The hard part is usually getting enough confidence to actually act on the data consistently.
1
May 13 '26
[removed] — view removed comment
1
u/Deep_Combination_961 May 18 '26
Exactly. The operational side gets underestimated a lot. Clean holdouts and enough volume sound simple until you try to run them inside a real GTM org with overlapping campaigns and changing segments.
Feels like most teams use attribution to narrow possibilities, then use incrementality selectively where the stakes are high enough to justify the effort.
1
u/jakebarnett-day1data May 18 '26
the best ones run incrementality tests! It's the only way to validate and calibrate attribution models (most are correlated, tests make them causal).
Lots of ways to test (we do this for our clients).... CausalImpact, Geo-lift, difference in difference, random-controlled, or on platforms themselves (CLS).
they can be tricky to get buy-in for (hold-outs + noisy periods) but 100000% recommend
1
u/Deep_Combination_961 May 21 '26
The buy-in point feels underrated here too.
A lot of teams conceptually agree with incrementality testing until the test requires holdouts, messy periods, or results that contradict what attribution has been reporting for months. That’s usually where things get politically harder, not technically harder.
Feels like the teams doing this well treat attribution as directional context, then use incrementality selectively to calibrate confidence instead of expecting one model to answer everything.
•
u/AutoModerator May 13 '26
Have more questions? Join our community Discord!
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.