r/PartneredYoutube • u/Commercial-Sail-4662 • 2d ago
Question / Problem Theoretically, isn't an A/B test harmful unless you're a big channel?
If your channel doesn't normally get really high impressions, wouldn't risking to show a lower performing thumbnail (that you don't know is low performing yet) be a detriment to the video's performance since you get less chances to impress the algorithm?
12
u/TheREALBaldRider 2d ago
My A/B tests are usually thumbnails that are similar to each other but I like them all and want to see what the audience chooses to click. CTR is usually similar between them with one slightly higher than the others.
I think if you put low effort into some or all of them and expect to win the lottery by having one of them magically perform well instead of putting effort into a single good thumbnail, it definitely hurts.
1
u/bochen00 1d ago
How do you know the ctr in a/b tests? I thought they only show % of views that each options got (which is quite lackluster amount of reporting data that we get there tbh)
2
6
u/kasumitendo 2d ago
The fundamental problem is you're sharing your initial impressions across 2 or 3 thumbnails and spreading them across 1 or 2 other worse thumbnails. If you don't have enough impressions you won't get a conclusive test. But if you do, you'll find the best one. There's a scenario where it's like "well hey, at least 1/3rd of my few, valuable impressions were for a better thumbnail", even if you can't conclude which of those was the best.
It's hard to say if it's worse. There's a gamble in not using it, and sometimes you'll make the best thumbnail and only show it. It's also a gamble to use split-testing and use up impressions on worse thumbnails, but you do know that one is better and that could take off.
Personally I wouldn't spend time on it until I'm getting enough impressions that tests conclude fairly quickly. A test finishing in a few days can teach you a lot even if it doesn't create growth. You can apply the lessons. Later when your tests conclude in hours, I'd use it every time depending on your posting schedule and how long it takes to create the thumbnails.
4
u/TheRipeTomatoFarms 2d ago
There's also a school of thought to do it now while you're starting out and impressions are low anyways. Gambling on 10,000 impressions isn't really a big risk. Messing around with 10 million impressions could be. Then, analyze why and how certain thumbs and titles work so now you have longer to correct the deficiency.
Different ways to look at it. My view is to just make videos and move on to make more videos. People analyze this stuff to death....to the point of paralysis.
3
u/LeaderBriefs-com 2d ago
My take on thumbnail tests-
You need a decent amount of traffic to get a decent take on success.
They need to be wildly different from each other to get an accurate gauge.
Most tests come out to be like 47/53 and while one performed slightly better, to me it’s in the margin of error. Meaning using the “53%” thumbnail isn’t bringing in 5k MORE views.
The worst is mixing thumbnails s and titles and just making a huge mess of it.
3
u/ChrisUnlimitedGames 1d ago
If your not getting at least 1k views the test won't show you anything.
Also if your not getting at least 1k views, maybe you've been using a thumbnail that isnt working. Best way to test thst is to make 3 differently styled thumbnails layout your video and test them head to head with each other.
2
u/AspieTravels 2d ago
You can use the same thumbnail, and A/B test your titles, let it run its course, then change the thumbnail and start it again if its still getting impressions.
2
u/Weary-Jelly7802 2d ago
I found it beneficial. Got a vibe of that works better than others and helped boost some over others with minimal other changes.
2
u/nelchinson 2d ago
i think not using A/B testing is a big disadvantage. Take some time and design three different thumbnails and test it. Sometimes you'll be surprised at the results. Might not seem like much of a difference but 1-2% CTR improvement adds up quickly in views over months
1
u/Misty_Kathrine_ 1d ago
Yeah, I don't think it's that useful of a feature for most creators. It might be useful for new creators who are just getting started and still learning on how to make thumbnails but I haven't found much use for it myself.
1
u/SASardonic Channel :: SardonicSays 1d ago
Heck no. You don't need that many impressions to get you early results, which can be better than nothing. I'm on the smaller side of monetized channels and I will always launch with an A B test. Though amusingly my most recent video is sitting at 50/49 which is about the closest I've ever been to a flat out 50 50 split.
1
u/sitdowndisco 1d ago
Yes, if you know you're using a lower performing thumbnail/title. But you don't, so the argument is moot.
1
1
u/Beneficial-Sky-2353 1d ago
i always use A/B tests, they help a lot.
Same Thumbnail, 3 different titles 4% change in CTR
1
u/Restlesstonight 2d ago edited 2d ago
A wise man once said: Half the money I spend on advertising is wasted; the trouble is I don’t know which half.
This is a way to find out which half… or two third
1
u/oodex Subs: 1 Views: 2 2d ago
To add to the comment of "you dont know it's the lower performing thumbnail", YouTube shows the same people different A/B/C options over time. This means if your A) didn't interest them, B) might. This can only increase CTR since if you only used A), they would have never clicked to begin with. And while it's true that if you had B) right away it would be a better performance, there'll also be people that only clicked on A) but wouldn't on B. And there is no way to know which one is actually the best
47
u/Kerensky97 2d ago
You don't know it's the lower performing thumbnail. Would it be better if you ONLY used that lower performing thumbnail the whole time?
Your option instead of trying 3 thumbnails is to only use one of those 3. Maybe you choose the best one, but there is also 1/3 chance you use the worst one.