How many impressions are enough to test a creative?

[benpof]

I get this question asked ALL the time so let’s take a moment to consider our options. Here’s some ways to test your creatives systematically.

1) Timeline. More about that: How many impressions are enough to test a creative? « Ads.pof.com Blog

2) Payout based. Usually 1x-3x the payout you would receive each CPL or CPS etc.

3) Budget/Impression based. The only danger here is that you may under/over test an ad. For example, $5/day doesn’t get you much data. But if that is your budget, make sure that $5 is spent on a very small niche. Better to gain a lot of insight on a small group than to learn very little on a big group.

Honourable Mentions:

4) Relative Performance: Look at your set of ads, is 1 ad CLEARLY under-performing relative to the rest of your set? It’d be a safe bet to kill the ad before it hits any of the above mentioned thresholds.

It costs money to test. Everyone knows that but the danger of under-testing is just as damaging as over-testing so please make sure you’re spending your advertising dollars wisely. Do you test your ads in a way not mentioned here? I’m sure the community would love to hear about it, please leave a comment!

Ben

 

[CashAdsSarrah]

The # of impressions is actually not the best criteria to properly split test creatives. For example, let’s say that the magic # was 30,000 impressions. Well, depending on the size of the demographic you’re targeting, it may take 30 minutes or it may take 30 days to hit your goal.

A better way is to run a full week’s worth of traffic. Account for things like weekday vs. weekend traffic, daytime vs. afternoon vs. evening vs. late night traffic (or simplified “at work” vs. “after work” traffic).

Also, when collecting & comparing data, never compare today’s performance with yesterday or the day before. Instead compare data week by week, meaning Tuesday (today) vs. last Tuesday vs. the Tuesday before that.

So how much testing is enough? A week’s worth.