Plan your ad test for millions of people, not thousands
Updated: Oct 11
Small ad tests usually cannot tell you if an ad paid for itself.
The study. Two economists studied 25 large field experiments on display ads. US retailers and financial companies ran the tests with a large web portal.
The finding. Each person's spending changes a lot from week to week. The effect of an ad is very small compared with this normal change. So a test must be very large to measure if an ad made a profit.
Why it matters. Many teams run a small ad test, see no clear result, and decide that the ad works or does not work. Both decisions can be wrong. The test was too small to know.
The takeaway. Calculate the sample size before you run an ad test. If you cannot get enough people, measure a nearer result, such as site visits, and say that the profit is not known.
Want to read more?
Subscribe to plaintalkbusiness.com to keep reading this exclusive post.
