Use these 7 tested rules to run better experiments

This image shows a Chestnut-sided Warbler from John James Audubon's The Birds of America (1827-1838).

Most ideas fail. A few ideas give very large gains. Gains usually become smaller when you use the idea for all users. Plan for all 3 results.

Who it is for. This playbook is for product and growth teams. These teams run A/B tests (tests that compare 2 random groups), or they want to start.

How to use it. Each rule gives the evidence, 1 action and a link to the full study. Start with the rule that fits your biggest problem this quarter.

1. Test more ideas with smaller tests

Evidence. In 1,450 Bing experiments, the top 2% of ideas made 74.8% of the gains.

Do this. Increase the number of tests that you run each month before you increase their size.

Read the study →

Want to read more?

Subscribe to plaintalkbusiness.com to keep reading this exclusive post.

Subscribe Now