On your dashboard, you see that the variant conversion rate is higher then the control rate. Looks like a winning test! The numbers look good … Read More

So first: all experiments start with a question about capability rather then outcome. Before you run the test, you want to know what the design … Read More

Sample size isn’t something that’s written down in some book. It isn’t a static value. Instead, there is an appropriate number for you based off … Read More

I have a hunch my new web design is doing better then the old one. I feel that my new email subject lines are cutting … Read More

The hypothesis is that your new teaching method boosts test scores. Time to prove it! You’re ready with the data, enthusiasm, and determination. You probably … Read More

Your hypothesis begins as a guess. Perhaps you believe your customers is less happy. Maybe you think a new product feature are more popular then … Read More

But what you’re seeing is a two by two table, and it looks uncomplicated. On one side are those who lived close to factory or … Read More

If you’ve read political polling reports or articles on health studies, or been subjected to tech company’s A/B tests, then you know what an odds … Read More

ANOVA is done: You run the test and get a significant F statistic. Your peers are impressed, the p-value is tiny, and yet… does it … Read More

So you run an analysis of variance and it’s significant. Great! But you don’t know anything except that from that single test result. A small … Read More