Stop Calling the First Significant Day a Win

By

Checking an A/B test until it crosses p < 0.05 can turn a nominal 5 percent false-positive rate into almost 28 percent. I use a seeded simulation to show how large the damage gets and compare the fixes that keep early stopping honest.

The post Stop Calling the First Significant Day a Win appeared first on Towards Data Science.

Source: Towardsdatascience.com

Original source: https://towardsdatascience.com/stop-calling-the-first-significant-day-a-win/

Post Views: 5

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *