AB Test Calculator
Compare two conversion rates for statistical significance.
An A/B test checks whether one variant converts significantly better than another.
How the Math Works
The AB Test Calculator uses a two-proportion z-test to determine if the difference between two conversion rates is statistically significant. It first calculates a pooled proportion by combining the successes and total trials from both groups. Then, it computes the standard error using the pooled proportion and sample sizes. Finally, it derives a z-score by dividing the difference in conversion rates by the standard error, which is compared against a critical value to assess significance.
Practical Applications
To apply this calculator, input the number of conversions and total visitors for both the control (original) and variation (test) groups. The tool will compute the z-score and p-value, indicating whether the observed difference is likely due to chance or represents a true effect. This helps validate if changes to a website, email, or product genuinely improve performance before full implementation.
Day-to-Day Use
In daily life, this calculator aids in making evidence-based decisions without relying on intuition. For example, a marketer can quickly test email subject lines or a product manager can evaluate feature changes. By identifying statistically significant improvements, it prevents costly mistakes from random fluctuations and ensures resources are invested in strategies that deliver measurable results.
Worked example
10% vs 13% on 1000 each gives z about 2.04, p about 0.041 (significant).
FAQ
How big a sample do I need?
Use a sample-size calculator based on your baseline rate and the uplift you want to detect.