What is this A/B significance calculator?
A free, browser-based calculator that tells you whether the difference between two conversion rates is real or just noise. Enter the visitors and conversions for each variant and it computes both conversion rates, the relative uplift, the p-value, your confidence, and a clear verdict — significant or not — at the confidence level you choose. Everything runs locally in your browser, so your data is never uploaded.
Methodology and assumptions
We use a two-tailed, two-proportion z-test with a pooled standard error — the standard method for comparing two conversion rates. The z statistic is (rateB − rateA) ÷ pooled standard error, and the p-value comes from the standard normal distribution. A result is flagged significant when the p-value is below (1 − confidence).
- Each visitor's outcome is independent (a fresh visitor, randomly assigned).
- The sample is large enough — a rule of thumb is ~10+ successes and ~10+ failures per group.
- Don't repeatedly peek and stop early — decide the sample size up front to avoid false positives.
How to use it
- Enter visitors and conversions for variant A (control) and B (variant).
- Pick your confidence level (90% / 95% / 99%).
- Read the verdict, uplift, p-value, and confidence on the right.
Common use cases
- Deciding whether a new landing page truly beats the old one.
- Checking an email subject-line or ad creative test before rolling it out.
- Avoiding shipping a change that only looked better by chance.
