Skip to content

A/B test significance

Enter visitors and conversions for two variants. Reavlo tells you whether the difference is real, and how big it probably is.

  • Runs in your browser
  • Nothing is uploaded
  • Free, no sign-up
  • method v1.0.0
  • Updated Sep 2026

Input

Variant A (control)
Variant B

Updates as you type · nothing leaves this page

Result

Nothing on the bench yet

Enter visitors and conversions for both variants, or load an example.

How to use it

Enter the visitors and conversions of each variant as your testing or analytics tool reports them. The result updates as you type.

  1. Use the same date range and the same conversion for both variants.
  2. Pick the confidence level you decided on before the test started. 95% is the common default.
  3. Read the verdict, then the interval: it tells you how big the difference probably is, not only whether it exists.

How Reavlo tests this

The calculator runs a two-sided two-proportion z-test, in your browser.

  1. Conversion rates: pA = conversions A ÷ visitors A, and the same for B.
  2. Pooled rate: p = (conversions A + conversions B) ÷ (visitors A + visitors B).
  3. Standard error under "no difference": SE = √(p × (1 − p) × (1 ÷ visitors A + 1 ÷ visitors B)).
  4. z = (pB − pA) ÷ SE, and the two-sided p-value from the standard normal distribution.
  5. The difference is significant when the p-value is below 1 − confidence (0.05 at 95%).
  6. The interval for pB − pA uses the unpooled standard error √(pA(1 − pA) ÷ visitors A + pB(1 − pB) ÷ visitors B) and the critical z of your confidence level (1.645, 1.960 or 2.576).

Lift is (pB − pA) ÷ pA. The normal distribution is computed with a complementary error function accurate to about 1 part in 10 million.

Near the threshold the pooled test and the unpooled interval can disagree (for example "not significant" while the interval excludes 0). The result is then marked borderline: treat it as inconclusive.

Limits. The z-test relies on a normal approximation. When a variant has fewer than 5 conversions or 5 non-conversions, the result is flagged. The test assumes you fixed the sample size in advance: checking every day and stopping at the first significant result inflates false positives.

Questions

What does a p-value of 0.02 mean?

If there were no real difference between the variants, a difference at least this large would show up in about 2% of tests of this size. It doesn't mean there's a 98% chance B is better.

Why is my test not significant even though B looks better?

With few visitors, random variation alone produces differences of that size often enough. The interval shows the range of differences that are still consistent with your data.

Can I use it for more than two variants?

Compare each variant with the control separately, and be aware that every extra comparison raises the chance of a false positive.

Related tools

MeasurementIn browserUTM link auditorPaste your campaign links. Find inconsistent sources, missing parameters and naming drift.Checker · Findings