How it works
z = (rate B − rate A)/√(pooled rate × (1 − pooled rate) × (1/nA + 1/nB)). Requires independent observations and a suitable normal approximation. Sparse outcomes, repeated peeking and multiple comparisons can invalidate interpretation; no automatic winner or causal claim.
Worked example
If version A converts 100 of 1,000 visitors (10%) and version B converts 120 of 1,000 (12%), B is up 2 percentage points, a 20% relative lift, with a pooled z of 1.43 and a two-sided p-value of about 0.15.
How the conversion-rate difference changes with group A visitors
Other inputs keep their starting values: Group A conversions: 100; Group B visitors: 1,000; Group B conversions: 120. Examples only, not recommendations.
| Group A visitors | Conversion-rate difference (percentage points) |
|---|---|
| 500 | -8 |
| 1,000 | 2 |
| 2,000 | 7 |
Cite this page
ToolOctopus. “A/B test calculator.” Updated . https://tooloctopus.com/ab-test-calculator.
Add an access date if your instructions require one.
Citation formatting uses citeproc-js by Frank Bennett and Citation Style Language styles. License and source code.