Two jobs, one tool. Size a test before you launch, so you know if your traffic can ever prove the lift you're chasing. Or check a finished test to see whether the result is real or just noise. The exact math is shown, and your numbers never leave this page.
A free tool from Online Hub, a CRO studio for marketplace, C2C and booking platforms in the US, UK and Canada. When you want more than a rough read: Growth Audit Lite ($249, self-serve) or the Growth Audit ($2,500, done for you).
Use the conversion rate and traffic of the page or step you'll run the test on.
The defaults are the standard, defensible choices. Change them only if you have a reason.
No black box, this is the standard frequentist test for comparing two conversion rates, the same one mature experimentation tools use.
For a two-proportion test with equal traffic split, the visitors needed per variation is:
n = ( z_α·√(2·p̄·(1−p̄)) + z_β·√(p₁(1−p₁) + p₂(1−p₂)) )² ÷ (p₂ − p₁)²
where p₁ is your baseline, p₂ = p₁ × (1 + MDE) is the target rate, p̄ = (p₁+p₂)/2, z_α comes from your confidence level (two-sided uses α/2), and z_β from your power. With more than two variations, α is split across the extra comparisons (Bonferroni) so the false-positive rate across the whole test stays at your chosen level, which is why 3 or 4 variations need more traffic each. Run time is total sample ÷ weekly traffic.
The observed rates are p₁ = x₁/n₁ and p₂ = x₂/n₂. Using the pooled rate p̂ = (x₁+x₂)/(n₁+n₂), the test statistic is z = (p₂ − p₁) ÷ √( p̂(1−p̂)(1/n₁ + 1/n₂) ). The two-sided p-value is 2·(1 − Φ(|z|)), and confidence is 1 − p. The confidence interval on the lift uses the unpooled standard error √( p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂ ). Φ is computed with a numerical approximation accurate to ~1e-7.
This assumes a fixed sample decided in advance, random assignment, and one look at the end. Peeking at a running test and stopping the moment it crosses 95% inflates your false-positive rate badly, size the test first, then wait. It also won't catch a broken test (uneven split, tracking bugs, novelty effects). It's the statistics, not the judgement.