Sample size scales with 1/effect². Detecting a 5% lift instead of a 10% one costs four times the traffic, not twice.
The difference between the two rates is squared in the denominator. A 5% conversion rate moving to 5.5% is a difference of 0.005, and squaring it gives 0.000025 — which is how a numerator of about 0.1 turns into 31,000 visitors per variant. Teams plan for the effect they hope for and budget for the effect they expected, and the gap between those two is quadratic.
It assumes you fixed the sample size before starting. Stop the moment a result looks significant and the arithmetic no longer applies — repeated peeking pushes the false-positive rate from 5% towards 30%, so the test that finished early was never the test you sized. Sequential and Bayesian designs exist precisely to allow looking.
At 5% the rule says 57,760 and the exact answer is 57,760 — within half a percent. It holds to within 5% up to 17.4%, then drifts.
The inverse square is right about the shape and slightly wrong about the size, because the variance term moves with the effect as well: a bigger lift means a bigger second proportion, and near 10% baseline that pushes p(1−p) up. Anchored at a 5% lift the rule under-books by about 9% at the large end — small, but it is the direction that leaves a test underpowered.
The rule against the exact answer, computed across the range. Inside the shaded band the shortcut is close enough to use; outside it, reach for the calculator.
Estimate with the rule, then check it against the calculator that models it properly.
Open A/B Test Sample Size Calculator →Sample size scales with 1/effect². Detecting a 5% lift instead of a 10% one costs four times the traffic, not twice. The difference between the two rates is squared in the denominator.