.avif)
False Negatives
.webp)
The result of a statistical test that fails to detect a real effect, wrongly concluding that there is no significant difference between two variants, when in fact there is. In the context of CRO andA/B testing, a false negative can lead to the rejection of a variation that is nevertheless performing well, simply because the test lacked power (e.g.: sample too small, insufficient duration, MDE too low).
False negatives are often contrasted with false positives, which falsely indicate that an effect is significant when it is not. The balance between these two errors depends on the statistical parameters chosen (confidence level, power), and must be carefully calibrated to avoid erroneous decisions in optimization tests.
Similar definitions
MDE (Minimum Detectable Effect)
The smallest difference which, if it exists, can be reliably detected in a statistical test.
Statistical power
The probability that a test will detect a real effect, if such an effect exists. High power reduces the risk of false negatives.
Uplift
The increase in conversions or value generated by a specific action, often measured in A/B tests.
Mann-Whitney U Test
A non-parametric test used to compare two independent samples to determine whether they come from the same distribution.
p-value
A statistical measure that indicates the probability of observing the data collected (or something more extreme) under the null hypothesis.
Frequentist
A statistical approach that estimates model parameters based solely on observed data, without using any a priori information.
.svg){kind=small}
