.avif)
False positives
.webp)
The result of a statistical test which wrongly indicates that an effect is significant, when in fact it is not. In CRO andA/B testing, a false positive can lead to the validation of an ineffective variation, which may result in counter-productive decisions (e.g.: generalization of a version that brings no real gain).
False positives are linked to the confidence level chosen for a test (often 95%): there is always a probability (e.g. 5%) of concluding that a difference is simply due to chance. This phenomenon is all the more critical if many tests are carried out simultaneously, with no control over the false positive rate (e.g. via Bonferroni correction or false discovery test).
False positives are the opposite of false negatives, which miss an effect that is actually present. Striking the right balance between these two types of error is essential for reliable experimentation and a sound CRO strategy.
Similar definitions
MDE (Minimum Detectable Effect)
The smallest difference which, if it exists, can be reliably detected in a statistical test.
Statistical power
The probability that a test will detect a real effect, if such an effect exists. High power reduces the risk of false negatives.
Uplift
The increase in conversions or value generated by a specific action, often measured in A/B tests.
Mann-Whitney U Test
A non-parametric test used to compare two independent samples to determine whether they come from the same distribution.
p-value
A statistical measure that indicates the probability of observing the data collected (or something more extreme) under the null hypothesis.
Frequentist
A statistical approach that estimates model parameters based solely on observed data, without using any a priori information.
.svg){kind=small}
