This article applies to Nelio A/B testing versions prior to 5.0.
If you are looking for documentation for our newest version, please bookmark neliosoftware.com/testing/help/
Nelio A/B Testing uses the G-test statistic for computing the statistical significance of the results of an experiment. The G-test statistic is a measure of how much overall variation there is from an ideal prediction that you would expect if all versions were the same. It produces a confidence value that tells you how “trustable” is the fact that one alternative is better than the other/s.
A high confidence value tells you that you can be certain that choosing the best alternative is the best option to improve your conversion rate. A low confidence value means that with the current results, even if one alternative has so far a higher conversion rate than another, this could still be due to random effects so you have to interpret the results with caution.
We recommend waiting for at least a 90% confidence value in order to be sure to choose the best alternative.
The more visitors you have and the bigger is the observed difference between the alternatives, the higher is the confidence value.