This article applies to Nelio A/B testing versions prior to 5.0.
If you are looking for documentation for our newest version, please bookmark neliosoftware.com/testing/help/
The duration for which you need to run your A/B testing experiments depends on several factors such as follows:
- The complexity of the experiment. The more complex the experiment (more alternatives, with less differences between them, ...), the longer you will need to run the experiment to get significant results.
- The number of visitors who participate in the experiment. The more number of people you test, higher is the reliability of the test results.
Stopping an experiment too early may result in not get meaningful results, while keeping an experiment running for too long could make you lose conversions and sales due to poor performing alternatives in the experiment.
We’ve investigated the amount of time the experiments of our customers have been running. In average, all the experiments were running for less than a month and most of them didn’t surpass two weeks. In our opinion, that’s not too much regarding the benefits you can get in return! Also, if you fail to find a winning combination within these amount of time, you may conclude that the alternatives do not stand a chance to beat the original version.