This article applies to Nelio A/B testing versions prior to 5.0.

If you are looking for documentation for our newest version, please bookmark

Nelio A/B Testing is compatible with cache systems (including WordPress-specific cache plugins). When loading alternative content, Nelio adds one or more GET parameters in the URL:


that specify whether the original version (nab=0) or one of its alternatives (nab=1) has to be loaded. As a result, your cache system will see two different URLs, and will cache both of them as if they were two different pages (even though the latter is just a variant of the former). In other words, you'll have three pages cached: the first page without an attribute nab, the second with nab=0, and the third with nab=1.

The number of experiments and alternatives and the Exponential Growth

Imagine you define a Headline experiments for your two latest posts (one for each). For instance:
  • Post One
    • Headline A
    • Headline B
  • Post Two
    • Headline X
    • Headline Y

With this setup, there are 4 possible Latest Posts pages:
  1. A visitor sees both original versions: A X
  2. A visitor sees one original version and one alternative: A Y
  3. A visitor sees one alternative version and one original: B X
  4. A visitor sees both alternative versions: B Y

Which means your cache will have to cache each version. So far, it doesn't look like a problem, right? Now, what if numbers get bigger? Well, if you had three alternatives instead of two, the number of possibilities would be 9 (3x3). Not too much? What if you had 5 headline experiments with 4 alternatives each? Then you'd have 1024 possible Latest Posts pages!

Now think that all these pages have to be created for almost all your pages in your site (since any page in your website can potentially include a reference to the tested post). Those are some big numbers!

Divide and Conquer

In order to overcome this problem, Nelio A/B Testing offers a simple, yet very effective solution. We divide your experiments into one or more groups and, therefore, we reduce the amount of possible combinations. Following the previous example, if you had 5 headline experiments with 4 alternatives each, we could create the following groups:
  1. Group 1
    1. Experiment A (4 alternatives)
    2. Experiment B (4 alternatives)
  2. Group 2
    1. Experiment C (4 alternatives)
    2. Experiment D (4 alternatives)
  3. Group 3
    1. Experiment E (4 alternatives)

This way, we'd have 16 combinations for the first, 16 for the second, and only 4 for the third. As a result, the previous setup would no longer require your cache system to cache 1024 possible combinations, but only 36.

How to configure Nelio A/B Testing

You can configure Nelio A/B Testing to use (or not) the previously described solution. Just go to Nelio's Settings and modify the value of the option called Test Mode:

  • Group Experiments and Divide Visitors: your experiments will be divided into groups (as we just described). This will reduce the total amount of possible combinations, but each of your visitors will only participate in some experiments (that is, she'll participate in the experiments of one group or the other, but not in all of them).
  • Test All Experiments with All Visitors: experiments won't be divided into groups and all visitors will participate in all experiments.

Moreover, if you want to completely get rid of the exponential combination problem, also make sure to disable Site-wide Consistency.