CODEWITHSUNDEEP
reverse-engineeringab-testingoptimizely
Vlad Mysla
Vlad Mysla

Reputation: 1181

How can I make sure that only one Optimizely Experiment is running on a page and it gets fair traffic?

Using Optimizely I can create one or more experiments in a same project, but there is no guarantees that they will not executed simultaneously. For instance, if i'll create 2 experiments for a particular URL with Traffic Allocation for 100% visitors - all of them will receive both experiments. This isn't good, as experience given by first experiment will be combined with second one; they may conflict and potentially affect final numbers.

Official documentation tells that you can use custom JavaScript to setup experiments in the way that only one of them in a group will be activated for a visitor:

https://help.optimizely.com/hc/en-us/articles/200040205-Mutually-exclusive-experiments

I'm ok with JavaScript, but unfortunately, it requires to have predefined list of exclusive experiments in code and copy-paste that script into each of experiments. Also, that technique doesn't takes into account Targeting rules, as having all those experiments in a list doesn't guarantees that all of them will be activated.

I'm looking for JavaScript, which can be placed above/below Optimizely snippet. That code should know which of experiments are matching for current page view and activate only one of those experiments.

Update: 12 July 2014 Optimizely updated their code, so it seems like now everything becomes even more complicated. Anyway after long reverse-engineering and debugging session, it looks like problem might be solved in case if we can randomize order of existing experiments before testing them for traffic allocation rules (after today's changes - unsegmented audience).

Upvotes: 1

Views: 810

Answers (2)

expertCode
expertCode

Reputation: 533

Step 1: Change the activation mode on both experiments to "manual activation".

Step 2: create a third experiment (proxy) with one variation. So you will have the control and the variation 1.

Step 3: On the control manual activate the experiment1 and on the variation 1 manual activate the experiment2

Conclusion: The proxy will split the traffic between both experiment (50/50) and guarantee that the experiments will never run at the same time.

Upvotes: 2

Vlad Mysla
Vlad Mysla

Reputation: 1181

Optimizely can't guarantee that only one experiment will be activated per a given page. Also, the method that they propose can't guarantee that those experiments will get 50%/50% of the traffic.

In a case when you known about having those 2-4 concurrent experiments on a page and you want distribute traffic with in a specific proportion, you have to add randomization code to the:

  1. Script above the Optimizely snippet on the page.
  2. Project JavaScript
  3. Global JavaScript of the each concurrent experiment
    if(!window.trafficSegment) {
      window.trafficSegment = Math.floor(Math.random()*4+1);
    }

This code generates random number from 1 to 4 with a same probability - 25%. Now you can use this variable in your Audience targeting conditions.

enter image description here

Another experiment can have an audience with targeting for a second half of the traffic:

var ts = window.trafficSegment;
return ts >=3 && ts <=4;

This isn't a perfect solution but that's the simplest way of doing this without patching Optimizely code.

Upvotes: 0

Related Questions