How to A/B test two different rules
Control Group
Control group is a setting that is available in the exposure menu for behavioral targeting rules.
A rule that uses a control group is displayed to everyone who is included in the targeted segment except for a random group which is the control group. Essentially what is being tested is the difference in conversion rate between those who see the rule and those that do not.
A/B Testing
You can assign two rules to the same priority group in order to test them against each other.
To run an A/B test:
- Set a control group of 50% to rule A and assign it to a
priority group.
- Set rule B to the same Priority group but with a lower priority.
In the example above Rule A has a control group and is assigned to group F with priority 1. Rule B doesn't have a control group but is assigned to group F with priority 2. Rule A will trigger 50% of the time. Rule B will trigger whenever rule A does not.
Understanding the results
The results are measured by clicking on the "stats" icon next to the rule with the control group. Assuming that the rule with the control group was rule A, the statistics can be interpreted as follows:
