A/B Testing batch campaigns in the email editor
To A/B Test a batch campaign in the email editor, proceed as follows:
1. Create your versions
On the Content Creation page, open the Versions tab. Your email will be shown as Version #a.
On the Versions tab, copy the email and create a second version.
Make your changes to the new version and save them. You can change anything you like in the version, but the version name (Version #b) is internal and cannot be changed.
Repeat this for every version you want to create.
You can edit, copy and delete versions on the Versions tab. The original version (Version #a) cannot be deleted.
Contacts of a CC list receive all versions of a multi-version email campaign (e.g.: A/B split test batch email campaign).
When you are finished with your versions, open the Scheduling page.
Steps before launching your campaign
- Once you have switched to Advanced Scheduling and performed any actions there you will not be able to switch back to Simple Scheduling.
- Engagement rates within a segment are not taken into account when using A/B testing batch campaigns.
- This means that clients should use segments that contain contacts with similar engagement, otherwise the results might be skewed. Contacts from a segment containing both inactive and highly engaged people are not divided into test target groups for test versions equally. It is possible that some versions receive a higher proportion of contacts with high or low engagement rates.
- The Launch Now button is not active unless you Check personalization.
Click Check Personalization to review the availability of the personalization variables used in the email campaign, such as: title, first name, last name.
The system checks whether the contacts in the selected recipient source have the corresponding entries in their individual profiles. The results are presented in the Check personalization window.
If the availability of a variable is below 100%, you can decide to insert an Alternative text or to Omit recipients who do not have a corresponding entry.
Launch the test versions
Under Select launch type, add the percentage of contacts who will receive the first version, and give that version a name. This name will be added to the campaign name to differentiate between the versions.
Contacts are randomly selected from the launch list and then distributed among the different test versions based on the specified percentage.
Please do not enter a value lower than 1% or values with decimal separators.
Select version to send
Select the version you want to send for this test.
Set the launch date. You can choose to Launch Now or Launch later at a specified date. You can launch on selected day/time, if launch is not right now.
If you choose to Launch Now, the campaign will be launched when you have activated the Final mailing option.
Due to system requirements, the final launch needs to be scheduled at least one hour after the time of the last scheduled test.
As you schedule versions for launch, they are listed in the Scheduling page:
There is no limit to the number of versions you can create and test but you cannot send tests to more than 90% of the launch list. If you simply want to send different versions of the same email without waiting to see which performs better, see Sending different versions to fixed percentages of the launch list below.
When you have launched all your versions, select the criteria for the Final mailing. You can set the toggle to Manually and decide which version to send yourself, or set it to Automatically and let Emarsys choose the version based on these possible criteria:
If you choose the automated version, after collecting some sample responses (it takes approximately one hour but it may vary from account to account), Emarsys will decide which is the best-performing version. When you are ready to launch this version, choose from the following options:
- To send it now, select Launch now.
- To send it later, select Launch later and set the launch date.
Launch is not a single event in case of A/B testing batch email campaigns. Launching A/B test mails means that the different email versions are partially launched. Only one version is launched at a time.