Using A/B tests in Blueshift, you can see which variation of a template performs better. You can test several templates that may have different forms of personalization or entirely different content. For email templates, you can also conveniently test different subject lines for the same content.

You can set up the A/B test from the A/B Test tab of the trigger.

jb_abtest.png

Email subject line tests

To set up a Subject Line A/B test, complete the following steps:

  1. Select an Email Template from the Email tab of the trigger.
  2. Click Create A/B Test or go to the A/B Test tab to set up the test.
  3. On the A/B Test tab,  select Subject Line as the Type.
  4. Click Add Variation to add a new subject line and the percentage of users to allocate to it.
    • You can use the same markup to customize the subject as you would for the email template.
    • You can add up to 20 subject lines to test.
    • The first subject line is your control (the default subject line of the email template) and cannot be edited.

ABTesting_subjectline_Feb2022.png

After you set up the Subject Line A/B test, you can view the information for the test on the journey component in both the Compact and Detail modes.

jb_abtest_subjectline_view.png

Message content tests

To A/B test non-email templates or content beyond just subject lines, you must create several templates to use in the tests. If your templates are similar to each other, you can use the “clone” functionality after creating one template and edit the cloned template.

jb_clone_template.png

To set up an Entire Creative A/B test to test the message content, complete the following steps:

  1. Click Create A/B Test from the Email tab or go to the A/B Test tab to set up the test.
  2. On the A/B Test tab, select Entire Creative as the Type.
  3. Click Add Variation to add a new template and the percentage of users to allocate to it.
    • You can add up to 20 templates to test.
    • The first subject line is your control (the default subject line of the email template) and cannot be edited.

ABTesting_creative_Feb2022.png

After you set up the Entire Creative A/B test, you can view the information for the test on the journey component in both the Compact and Detail modes.

jb_abtest_content_view.png

Archive a variation

After you launch a campaign, you can archive any of the A/B Test variations by setting the percentage of users allocated to it to 0%. You must redistribute the users to other variations and ensure that the total percentage of users is always equal to 100%.

  • For a launched event triggered campaign, the variation that you archive is available in the Archived variants as soon as you Save the campaign.
  • For a paused event triggered campaign or paused segment triggered campaign, the variation that you archive is available in the Archived variants after you Save and then Launch the campaign.
  • For a launched segment triggered campaign, you must pause the campaign to archive any A/B Test variations.

Clone a variation

If your A/B Test variations are similar to each other, or if you want to copy the contents of any variation, you can Clone the variation. You can clone active and archived variations. Cloning is an easy way to revive archived variations.

When you clone a variation, the subject line or the template is copied to the new variation along with any tracking parameters and notes. You can then edit the variation and use it in the A/B test.

ABTest_clone_variant.png

Measuring & Optimizing Campaigns

Once the campaign is live, you can access the detailed analysis under the Reports > A/B Test Results tab of the campaign studio. For each template, you will see the number of users, impressions, clicks as well as orders/revenue and custom goals data. This helps you measure template changes and their effect on downstream conversions and custom goals.

For more information, see A/B Test Reports.

A/B Test Email Components

When you run an A/B test on the entire creative, you can test the overall effectiveness of a template. The default A/B test report looks at the total conversion ( e.g. clicks) for the entire creative. However, if you wants to look at the conversion for a specific component, like a recommendation block, then you can use the template click heat map.

For example, let’s say you want to include a “customers who bought X also bought Y” recommendation in your email and you want to figure out what look back period to use for this recommendation so that you get the best engagement. You can create a recommendation scheme, Rec A, with a 30-day look back period and include it in Email template A. Add another scheme, Rec B, with a 90-day look back period and include that in Email template B. Run an A/B test on templates Email A and Email B. Now when you look at the click heat maps for these templates, you will be able to check the click through rates on the recommendation blocks Rec A and Rec B. With this information, you can determine which of the look back periods had a higher engagement.

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.