Defining A/B Testing
Act-On offers A/B testing functionality so that your organization can determine which emails will prove most successful for your ongoing marketing campaigns. The A/B email testing process allows you to send one variation of your email to a subset of subscribers and a different variation to another. The email that garners the highest number of clicks wins and can automatically launch to the remainder of your marketing list.
A/B testing allows you to try different subject lines, calls to action or combinations of the two. You can also add five email messages to a single test.
A/B Test Set Up
- Select Outbound in your navigation sidebar and click A/B Test Messages.
- You will see a new screen titled ‘A/B Email Tests.’ Click ‘New A/B Test.’
- Provide a name for your test along with descriptive text.
- Next, add a Distribution List(s) to your test. Click ‘Edit’ and select your list(s).
Click Here for a Demonstration
The Distribution List(s) section allows you to add a ‘Suppress Against’ list. Use this feature if you have a group of recipients that should not receive your A/B test email.
If you have questions about Distribution List(s), please contact your Marketing Automation Team.
- To add messages to your A/B test, click ‘Add Message.’
- A pop-up titled ‘Add Message to A/B Test List’ provides multiple ways to add messages to your test. Act-On allows you to add up to five email messages to a single test.
To test subject line variations, add a duplicate of the first message and click the ‘Edit’ pencil icon.
Your message will open in the Email Composer. Edit the Subject Line and click ‘Save.’ Your alternate subject line will now appear on the A/B Email Test screen.
- Enter your Sample Size. You have two options for the Sample Size value:
- Records: Total number of contacts added to your Sample Size. This value must be ten or higher.
- Percent of Distribution List: Percentage of your total list. This value must be 33% or less.
The A/B test will divide the Sample Size between all test messages.
- Records: Total number of contacts added to your Sample Size. This value must be ten or higher.
- Select a date, time, and time zone to schedule your initial A/B test.
- Select a Final Run option.
Act-On selects the email that garners the highest number of clicks, not opens, as the winner of the A/B Test. The platform can automatically launch the winning message to the remainder of your marketing list. Your Final Run selection determines the action taken.
The Wait option allows you to select a specific number of days and hours to rest before sending the winning message. Send On provides you the option to schedule a particular date and time. Choose Not Scheduled to return to the A/B Test later to send.
- Click ‘Save’ at the top of the A/B Email Test screen.
- Once an A/B Test has been set up and saved, you will need to schedule the test to run.
From the A/B Email Test screen, click ‘Schedule Test.’
Or click ‘Schedule Test’ from the A/B Email Tests screen.
Winning Messages
Once a test is complete, you can override Act-On's choice and choose a winning message. Open the test and hover over your desired message; click the box that appears. You must click 'Save' to commit your change.
Distribution List Issues
Your final send might be distributed to fewer contacts than your test if it included numerous suppressions.
To alleviate this issue, employ good list maintenance and hygiene practices. Removing hard bounces and opt-outs from lists and segments will reduce the total number of suppressions. If these measures do not remove distribution list issues, select a smaller sample size for your test.
Active Tests
The 'Active' tab on the A/B Email Tests screen allows you to view test statuses and access dashboards for running tests.
Canceling Tests
Act-On allows you to cancel a test from the 'Active' tab on the A/B Email Tests screen.
If the test has not run, the canceled test will return to the Drafts tab, where you can edit and relaunch it. You will receive a prompt to confirm the cancellation if the test has run. After you confirm, Act-On will cancel the final run, and the test will move to the Completed tab. You cannot undo this action.
If you do not wish to cancel the final run but only remove its scheduling, edit the test via the dashboard. Change Final Run to ‘Not Scheduled’ and then click ‘Save.’
Completed Tests
The Completed tab on the A/B Email Tests screen displays all completed tests and those canceled after Act-On sent the test run. Click a completed test to see a reporting dashboard.
When viewing the message report for the final send, contacts who received one of the test versions will show as suppressed for "Suppression List."
