Silverpop how-to: combining A/B tests with Send Time Optimization

jaredThis week, we welcome a guest post from one of our Web Marketing Managers, Jared Ruppert. Jared is Silverpop Certified and in charge of designing and optimizing email campaigns for many Whereoware clients. Here, he explains the benefits of Silverpop’s A/B testing and Send Time Optimization (STO) features…and shares a nifty little trick for making the two work together!

Want to increase open rates and ultimately boost your bottom line? Silverpop’s A/B testing and Send Time Optimization (STO) features are simple to use and can drastically improve your email performance. Silverpop’s A/B testing feature makes it easy to test things like “from name” and subject lines. The platform also offers automated Send Time Optimization (STO), which allows you to send your message to contacts at the precise day and time they are most likely to check messages (identified based on past behavior).

Sounds great, right? Easy to use, capable of improving your ROI…it’s a no brainer! However, there’s a drawback – currently, if you want to run an A/B test and use STO, you have to run the test for a full 25 hours before the remainder mailing sends out. Whether you’re promoting new products or advertising a sale, very few marketers have time for this kind of delay. You’re left with the decision to run one test or the other. So, which is more important? Finding the optimal subject line? Or sending at the ideal time? Both could make a huge impact on revenue, so the decision is a difficult one.

What if you don’t want to have to choose? After all, it’s important to send out the best email you can every time. You’re in luck! Even though Silverpop doesn’t allow you to run A/B testing and STO together, there’s a way to get around this. We have a solution that allows you to test for a couple of hours, and then send with STO in less than 24 hours. Let’s take a closer look.

How to

To truly get the best of both worlds, you’re going to run both tests in much the same way you would when running them independently, but you’ll make a few changes along the way.

  1. Run your normal A/B test. Note: make sure to send the winning subject line report to your email address (rather than automatically send winning subject line to remaining contacts).
  1. After your testing period (in this example, 4 hours), Silverpop will send you an email with the winning subject line.
  2. Continue on to “A/B status page” for the next step.

Typically, this is where you would conclude your A/B test by clicking “Send Winning Subject Line to Remainder.” However, doing so would not incorporate Send Time Optimization, which is our goal in this experiment. In order to use STO with the remaining database, you must cancel the A/B test.


  1. After you’ve cancelled the test, return to your original mailing, and update the subject line to reflect the winning subject line. Continue by pressing “Send > Send Mailing”

In order to send to the remaining database (without re-sending to the tested population), you must select a specific contact source. Silverpop automatically generates a contact list that contains the remaining database for every A/B test. To access this:

  1. Press Select
  2. Choose Queries > AB Test Shared Remainder > Select [Mailing Name remainder]
  1. Make sure to select “Use STO”
  2. Click Send.

Congratulations! You’ve now used two of Silverpop’s most powerful features to optimize your mailing, and even combined them into a more sophisticated experiment to ensure that you’re getting the most bang for your buck. By continually optimizing in this manner, you’ll be able to ensure that your emails just keep getting better.