Email A/B Testing: Top 6 Tests to Boost Your Campaign Results Today

When running an A/B test you generally want to focus on one variable at a time leaving all other elements unchanged (multivariate tests require a much deeper discussion). Several beginner elements to test for include:

• Email Subject Lines
• Email Content
• Send Time

Here are some results from A/B tests we’ve run in the past:

Subject line testing can have a significant impact on your campaign open rates. What’s more, the right subject line shapes the perception of the entire email. This can be done through creating a sense of urgency with your subject lines, like “one day left” or giving your subscribers exclusive offers.

In these tests, we ran subject lines that included the exact offer ($X off or X% off) vs. ones that did not include them.

Results
We took a look on a couple of our clients’ email campaigns, where the subject line had either a sense of urgency, exclusive offer, solving a problem (“For Email VIP’s – Get Our Black Friday Sale Early!”, “How To Style The Sidecar 3 Ways”) vs. a coupon code (“Get 10% Off All Protein Snacks”, “5% Off Plus Free Shipping”). In all cases offering the exact discount in the subject line lead to increase in revenue by 18.3% -100%, however, both the number of clicks and opens dropped on average by 20%-23%, if the subject line didn’t include the sense of urgency or other kinds of incentive. That’s why, as the third example testing suggests, the winning subject line is the one which combines an offer and a sense of urgency, yet is not overloaded (like “All-season flannels. 20% off for today only.”)

Interpretation
Even though the open rates actually seemed to drop when using the exact offer in the subject line, the subsequent revenue generated increases. By putting the exact offer front and center, it may dissuade those with lower buying intent, thus depressing opens, yet those who do open have a higher likelihood of actually purchasing.
Additionally, urgency is a strong catalyst, but it doesn’t stand up very well on its own. Adding urgency doesn’t work if your offer is full of distractions, or if your value proposition is poor. As a result, a sense of urgency and an attractive, clear offer work best together.

Questions make users think. More specifically, questions make them curious about your email, which can be powerful in getting open. At the same time, it is thought that effective email subject lines should have a direct call to action. So, we decided to run some A/B tests around these.

Results
The subject line in the form of a short, concise question is the winner. For example: “How can we help you next spring?” boosted Open Rate by 46.08% and Click Rate by 344.44% in comparison with “Take our survey & get $50 towards your next order”. A subject line without any CTA’s like “Is 3 months free groceries ok?” likewise got more opens than “3 months free groceries? GO!”.

Interpretation
While a question-form subject line may not always drive the highest click rates or ultimate conversions, it is useful in maximizing open rates. The curiosity generated by the question tends to do well in driving at least that initial open.

Promoting new products with email marketing can be highly effective, but it does take some work to learn how to do it well. Specifically, is it better to promote multiple products or one product in your unsegmented email campaigns?

Results
In the specific case, we tested, focusing on a single product can have a sizeable impact on metrics across the board. Of course, product selection is highly important in this case – but depending on your list and audience, showcasing singular, popular products can outperform a wider product showcase of multiple products in some cases.

Interpretation
Promoting one product helps to create an email with easy-on-the-eyes design and short, digestible content. It makes a clear message for the customer because it can grab the client’s attention to one single offer, whereas promoting multiple products can distract from a single, focused CTA.

How does pricing influence your email click-through? Is it a better idea to hide the pricing or use it as an anchor or even as a value proposition in email campaigns?

We wanted to find out the answer to these questions, so we run two A/B tests on showing vs. hiding product pricing and examined the impact.

Results
Surprisingly, in both tests, mentioning a price helped to increase the click rate on average by 10%. Moreover, for the second test, the revenue also increased revenue by 20%.

Interpretation
Placing the standard information we all use to make buying decisions (such as price) can actually stimulate your customers to purchase.

For example, one of the main reason why shoppers abandon their shopping carts is because they were presented with unexpected costs (56%). Customers don’t want to seem to like hidden costs, so mentioning prices upfront in your newsletter helps to shape the right perception of your products.

Email coding is a fine balance between including images and optimizing for images blocking. We went ahead and ran a test with several subscriber segments for one client to see if Image-Heavy or Text-Heavy emails performed better for them.

Results
We tested a regular email campaign on different groups of subscribers. Some segments saw a boost by 50% & 70% respectively for open & click rates with the text-heavy email. In contrast, for other segments the open rate for the text-heavy version was lower by -26.18%.

Interpretation
The test isn’t completely conclusive. In fact, as we saw – different segments even within the same email list can respond differently to the image-heavy vs. text-heavy versions.

Everyone wants to know, but few actually test it. So, what is the ideal send time? Unfortunately, there’s no single answer. Ideal sending times vary by industry, product, and your specific subscriber list. Here’s what has worked for in this particular case.

Results
Saturday mornings performed worse for engagement rates vs. Friday evening.

Interpretation
Saturdays are generally worse days to send. The other times and days of the week tends to vary for each store

Based on these A/B testing we’ve conducted, here are our suggestions on how to make your next email campaign really effective:

Need help with your email marketing? We have helped over 220 companies like yours!
Book a call with us and we’ll get to the bottom of it!