Know Your Numbers: How to Measure Email Deliverability and Reputation

Before you can improve your email deliverability, you need a way to accurately measure it.

There are several methods and tools for measuring email deliverability and reputation. These include:

• Inbox Placement Tests
• Delivery Rates
• DMARC Reports
• Postmaster Reports
• Relative Inbox Open Rates
• SenderScore
• Content Spam Detectors

Inbox Placement Tests
One of the best methods to measure deliverability rates is using inbox placement tests. These can be done manually or utilizing automated software services.

In essence, they work by:

1. Create a Seedlist of Email Addresses: This is the list of controlled email addresses that you'll later send email messages to. The seedlist typically consists of a sample of inbox domains from providers such as Gmail, Yahoo, AOL, etc. Seedlists can be upwards of several hundred such email addresses.

2. Send to the Seedlist: Once the seedlist is created or generated, the next step is to send a live email message to those email addresses. It's important that the email is a live one and not a preview, because often times preview emails use different infrastructure to send and will cause inaccurate results. It's also best to send the email message to your normal subscribers, with the seedlist simple attached as an addition. This way you'll get the most accurate readings.

3. Analyze the Inbox Placement Results: Once the email message gets received by the individual emails on the seedlist, the results are tallied up and graphically represented. This gives you a good sense of what percentage of the seedlist email address on each domain saw your email message land where, whether in the inbox, spam folder, or missing completely.

A typical report may look like:

While inbox placement tests are one of the more accurate ways of measuring email deliverability rates, they are not perfect. Seedlist email addresses typically do not open, click, or otherwise interact with the received email messages. Since many inboxes based future delivery on past user behavior, this could cause otherwise deliverable messages to be sent to spam.

Delivery Rates
Many email platforms will display a metric called email delivery rate. The terminology "delivery rate" is somewhat misleading though. On first glance, you may think it means the percentage of your emails sent that actually reach the inbox. This is not true.

The email delivery rate calculation is the volume of non-bouncing emails divided by the volume of emails sent. Bounces are basically messages that get rejected by the inbox provider because of issues like the email address being invalid, the mailbox is full, temporary or permanent server issues, etc.

Since bounces are often a small fraction of email sends, the reported delivery rate is usually very high, in the neighborhood of 95%-99%. However, this is very different from actual inboxing or deliverability rate, because the delivery rate doesn't account for where your email messages land. So even with a 95%+ delivery rate, a large portion of your email messages could still be landing in the spam folder instead of the inbox.

Still, delivery rate is one data point that can be useful, especially in the case when it's too low, because that means a large portion of your email messages is bouncing, which is definitely indicative of a problem.

DMARC Reports
Every email sender should have a proper DMARC policy setup. Not only does this help control authentication from malicious actors, it also generates useful aggregate reports that shed light on your SPF and DKIM pass/fails across multiple sending IPs.

Reviewing these reports consistently is another great way to measure your deliverability, as the details in the report help you see which email messages from which platforms are fully/partially passing or entirely failing DMARC authentication protocols.

Postmaster Reports
Some inbox providers, such as Gmail, allow senders to setup a Postmaster account to access further information about their email sender reputation, sending domain health, message volume, and feedback loop around user spam complaints.

This is an additional layer of data that helps round out the deliverability measurement profile, so we definitely recommend getting such accounts setup and sending properties verified. For smaller senders with low volume though, the data may not display at all, as a certain minimum volume threshold is necessary before reports are generated.

Relative Inbox Open Rates
One more indirect measure of deliverability is to compare the open rates by inbox provider in your email campaign reports.

This method isn't the most accurate, especially with Apple Privacy and other open pixel skewing elements, but it can give you a quick view of if anything seems particularly off.

For example, let's say you are seeing something like this:

In this case, we can see that Gmail matches closely to the expected open rate averages. Apple mail is significantly higher, but that's probably due to the skewed Apple Privacy settings.

Yahoo and AOL are significantly below the average open rate, so would be red flags of possible deliverability issues with these two inbox providers. You'll also want to reference the volume of contacts on each inbox, as small sample sizes could be biased.

SenderScore
Third party deliverability platforms can also provide unique data points around your sender reputation. We like to use Validity's SenderScore metric as another measure of email deliverability.

The SenderScore is evaluated on a sender IP level, so if you are running from a shared IP pool, you'll likely want to run all of the usual set of IPs through the tool to get a more holistic view of the score distributions.

Content Spam Detectors
In addition to all the previous measures of deliverability post-send, one way to measure potential deliverability pre-send is to run the content of your email campaigns through spam detectors to identify any issues.

For example, when we run our pre-flight deliverability checklist in Email on Acid, the result looks something like this:

The resulting descriptions can help you adjust your email content to better comply with popular spam filters like Barracuda and Spam Assassin.

Summary
We've gone through multiple ways to measure email deliverability, but measuring is just the start. Once you have a benchmark of your current deliverability and reputation, you'll need to know which deliverability factors to adjust in order to optimize your deliverability rates. You may also find that you have specific deliverability issues, which you'll need to fix.

If you want to work with us instead to audit, troubleshoot, and monitor your deliverability, we have a specialized email deliverability service exactly for this.