This first post is about IP address reputation with some background on why IPs are so important and why ISPs focus so heavily on the sending IP.

Why IP addresses?

ISPs built reputation around IP addresses because it was one bit of data that malicious senders / spammers couldn’t forge. The connecting IP is a fundamental part of the network transaction and if you forge an IP then SMTP can’t work. Because that was the reliable data they had to work with, that’s what they used. Even now, when there are other kinds of data, the IP address is still the first thing the receiving MTA sees.

What is IP reputation?

IP reputation can best be summed up as “past performance is an indicator of future results.” In other words if recipients responded well to mail from an IP address in the past, then they’re likely to respond well to new mail from that IP address.

How is IP reputation measured?

While each spam filtering company and ISP have their own ways of calculating the reputation of an IP address, there are some similarities in what they measure.

• How many non-existent email addresses is this IP attempting to deliver to?
• How many abandoned email addresses is this IP attempting to deliver to?
• How many “known bad” email addresses (spamtraps) is this IP attempting to deliver to?
• How many recipients complain about receiving this mail?
• How many recipients complain about not receiving this mail?
• How respectful of my resources is this IP?
• Does this IP keep connections open for long periods of time?
• Does this IP retry deliveries too aggressively?
• Does this IP stop mailing addresses after receiving a “user unknown” message?
• Is this IP address configured as if the associated machine was infected by a virus?
• Is this IP address listed on blocklists we use?

That is by no means an exhaustive list of what ISPs measure. If they can measure it they’ve tried. If the measurement helps them separate spam mail from not-spam mail then they’re using it.

How fast does IP reputation change?

IP reputation is often measured over multiple time periods. ISPs can look at a 1 day, 7 day, 30 day and 90 day reputation. A good analogy is stock prices. Prices can be very volatile in the short term, but more consistent over the long term. A single bad day, where one or more reputation measurements go bad, may affect delivery that day or the next day but won’t damage an overall good reputation. Likewise, a few days of improved mail may not be sufficient to counter months of poor reputation.

How is IP reputation used?

Mail from IPs with a high reputation is accepted faster and at a higher rate than mail from IPs with a lower or unknown reputation.  IP reputation can also influence whether mail is delivered to the inbox or the bulk folder.

Key IP Reputation takeaways

• IP reputation is about how recipients react to mail from that IP. Happy, content recipients turn into good delivery.
• Brief changes (for good or bad) don’t necessarily ruin delivery over the long term.
• Steady improvements will result in improved reputation.
• It may takes as much time to change a reputation in one direction or another as it took to establish the reputation in the first place.

Next we’ll look at content reputation, how it’s measured and used.

EDIT: A version of this information is available at the Word to the Wise wiki

EDIT: This post was also shared at CircleID

If any of you get the opportunity to go to a talk presented by any of the Return Path folks I encourage you to do so. They know their stuff and their presentations are full of good information.

One of the trends mentioned is the increase in reliance on domain reputation. It’s something I’ve been thinking about more and more recently. I wrote a little bit about it recently, but have focused more on the whole realm of content filtering rather than just domain reputation.

Domain reputation is where delivery is going. And I think a lot of senders are going to struggle with delivery as they find that IP reputation is not enough to get into the inbox.

One factor that is continually repeated is the size of the company. I’ve even seen a couple people suggest that corporate net worth should be included in the reputation calculation.

The problem with this suggestion is that just because a company is big or has a high net worth or is on the Fortune500 doesn’t mean that the mail they send isn’t spam. I’ve certainly received spam from large, name brand companies (and organizations). I’ve also consulted with a number of those companies who bought or appended a list and then had to deal with the fallout from a Spamhaus listing or upstream disconnection.

Sure, there is a certain logic to company size and prominence being a part of a reputation calculation. For instance, my experience suggests consumers who recognize a brand are less likely to treat mail as “spam” even if they didn’t sign up for the mail in the first place. Certainly there are large brands (Kraft, FTDDirect, 1-800-Flowers, OfficeDepot) that have been caught sending mail to people who never opted in to their lists.

Many people don’t realize that company size and prominence are already factored into the reputation scores. No ISPs don’t look at a mail and, if it’s authenticated, add in a little positive because it’s part of a giant, name brand company. Rather, the recipients change how they interact with the mail. Even recipients who didn’t sign for mail from Office Depot may click through and purchase from an offer. Some recipients recognizing the brand will hit delete instead of “this is spam.”

All of these things mean that big brands have recognition that takes into account that they are prominent brands. Elaborate processes and extra reputation points given to big brands don’t need to happen, they’re already an innate part of the system.

Over the years I’ve heard just about every excuse as to why a particular client can’t set expectations well. One of the most common is that no one does it. My experience this weekend at a PetSmart indicates otherwise.

As I was checking out I showed my loyalty card to the cashier. He ran it through the machine and then started talking about the program.

Cashier: Did you give us your email address when you signed up for the program?

Me: I’m not sure, probably not. I get a lot of email already.

Cashier: Well, if you do give us an email address associated with the card every purchase will trigger coupons sent to your email address. These aren’t random, they’re based on your purchase. So if you purchase cat stuff we won’t send you coupons for horse supplies.

I have to admit, I was impressed. PetSmart has email address processes that I recommend to clients on a regular basis. No, they’re not a client so I can’t directly take credit. But whoever runs their email program knows recipients are an important part of email delivery. They’re investing time and training into making sure their floor staff communicate what the email address will be used for, what the emails will offer and how often they’ll arrive.

It’s certainly possible PetSmart has the occasional email delivery problem despite this, but I expect they’re as close to 100% inbox delivery as anyone else out there.

We don’t get much spam on Yahoo, mostly because we don’t give our email address out much. The only spam we really get is from <stockbroker website>, and that all goes to the spam folder. We use the site for checking stock quotes – it’s free, and we never see any of the spam they send.

A typical email marketer would look at that and object loudly to her use of the “S word” to describe their email – it’s mail the subscriber signed up and gave permission for, and they have an ongoing relationship with the sender, and they haven’t unsubscribed, and, and, and…

But a delivery expert will point out that none of that matters one jot. Sure, the sender has a figleaf of permission, because they convinced the recipient to “subscribe to their mailings” (even if that was via the threat of withholding a free web service if they didn’t sign up). And that does provide some legal protection.

But as far as delivering email to recipients inboxes, let alone receiving any ROI for an email campaign, it’s pretty much irrelevant. The recipient perceives the mail as spam, and describes it as such to other people – “<stockbroker company> sends spam” is not the image you want to have. The subscriber doesn’t read the email, doesn’t want the email, certainly doesn’t pull it out of the spam folder and may well be hitting the “this is spam” button for messages that end up in the inbox.

You’re certainly not getting any benefit at all from that subscriber, and their relationship to the mail you’re sending them – not opening or interacting with it, categorizing it as spam, etc – is teaching their ISPs spam filters that your mail is unwanted spam. The reputation of your domain, your content and your sending IP addresses will suffer, and your delivery rates to all your subscribers will suffer.

If you’re forcing someone to give you permission, it’s not permission-based marketing.

The last two weeks the culprit has been Yahoo. They seem to be making a lot of changes to their filtering schemes right at the busiest email marketing time of the year. Senders are increasing their volume trying to extract that last little bit of cash out of holiday shoppers, but they’re seeing unpredictable delivery results. What worked to get mail into the inbox a month ago isn’t working, or isn’t working as well, now.

Some of this could be holiday volume related. Many marketers have drastically increased their mail volume over the last few weeks. But I don’t think the whole issue is simply that there is more email marketing flowing into our mailboxes.

As I’ve been talking with folks, I have started to see a pattern and have some ideas of what may be happening. It seems a lot of the issue revolves around bulk foldering. Getting mail accepted by the MXs seems to be no different than it has been. The change seems to be based on the reputation of the URLs and domains in the email.

Have a domain with a poor reputation? Bulk. Have a URL seen in mail people aren’t interested in? Bulk. Have a URL pointing to a website with problematic content? Bulk.

In the past IPs that were whitelisted or had very good reputations could improve delivery of email with neutral or even borderline poor reputations. It seems that is no longer an effect senders can rely on. It may even be that Yahoo, and other ISPs, are going to start splitting IP reputation from content reputation. IP reputation is critical for getting mail in the door, and without a good IP reputation you’ll see slow delivery. But once the mail has been accepted, there’s a whole other level of filtering, most of it on the content and generally unaffected by the IP reputation.

I don’t think the changes are going to go away any time soon. I think they may be refined, but I do think that reputation on email content (particularly domains and URLs and target IP addresses) is going to play a bigger and bigger role in email delivery.

What, specifically, is going to happen at Yahoo? Only they can tell you and I’m not sure I have enough of a feel for the pattern to speculate about the future. I do think that it’s going to take a few weeks for things to settle down and be consistent enough that we can start to poke the black box and map how it works.

Certainly, the “This is not spam” action – moving an email from the spam folder to the inbox, or clicking the “not spam” button in a web mail’s interface, is a strong positive reputational action. Some webmail providers use this data to decide which bulked senders deserve being let out of the penalty box – which should have their mail once again delivered to the inbox.

A client recently theorized that a great solution to their delivery problems would be to do this “en masse.” Sign up for hundreds or thousands of webmail accounts, send my mail to them, and click on the ”not spam” button for each of my own emails. That’ll greatly improve my sending reputation, right?

NO! ISPs have already thought of this. They watch for this. They’re really good at picking up on things like this. I know for a fact that Yahoo and Hotmail and AOL notice stuff like this, and I strongly suspect other webmail providers notice it as well.

What happens when Yahoo or Hotmail pick up on this type of unwanted activity? Well, if it’s at Yahoo, they’re likely to block all mail from you, 100%, forever. I’ve seen it happen more than once. Yahoo might even identify all of your netblocks, ones beyond the ones sending today’s mail or originating today’s activity. And good luck trying to convince them that you’re not a spammer – you have a better chance of winning the lottery two weeks in a row.

As for Hotmail – what would Hotmail do? Ask Boris Mizhen. Microsoft is currently suing him, alleging that he and/or his agents or associates engaged in this very practice.

1. Sender Score – http://senderscore.org/. Provided by Return Path. They collect data from some ISPs and blocklists. Using a proprietary formula, they calculate a sender score running from 1 – 100 for each IP address sending mail to their network. Higher scores means a better reputation. Can be inaccurate for IPs sending very low volumes of email. Some ISPs use Sender Score to feed into their delivery decision engines.
2. Sender Base – http://senderbase.org/. Provided by Ironport / Cisco. They collect publicly available data as well as data from their userbase. Reputation is reported as “good” “poor” or “neutral.” Senderbase scores feed into some ISP delivery decision engines.
3. AOL reputation – http://postmaster.aol.com/cgi-bin/plugh/check_ip.pl. Reports the reputation of IPs as determined by AOL. Uses a scale of “good” “poor” or “neutral”.
4. RoadRunner blocks – http://security.rr.com/amIBlockedByRR. Reports if a particular IP address is currently being blocked from sending mail to Road Runner.
5. Spamhaus blocks – http://www.spamhaus.org/. Reports if an IP is currently listed on any of the Spamhaus lists.
6. Sendmail Reputation – http://sendmail.com/sm/resources/tools/ip_reputation/. Reports reputation of an IP address as measured by Sendmail.
7. Trusted Source – http://www.trustedsource.org/. Provided by McAfee.
8. Commtouch – http://www.commtouch.com/check-ip-reputation/. Provided by Commtouch.
9. Barracuda Central – http://www.barracudacentral.org/lookups/. Provided by Barracuda, shows what IP addresses or domain names are currently blocked.
10. SNDS – http://postmaster.live.com/snds/. Provided by Microsoft / Hotmail / Live.com. Will show IP addresses that are currently blocked by Microsoft.

All this means is that there are a limited number of public sources of reputation data. Some ISPs provide ways that senders can check reputation at that ISP. But if a sender wants to check a broader reputation across multiple ISPs where can they go?

There are multiple public sources of data that I use to check reputation of client IP addresses.

Blocklists provide negative reputation data for IP addresses and domain names. There are a wide range of blocklists with differing listing criteria and different levels of trust in the industry. Generally the more widely used a list the more accurate and relevant it is. Generally I check the Spamhaus lists and URIBL/SURBL when investigating a client. I find these lists are good sources for discovering real issues or problems.

For an overall view into the reputation of an IP address, both positive and negative, I check with senderbase.org provided by Ironport and senderscore.org provided by ReturnPath.

All reputation sources have limitations. The primary limitation is they are only as good as their source data, and their source data is kept confidential. Another major limitation is reputation sources are only as good as the reputation of the maintainer. If the maintainer doesn’t behave with integrity then there is no reason for me to trust their data.

I use a number of criteria to evaluate reputation providers.

1. Personal history. I’ve been around this industry longer than most of the reputation sources. I’ve watched them develop, modify and adapt their policies over time.
2. How widely is the data used? This is particularly important when using blocklists. Lists that are widely used tend to have a more accurate listing policies as most large ISPs don’t want a lot of non-spam blocked.
3. How transparent is the provider? As I mentioned above there are good reasons for all reputation providers to keep their methods and data sources close to the vest. But that need for secrecy has to be balanced with enough transparency that their data and information can be verified by outside sources.
4. How do representatives handle themselves with listees and in public? Are they helpful or disdainful? Do they value the enduser and want the recipient to receive mail they want? The behaviour of employees and volunteers in public and with listees reflects the underlying values of the list and can be a major indicator of the value of the list.

Public sources of reputation data can be useful but not every source is created equally. When deciding what sources to listen to you need to evaluate the reputation of the source entity, too.

When you’re using machines in the cloud, changing IP addresses is as simple as initializing a new server. Spammers discovered this almost as soon as the EC2 cloud became public. They would set up a mailserver and send spam through that server until it was blocked. Then they’d just start another instance to avoid the block and keep spamming. They had an almost unlimited number of IP addresses to abuse and moving around was easy to do. Amazon did little to stop the spam coming from the cloud so many ISPs and spam filtering companies blocked email from the entire range of IP addresses allocated to the EC2 cloud.

Blocking large swathes of network space that are consistent sources of abuse is well accepted as a method of dealing with spam. Yes, this form of blocking has inconvenienced legitimate companies who aren’t actually doing anything wrong. But when a service provider doesn’t take sufficient action to stop customers from spamming through their networks, then ISPs will implement countermeasures.