Is their increased vigilance pissing you off? If so, your anger is misplaced. They are reacting quite sensibly to market conditions apparently imposed by Spamhaus. Ken Magill

While I agree with Ken that the ESPs are reacting to market conditions. Where we disagree is the idea that these conditions are imposed by Spamhaus. I don’t think all the uptick in ESP enforcement and compliance activity is the result of Spamhaus’ actions. I believe that many of the mass market ISPs are changing how they detect unwanted mail, and are fine tuning filters to reduce the amount of unwanted mail that shows up in the inbox.

One of the big changes is better tools for handling huge data sets. Bigger ISPs handle billions of messages a week. Even just collecting and storing the mail is a giant task. Storing it in a useable form was almost out of the question. But over the last few years there have been significant improvements in the speed and affordability of hardware to handle very, very large datasets. Likewise, there have been algorithm and software improvements in mining that data for useful correlations.

In practical terms, ISPs and filtering companies like Spamhaus don’t have to focus on complaints or trap hits or “simple” measurements. They can draw complex correlations and look at mail in a way that was simply impossible 2 or 3 years ago. This means they can better identify senders who had previously been able to slide in under the filters.

Spamhaus rolled out tools to monitor their spam feeds in a different way and have been listing a lot more “legitimate” senders because of it. ISPs are rolling out tools to better filter “greymail” and keep users inboxes full of mail that the users actually want.

One of the trends I’m noticing is that direct marketers are getting more aggressive. Whether it’s a response to the years of recession or a response to the slowly warming economy, I can’t tell. But there are a lot of direct marketers who are no longer afraid to break the law. For instance, my cell phone is getting multiple telemarketing calls a week, despite being a cell and despite being on the do not call list. My inbox is full of unsolicited email carefully engineered to get past standard filters, much of which violates CAN SPAM. I’m even getting the occasional unsolicited fax.

The increase in listings by Spamhaus are one example of the filtering screws being tightened. But it’s not just Spamhaus that’s driving this; ISPs and filtering companies are also filtering more aggressively. I’m seeing a lot more emphasis being placed on content and a good IP reputation is no longer a ticket to the inbox. Content must be clean and recipients have to want mail for it to get into the inbox.

For instance in many areas of marketing repetition is key. Repeat a slogan and forge an association between the slogan and the product in the mind of the consumer. More repetition is better. Marketers can even go so far as using the same ad to drive consumer action. Television advertising is a prime example of this. Companies don’t create new content for every advertising slot, they create one or a few ads and then replay them over and over. The advertiser doesn’t even really care if the consumer consciously ignores the ads. The unconscious connection is still being made.

In the world of email delivery, though, having many or most recipients ignore advertising is the kiss of death. Too many unengaged users and filters decide that mail shouldn’t go into the inbox. These don’t even have to be ISP level filters, but Bayesian filters built into desktop mail clients.

Sending repetitive ads over email may be an effective marketing strategy, but may not be an effective delivery strategy.

Am I off base here and missing something? Tell me I’m wrong in the comments.

My bayesian filter gets trained mostly by me hitting “this is spam” when spam makes it to my inbox. If I’m expecting an email “immediately” – something like a mailing list COI confirmation or email as part of buying something online – I’ll check my spam filter and move the mail to my inbox in the rare case it ended up there. Other than that I let it and spamassassin chug along with no tweaking.

I’m starting a data analysis project, based on my own inboxes, and as part of that I’m using some tools to look for false positives in my junk folders, and manually fixing anything that’s misclassified. I’ve been doing this for a couple of hours now, and I’ve found some interesting things.

1. Simple content filters work remarkably well out of the box, at least for my mail stream. Spectacularly well. There’s very little in the way of false positives. Very, very little.
2. Of those false positives there’s nothing I’d have been bothered about. It’s generic, unexciting junk mail.
3. Most of the systemic false positives seem to be correlated with the senders doing something bad. Heathrow Express, for instance, sent me mail every two weeks or so since I’d signed up. Then for no obvious reason they stopped sending for three months, then started sending again. Every mail they sent after that pause ended up in the junk folder, and I never missed them.
4. I get regular newsletters from ThinkGeek. Every one of those goes to the inbox. I occasionally get mails from them about my account (“you’ve got 420 geek points left”) that are kinda transactional, but not something I expect to see – and they all end up in the junk folder. Several other senders do the same thing, and get the same result.
5. Several companies have used tagged addresses to send me newsletters for a while, and also used them to send unsolicited facebook invites (to the tagged address, from facebook servers). The regular newsletters all go to the inbox, while the facebook invites all go to the junk folder. “Legitimate” facebook mail, meanwhile, keeps going to the inbox.
6. Apple send me a lot of newsletters – I’m a Mac and iPhone developer, I get their consumer newsletters, transactional stuff from our local store – lots and lots of newsletters. They all made it to the inbox except for one. The one that ended up in the junk folder was a one-off about recycling, and it wasn’t up to their usual design standards – it had ugly big green “call to action” headlines in it, very different to their usual clean design.
7. Just one sender hit the junk folder every time. The distinctive thing about their messages (apart from them not being something I missed) was that the plain text part of them was dreadful, just a bad lynx dump of the html section. Even a “No plain text for you! Go to this link!” would have been better.

I was surprised at how effective this simple content-based filtering setup had worked with little tuning other than hitting the this-is-spam button – both in it’s accuracy at removing spam while keeping a very low false positive rate, but also how well the false positives matched my judgement of “Meh. This mail isn’t interesting.”.

We spend a lot of time talking about the things you should do to make the mail relevant to the recipient – compelling content, consistent, predictable delivery schedules, clear consistent branding, use of a single consistent mail stream to communicate with a recipient rather than several different streams. Until I went through this exercise it wasn’t clear to me how much of an effect those things also have on fairly simple recipient-trained filters.

The most effective way to get mail delivered to the inbox, however, is for recipients to go into the bulk folder and mark the mail as “not spam.” Nothing is more effective at getting mail delivered to the inbox.

But there is a bit of a catch 22 there. If mail ends up in the bulk folder consistently, recipients tend to forget about it. Many people trawl through their bulk folder sporadically, if at all. If recipients aren’t engaged with mail and don’t know when they should see it, then they won’t miss it and won’t look for it.

So if mail is ending up in the bulk folder and recipients aren’t expecting it what can a sender do? One of the obvious answers is find another channel. Let recipients know through some channel besides email that they need to look in their bulk folder for a particular email.

In the past it was difficult to find non-email ways to connect recipients. I worked with customers who really had no other way to interact with recipients than email. They weren’t running a website, they didn’t have any other contact methods, they were really stuck. But a recent tweet from AppSumo shows how social media can be used to improve email delivery.

Using social media to improve mail delivery

Of course, not every AppSumo recipient follows them on twitter. But every recipient that does pull the mail out of their bulk folder tells Gmail that the mail really is wanted. Most ISPs heavily weight users removing the mail from their bulk folder. ISPs know that users who look for mail in the bulk folder really are engaged and really do want that mail.

Using social media is a great way to contact recipients through another channel to address delivery problems. Even just a few people pulling the mail out of the bulk folder will tell the ISP the mail is wanted. Plus, those people will now find that mail delivered to their inbox by default, no matter what happens with other users.

This is great, for the user. Take the common example where a commercial list has some highly engaged recipients and a bunch of recipients that can take or leave the mail. The ISP delivers the newsletter into the inbox of the highly engaged recipients and leaves it in the bulk folder of less engaged recipients.

With user focused delivery people get the mail they are interested in where they can read it and interact it. People who have demonstrated a lack of interest for a topic or a sender don’t see that mail.

This can get complicated for those of us trying to troubleshoot deliver problems, though. I have a couple mail accounts I use for testing at various ISPs. Even though I do very little to try and personalize the account I am seeing behaviour that leads me to wonder if ISP personalizing the inbox experience is going to make it that much more difficult to troubleshoot delivery issues.

I have to wonder, too, where this leaves delivery monitoring services in the future. If delivery is personalized, how can you know that the delivery monitoring addresses are representative any longer? Is there even a “representative” mailbox any longer?

But…

Most of what I do is simply think about email delivery from the point of view of a recipient and help clients better meet their recipient’s expectations. This works. This works really well. If you send mail that your recipients want your mail gets to the inbox.

Here’s the secret: ISPs and most spam filters have a design goal to deliver mail their users want. They only want to block mail their users don’t want.

Filters are not designed to block wanted mail.

Sure there are complicated situations where senders have gotten behind the 8 ball and need some help cleaning up. There are situations where filters screw up and block mail they shouldn’t (and aren’t quite designed to). Spam filters are complicated bits of code and sometimes they do things unexpectedly. All of these things do happen.

But these situations happen a lot less than most senders think. Most of the time when mail is hitting the bulk folder, or is throttled at the MTA the issue is that recipients don’t care about the mail.

Recipients aren’t engaged with a particular sender or particular brand. So ISPs react accordingly and that mail ends up slowly delivered or bulked. This upsets the senders to no end, but the recipients? The recipients often don’t care that some mail shows up in bulk or arrives Wednesday afternoon instead of Tuesday evening.

When recipients are engaged with a particular sender or brand, though? Delivery is fast and reliable. Mail is rarely delayed or bulked. When recipients want mail, they interact with it. They look in the bulk folder. They miss it when it’s not there. They complain to the ISPs when they don’t get it. The ISPs react accordingly and prioritize or “red carpet” that email.

The secret to really good delivery is to get your recipients to handle your ISP relations for you. Send mail they miss when they don’t get it, and you’ll discover most of your delivery problems go away.

They also point out that there isn’t a clear delivery hit to leaving those abandoned addresses on their list. It’s not like bounces or complaints. There isn’t a clear way to measure the dead addresses and even if you could there aren’t clear threshold guidelines published by the ISPs.

Nevertheless, I am seeing more and more data that convinces me the ISPs do care about companies sending mail that users never open or never read or never do anything with.

The most recent confirmation was the announcement that Hotmail was deploying more tools to help users manage “greymail.” I briefly mentioned the announcement last week. Hotmail has their own blog post up about the changes.

It seems my initial claim that these changes this won’t affect delivery may have been premature. In fact, these changes are all about making it easier for Hotmail users to deal with the onslaught of legitimate but unwanted mail.

[W]e realized that getting rid of true spam wasn’t enough, because 75% of the email messages that people reported as spam are really legitimate newsletters, offers, or notifications that you just don’t want anymore. We call this type of unwanted email graymail, and we’re excited to announce five powerful tools to help you take control of your inbox, get rid of graymail, and keep track of the email that’s important to you.

75% of mail reported as spam by Hotmail users is mail that Hotmail defines as legitimate mail. But they understand that it doesn’t matter that this is legitimate, that it’s opt-in, that it’s not spam. It’s still mail their users don’t want.

One of the reasons it’s a problem is that it clutters the inbox. Inbox clutter is a huge problem, and many of us have filters to try and keep marketing mail out of our inbox (I have boxes labeled “newsletter” and “commercial” and “lists” to keep bulk mail out of my inbox.

The good news is that Hotmail is trying to make it easier for their users to unsubscribe from your newsletters.

Click on unsubscribe, and we’ll do the rest – let the site know to stop mailing you, use Sweep to immediately clean up your mail and remove all the old newsletters from that sender, and finally send any new ones that come in to your junk mail until the sender takes you off their list.

In the past, Microsoft has used the List-Unsubscribe header to offer unsubscribe options to recipients. This functionality disappeared in a previous update. Hopefully, this blog post signals the eventual return of this functionality.

In any case, these tools are clearly designed to make it easier for the Hotmail user to organize and sort their mailboxes. It means that senders are going to have to work harder to engage recipients and make offers that much more appealing. Otherwise, your mail may never get read.

If you haven’t mailed an address in 5-10 yrs, would you include it in a re-engagement mail?

A number of people responded that addresses that old should not be mailed. I think the answer is more complex than can be handled in 140 characters.

Five to ten years is a very long time. Think about what you were doing 10 years ago. It’s easy right now, 10 years ago as a nation we were still reeling from the September 11 attacks. On a more personal note, Steve and I were just making the decision to start Word to the Wise. But what about 5 years ago? I can’t remember what we were doing or what our business goals and limitations were.

If you’re going to mail addresses that were collected 5 or 10 years ago, you must give some thought to a number of questions.

1. How has my target market changed in the last 5 – 10 years? How likely is it that customers from then would be interested in my products now?

People grow and change. As we move through different life stages, we have different needs and shop for different products. When thinking about whether or not to send mail to those old addresses, think about customer demographics. Is someone who wanted your product in the past also going to want your product now? What life stages are you targeting?

If you can honestly say that your product has a 10+ year target market, then mailing old customers may be acceptable. But if you focus on a narrow demographic it’s possible that your former customers are no longer interested in anything you have to offer, no matter how compelling the copy.

2. What do I have to offer a customer from 5 – 10 years ago? Is my current product line likely to interest them?

Just as people grow and change, businesses grow and change as well. When we first started Word to the Wise a lot of my consulting was directed at senders who were having blocklist problems and often didn’t have permission to send the mail they were sending. We didn’t have to talk about bulk folders, as most major ISPs hadn’t adopted the bulk folder yet. We didn’t have to talk about Feedback loops or “this is spam” buttons because such things didn’t exist yet. They primarily wanted to know how I could help them get and stay off the RBL or SBL.  In contrast, most of my current customers are opt-in senders who want information about how to engage users and get a better responses to their email.

Sure, old customers may be interested in new products and re-establishing contact with an old vendor. Others may have no interest at all. Some small percentage having an interest in your product isn’t sufficient. You need to be sure that a large percentage of recipients are going to want your new product.

3. How long does my product last? Are older customers still interacting with my product? Or have they forgotten I even existed?

There are pieces of software I’m using from 5 or 10 years ago. I’d be fine with a re-engagement email letting me know about other offers they have. But there are also bits of software I downloaded, tried and promptly forgot. I’d be annoyed if the vendor tried to email me. That really nifty pepper mill we bought 6 years ago? Love to hear from them about new stuff. That random kitchen gadget gathering dust in the back of a drawer? Not so much.

So much of making decisions about email is gauging how receptive recipients are to your message. When trying to decide to email very old customers, it’s important to understand your previous customer base.

4. What value am I bringing to the recipient? Do I have something new to offer? Can I push a new product or new launch?

The core of email deliverability is sending mail that your recipients want to receive. If you’re contacting recipients that haven’t heard from you in years, you need to put extra effort into making the email relevant for their lives. One of the ways you can do that is to share your excitement with a new product line, or a re-brand of your company.

Another way to make the email relevant is to make the email informative. Talk to the recipient about how you’ve changed in the intervening years and how your products can help the recipient. Your old customers are more likely to accept your intrusion if you have useful information for them  with your old customers

5. Where did I get these email addresses? Do I have a good audit trail for them?

This is where we get to those pesky details. Do you actually know where the addresses came from? Do you have even a partial audit trail. Can you tell what product was bought by the address? Do you know when the address was entered into your database? Do you even know if these are addresses of customers or not?

In my experience, most companies don’t have good audit trails for older addresses. They don’t know where the addresses came from. They don’t know if they’re actual customers. These are the things that cause re-engagement to fail totally.

You should NEVER mail old addresses unless you can identify where the address came from and the specific purchase that address is associated with. If you don’t have that data, then your delivery is going to be awful. You can only aspire to get into the bulk folder. More likely, you’re going to end up with mail blocked at many ISPs.

For the sake of argument, let’s say you do have that data. Someone at your company set up a database that captured everything you may need to mail old customers.

It’s not enough to have the audit data, you should take a deep dive into the data itself. How many of the addresses are at any of the dozens of domains that have retired in the last 10 years? How many are @home.com, @attbi.com, homestead.com or mcimail.com? None of these domains exist any longer. How many are @compuserv.com, @prodigy.net or earthlink.net? These are domains that were popular long ago, but are no longer in wide use. It’s unlikely your customer still has that address.

Still thinking about mailing that list, because it’s mostly @aol.com or @hotmail.com addresses? That may still risk your delivery. Old addresses at major domains are often turned into spamtraps and mailing these addresses may result in blocking.

Statistics show that 30% of email addresses are abandoned by their owners in a year. That means that even 5 years back only about 20% of those addresses are still in use by your customers. The others are abandoned, turned into spamtraps or just won’t deliver. If 80% of your list goes into a black hole, how much does each sale have to be to make it profitable to contact those old customers?

Each question should take an average business quite a bit of time to answer. The first 3 questions are about the intersection between you and your customer. They’re about you, the business, honestly evaluating your product (then and now), your target market (then and now) and the chance that you will meet their needs now as you met them then. The fourth question is about what you want to tell your old customers. But none of those questions are even worth asking unless you know you have a database worth sending to. And even if you do, will the ROI on a mailing be enough to justify the expense to put together an effective re-engagement campaign?

As a past guest and/or meeting planner of Millennium Hotels and Resorts we are pleased to share these occasional special offers. If you no longer wish to receive email communications from us, please click the unsubscribe link. Please note that this broadcast is sent from an address which is not monitored. If you have questions about the offer, please contact us directly. Our hotel contact details may be found in this email offer above or you may visit www.millenniumhotels.com.

This is the footer of an email sent by a hotel I stayed at sometime in the past. I thought it was a year ago, but I checked with the groom of that particular wedding party and it seems it was 2 years ago. My, how time flies.

But where is the relevancy to me? I went to the hotel as a part of a block carved out for the above mentioned wedding. It was two years ago and I’ve not been back to that city since. I can understand that there are long periods of time between visits to a city. And I can understand that sending emails to people who haven’t stayed in your hotel in 2 years might make good marketing sense.

What I can’t understand is why this hotel thought that this particular deal might be of interest to me. Remember, I visited said hotel as part of a wedding party. The hotel knows I came just for a wedding. So a good “come back” message might be to visit my friends and take them out for dinner to remember their wedding day. Or something slightly relevant to what I did the last time I was in that city.

What is clearly irrelevant is an offer to come visit the hotel and get tickets to see the local baseball team play. Wait? What? Baseball? Baseball in a city 2 timezones east of me? Really? They know where I live, they could have even made it slightly more relevant by offering me tickets to see one of my local teams play their team. Sadly, no, they didn’t even do that.

Email recipients want mail that speaks to them. That brings them an offer or an idea or information. That makes their lives easier or better or happier. What they don’t need is irrelevant junk clogging up their inbox.

What senders want is email that persuades recipients to buy something.

In this case Millennium hotels gave me fodder for a blog post, which does make today a little easier. But I also unsubscribed from future emails because it’s unlikely I’m going to return to that particular city any time soon. And if I do I’ll probably stay with friends and not in a hotel.

What did Millennium hotels get out of this? Not a whole lot, other than a lot of unsubscribes and probably a number of complaints. Oh and a blog post talking about how badly they ran this particular marketing campaign.

It’d be nice to have a tool to uncover the zombie email addys, but until then, read this from @wise_laura: http://bit.ly/jxjZ9M Kelly Lorenz

There’s currently not a programmatic way to make those determinations, but this is where the relationship between senders and receivers comes in. All you have to do is ask recipients what they want. I know, I know, most marketers have a low opinion of the ability or desire of their recipients to respond to an email. And there are some actual challenges to getting a recipient’s attention when they’ve tuned out a particular sender or type of email.

Separating out zombie addresses from real addresses is a challenge. But it’s part of maintaining a healthy email marketing program. When I’m working with clients who are mailing old lists we first talk about their product and their goals. Then we look at what information they have for recipients. My goal is to separate out those addresses we know are good from those we know are bad from those we don’t know. It’s the ones in the don’t know category that we focus on.

I don’t recommend just deleting the addresses that are non-responsive. One of the reasons is that I read mail with images off by default and so I often look like one of those non-responsive recipients even when I’m actively reading and receiving mail. Instead, I like to look at what other data the customer has about a recipient to help classify them. Of course, there are often a subset of users we cannot identify if they’re zombies or not. For these addresses we plan a re-engagement campaign.

Clients who have gone through this process have seen an improvement in delivery and an improvement in responses and ROI.
Until and unless ISPs start rejecting mail for zombie accounts it’s the best chance senders have to separate the barely there from the not there at all.