Well, that got someone’s attention. The person managing their twitter account tweeted at me with an email address and a suggestion to send him my address so he could take care of it. I sent the mail as asked and even got a reply.

Unfortunately, the reply was “I clicked the unsubscribe link at the bottom of the message for you.”

I dunno, maybe his mouse is a magic mouse and, somehow, the click from that magic mouse will be more effective than a click from my not-magic mouse. I’m not holding out much hope, though. I have no doubt that my sales address will keep getting invited to raves in London long after I retire.

Despite the evidence that correctly setting expectations results in better delivery and higher ROI on lists some senders go out of their way to hide terms from recipients. I’ve heard many of those types of comments over the years.

If we tell recipients how often we’re going to mail them, we don’t think they’ll opt-in.

If you mail people, even those who have opted in, more than they want to be mailed, then they’re going to complain or ignore your mail. And that results in poor delivery. If you’re mailing so much even you think it’s going to drive subscribers away then maybe you need to re-think your email program.

They opted in at one point, and even though they opted out, we thought that they’d be interested in these other things we’d like to sell them.

No, the recipient opted out. An opt-out should be persistent. You cannot arbitrarily decide to opt your unsubscribes back into a new list. That’s going to cause you problems, either with your ESP or your subscribers or both.

We don’t care if the addresses don’t belong to the people who submit them, we’ll just mail those addresses anyway.

If you mail addresses that belong to people who never opted in to your mail, then you’re spamming. There are certain lists that are targets for this kind of abuse, usually partisan or highly political lists. That just means that the senders need to be even more careful about their subscription policies and setting expectations. Failure to do so results in delivery problems at major ISPs.

Setting expectations and listening to recipients is a vital part of successful delivery. Your recipients are your best allies in getting mail delivered to the inbox. Trying to deceive them or second guess their desires leads to diminished returns. Not only are you spending more money and time in strategy, but the more complex the system the less likely it is to be right.

Be clear. Be honest. Be recipient friendly.

Most of the culprits are marketing companies. United Business Media is a huge problem, for instance. I never even signed up for their mail, but they bought an address I’d used to register for a conference. I unsubscribed at least a dozen times, but the mail kept coming. Of course, it wasn’t actually mail I’d unsubscribed from. Every email was part of a different list.

There was no way to find out what lists I was on through their unsubscribe page and preemptively unsubscribe. I tried mailing their privacy department, but it took multiple emails to get any sort of response. Finally, someone responded that they had removed me from all their lists.

Illegal? Probably not. Annoying? Totally.

This is the reason I don’t unsubscribe from mail if I don’t recognize the sender. Too many people who “acquire” my email address without permission don’t actually pay any attention to the law, much less best practices.

The other time I see this problem is with some of the addresses I’ve used for testing customers and their vendors. I unsub from any lists I’ve signed up for when I’ve collected the information I need. It’s not totally unheard of, though, for those addresses to lay dormant for years and then start receiving mail again.

This is a problem. They’re “reactivating” addresses. Again, they’re probably different “lists” so it’s not a CAN SPAM violation, but I don’t really care. I unsubscribed. I don’t want any more of that mail. I really can’t figure out what possesses companies to just decide, after not having interaction with subscribers for years, that the right thing to do is just add those addresses to a new list.

It’s not even like they try and re-engage me. Or ask me to opt-in. All they do is start sending me copies of the Annoying Meme of the Hour newsletter. It’s even more frustrating because I know that the sender has been exposed to best practices. I have spent anywhere from weeks to months helping them create a email marketing program that shouldn’t do this kind of thing.

I’ve tried talking to some clients after this happens. Usually, the issue is the marketers or IT staff that I worked with are gone. A new, shiny marketing group has moved in and decided that they had this huge database and of COURSE they should mail it, all of it, opt-outs notwithstanding.

It happens to me as a consumer and subscriber, too. In those cases I don’t have much recourse beyond reporting it as spam and blocking the mail. I don’t trust that a new unsubscribe will work, since the last one didn’t. I have to take other steps to make the mail stop.

In this case, I am much less persistent than the sender is. I think it would be better if senders actually believed me when I said I didn’t want their mail. But I don’t expect that will ever happen. Too many senders think they know better.

In the recent past, I was dealing with a client’s SBL listing. We were talking about how their fairly clean subscription process ended up with multiple Spamhaus spamtraps on the list. They mentioned bounce handling, and that they’d not been correctly managing bounces for some period of time. Their bounce handling system was broken and no one noticed.

Last year, I was working with another client. They were looking at why some subscribers were complaining about unsubscribes not taking. A bit of poking at different forms and they realized that one of their old templates pointed to an old website. Their unsubscription form had broken and no one noticed.

Another client insisted that their engagement handling removed any addresses that didn’t open or click on mail. But after ignoring their mail for 6 months, they still hadn’t stopped mailing me. Their engagement handling was broken and no one noticed.

Periodic monitoring would have caught all of these things before they became a big enough problem to result in a Spamhaus listing, or recipient complaints, or lawsuits for failure to honor CAN SPAM. Unfortunately, many companies don’t check to make sure their internal processes are working very often.

Email marketing is not set and forget. You need to monitor what is happening. You need to make sure that your processes are still in place and things are still working.

I talked about Millenium hotels sending me with an utterly irrelevant ad earlier this week.

@Yahoomail direct message spammed all their twitter followers with an ad for something related to the new Yahoo mail product.

Anyone watching my twitter feed yesterday probably noticed me complaining about spam from Dell.

All of these things are just examples of sloppy marketing. In Dell’s case it’s even worse because they sent me multiple copies of the spam to different addresses. Two copies of the same “SHOP NOW!” email to different addresses, one of which has never been given to Dell.

Mail to the first address is unquestionably spam and I did send in a complaint to Dell’s ESP. That address is never used to sign up for anything. I did try clicking on the “update your subscription” link in the footer and Dell’s website helpfully told me that address was not on their mailing lists. Looks like Dell bought a list.

The second address is one that was involved with the purchase of software from Dell last July. This is the first non-transactional mail sent to that address. I can’t necessarily call the email spam as I did give it to Dell during the course of a transaction. However, Dell could have done a lot better in managing our “relationship” than they did.

Dell collected my email address as part of a transaction in July 2010. They did not start sending marketing mail to this address until May 2011. While Dell is a major brand and most people would recognize the name and may be a little less inclined to hit “this is spam” waiting 10 months between a purchase and regular mailings is a bad idea.  People who don’t use tagged addresses may forget they gave the sender an email address and automatically send in a spam complaint.

Sitting on an address for 10 months means Dell really should have done a welcome series, or even just a single welcome email, to ease the transition from no mail to regular mail. But, no, they just send me an email advertising their sales.

We’ve been Dell customers for quite a while, and all of our purchases have been enterprise grade hardware or software to run on those servers. We’ve never purchased anything remotely like office computers. But the sales flyer was for desktops, printers and monitors. Dell knows what I purchased from there, so why are they sending me ads for things I’ve never bought?

We have our own Dell sales rep, and my only involvement in the transaction is source of payment. Adding me to a product list really feels like spam.

Then there was the email itself.  The “update your subscription” link was broken and told me I wasn’t subscribed to their list. I mentioned it to Steve and he pointed out that particular link had been broken “forever.” How long has it been since anyone inside of Dell has checked that their footer links work?

What is Dell up to? Who knows. But they unarguably are sending mail to addresses that never opted in. And even if you consider an email giving during a purchase process their handling of that particular address was appalling and in violation of almost every good practice out there.

A company I bought something from a while back added me to their newsletter. They seem to be having trouble making sales this quarter, as they’ve gone from an occasional email every few weeks to bombarding me with increasingly desperate offers in the past week or two. So I do what most recipients do in that situation (well, the ones who don’t just mark the mail as spam, anyway). I click the unsubscribe link.

I get a perfectly normal, standard unsubscription page, with a nice, prominent “Unsubscribe from all” button with good text explaining that that will remove me from all of the companies mailing lists. No requirements to log in, set dozens of checkboxes or provide a password I don’t have. So far this is a textbook example of a good unsubscription process.

I click the button. Nothing happens. That’s not good.

So I grab one of the people I know over at that ESP and we start looking at it. He clicks the button, and it loads a new page saying that I’ve been unsubscribed from all of the companies mailing lists.

A bit more testing shows that the unsubscription works if you use Internet Explorer or Firefox, but not if you use Safari. The cause of the bug was threefold:

1. the unsubscribe button used javascript and ajax and hidden form hacks to do exactly the same thing a standard html form would do, but in a more complex, fragile way
2. the webserver customized the content of the page it sent based on the browser being used – and if that browser were something other than Firefox, Internet Explorer or (probably) Chrome it sent broken content missing a bunch of javascript
3. nobody had tested the unsubscription page, at least not using Safari

If this were a software development blog post I’d focus on points 1 and 2, but we’re mostly an email blog so I’m just going to look at point 3.

If your unsubscription link only works in Internet Explorer and Firefox, it doesn’t work for a quarter of recipients. If it only works in those two, plus Google Chrome, it doesn’t work for about 13% of recipients.

And if it doesn’t work, people are going to mark it as spam – and you really don’t want that.

It’d be nice to rely on your web development group to use robust web coding techniques, and to QA all parts of your website exhaustively across different browsers, but if you’re not actually doing the web development yourself you likely don’t have much visibility into what they’re doing. For much of your web interface that’s just fine – it’s used by customers, and if they find a browser-specific bug they’ll contact support. Unsubscription handling is a bit different, though, as it’s mostly used by people who aren’t your customers – and there’s potential for bigger problems.

It’s worth asking your developers about what browsers they QA with, but it’s also pretty easy to do some basic testing yourself. Test it with Firefox, Chrome and Opera – they’re free to download, if nobody in the office has them installed already. Grab an iPhone and try with that – that’ll both test that it works with Safari, and that it’s usable on a small screen. Just sign up for the list, get an email, copy the unsubscription link into each browser’s address bar and test that the process works.

Redirection links are simple in concept – you include a link that points to your webserver in email that you send out, then when recipients click on it they end up at your webserver. Instead of displaying a page, though, your webserver sends what’s called a “302 redirect” to send the recipients web browser on to the real destination. How does your webserver know where to redirect to? There are several different ways, with different tradeoffs:

The simplest approach

The simplest sort of redirection link includes the final destination in the link itself – something like http://click.example.com/cnn.com/WORLD/. The webserver at click.example.com would simply strip off the first part of the link, and redirect to the remainder – cnn.com/WORLD/.

This is nice, because it’s fairly transparent to the recipient – when they hover over the link in their mail client or webmail it’ll be fairly clear where it’s going.

But it has several limitations. One is that you can’t really record very much data about the click – you know where it was redirecting to, but almost nothing else.

The bigger problem is that it’s very easy for a spammer to abuse – they can send out spam that has the link http://click.example.com/onlinepharmacy.ru/order.html, to hide their real link from spam filters, and your webserver will happily redirect recipients to go there. Or, worse, that can be used to redirect to a website hosting viruses. That can cause all sorts of problems for your reputation, up to and including having your redirection webserver blacklisted by antivirus and antiphishing organizations, meaning it’ll be blocked by many web browsers.

Add some metadata

Some of the things you might want to be able to record about a click would be which customers mail it was found in, which mailing campaign and which recipient it was sent to. This would let you do more sensible reporting and click-tracking, and also let you spot when a link is misused in some way (for example, thousands of clicks on a url that was sent to just one recipient).

That might look like http://click.example.com/123/456/789/cnn.com/WORLD/. Your webserver would strip off the first four parts, recording a click for customer 123, campaign 456 and recipient 789, then redirect to the remainder – cnn.com/WORLD/

This lets you do better reporting and is still fairly transparent to the recipient, but can still be abused in the same way.

Use a database

If you stored every link you wanted to redirect to in  a database you could simply store a unique key for each link – so you might record that key 2718 means http://cnn.com/WORLD/. Then the redirection URL might look like http://click.example.com/123/456/789/2718

This lets you do good reporting and is much more difficult for spammers to abuse (but not impossible – if the spammer signs up for a free or demo account on your system, then sends a test email to themselves, they can then reuse the links that they received in that mail).

But it’s fairly opaque to the recipient – they have no idea where the link will go. And it requires maintaining a database of every link you’ve ever used, for as long as it’s valuable (which could easily be several years if a recipient goes back to an old newsletter) and requires a database lookup for every click – which adds a fair bit of infrastructure you need to keep working 24/7 just to make links work.

Use a database and a cosmetic link

You could take the database format and add the final destination link on the end – like this http://click.example.com/123/456/789/2718/cnn.com/WORLD/ – and then just ignore everything after the url key (2718). That’ll work exactly the same way, but the final destination will be fairly transparent to the recipient.

This still can’t be abused by spammers, as if they try to use http://click.example.com/123/456/789/2718/mypharmacy.ru, it’ll still just redirect to http://cnn.com/WORLD/ as the only meaningful bit of the redirection link is the “2718“.

Cryptographically sign your links

A different approach is to record all the information you need in the link and to also add a cryptographic signature to prevent people from misusing it. This is much simpler than the word “cryptography” suggests, you just need to use a magic word (we’ll use “albatross”) and know about the md5() function.

You start off with the same destination string we used in Add some metadata – “/123/456/789/cnn.com/WORLD/“. Then you add the magic word on the end, to give “/123/456/789/cnn.com/WORLD/albatross“, and take the md5 “hash” of that. That’s some cryptographic black magic that’ll give you a string of letters and numbers that’s a “fingerprint” of that string. It’ll look something like “609a78b941bdf9f045cadcfa2e09d54c“. Then you combine that with the destination string to look like this:

http://click.example.com/609a78b941bdf9f045cadcfa2e09d54c/123/456/789/cnn.com/WORLD/

Then, when your webserver sees this link it splits it into the hash (609a78b941bdf9f045cadcfa2e09d54c) and destination string (/123/456/789/cnn.com/WORLD/). It then does exactly the same thing you did when you created the link – appends the magic word to the destination string to give “/123/456/789/cnn.com/WORLD/albatross” and takes the md5 hash of that string. If the result of that matches the hash in the link, it knows it’s a valid redirection link and it can record the click-tracking data and forward to the destination link. If the result doesn’t match it knows that the link has been tampered with, and can return an error page.

To generate the link in PHP would be something like this:

$destination = "/$customerid/$campaignid/$recipientid/$link";
$clicktrack = 'http://click.example.com/' . md5($destination . 'albatross') . $destination;

This is much cheaper to generate and validate than using a database, even a typical in-memory database.

Which to use?

Don’t use the simple approach – it’ll get abuse sooner or later and you’ll regret it. Any of the database or cryptographic approaches work just fine, though the cryptographic approach may be easier to scale up and maintain. The database approaches make it easier to disable a link, or direct it to somewhere else at a later point, in case of abuse or some other need.

What else is it good for?

You can use the same sort of approach to validate unsubscription links and VERP return paths for bounce handling. And “open tracking” using these sort of links for image URLs, if you find that a useful metric to offer.

There’s been an ongoing discussion about adding thumbs up / thumbs down style buttons to email clients. While I am dubious this is a useful feature or something that recipients will use, if there are others in the industry that think it would be useful then I strongly suggest they go ahead and create it.

In fact, there are a couple things that have been asked for in email interfaces that aren’t currently provided. Last October I blogged about adding an unsubscribe button to email clients.

Now, many email clients have not implemented an unsubscribe button, but I’m told it’s not difficult to write plugins for many common email clients (mail.app, thunderbird, outlook). I think that if a few senders should get together and write the unsubscribe plugins and make them freely available to recipients. If the tools are out there, and the recipients want them, then they’ll be used. There’s nothing stopping senders from creating the tools they want created. Hire a few developers and get it done. You’re the marketers, market the benefit to the recipients to use your tools to improve everyone’s life.

The same thing goes for a thumbs up / thumbs down mechanism. If you write it, and it is useful (and useable) then users will demand that the proprietary interfaces provide the same functionality.

In the blog post I initially made that suggestion one of the comments said

To suggest that email marketers ought to get together and write plug-ins for popular email clients in order to fix the problem misses the point – this is a feature that ought to be a standard part of the software/web interface, as a plug-in it’s subject to vagaries like incompatibilities when the software is upgraded (see how many Firefox plug-ins show errors immediately after an upgrade).

Which shows some gross misunderstanding of how feature development works. Every feature started off with someone saying the mail client needs to do X either because they wanted the feature or because their users were asking for it.

In the case of unsubscribe buttons, or thumbs up/down buttons, end users aren’t currently asking for the functionality. That’s not to say they wouldn’t use it if it was there, just that they aren’t asking for it. The way to get the functionality inserted as a standard part of the software/web interface, is to get users to ask for it. In order to get users to ask for it, the best way to start is to create a plug-in that they like and use. If they like it in their Outlook interface at work, then they’ll ask for it in their webmail interface at home.

Many plug-ins are written by a single coder as a hobby or functionality they wanted or needed for their own mail. This is why there are maintenance failures this is a hobby or labor of love and real life™ interfered in the upkeep or the functionality was no longer needed by the maintainer or any number of reasons. This can trivially be resolved by someone being paid to maintain the plug-in and keeping up with the new versions of mail clients throughout the development cycles.

Is this guaranteed to lead to the email interface improvements senders are asking for? Of course not, nothing is guaranteed. But I can assure you that actually creating the protocols and buttons and interfaces will lead to widespread adoption faster than simply waiting for someone else to do what you want.

Since the poster took the time to link to my blog, I thought I’d take the time to look in detail at his post and talk about how likely it is to work.

Email marketing is both one of the most cost effective methods of reaching your customers and the most loathed.

Recipients don’t loathe mail they want. If your recipients actually loathe what you’re sending then you’re doing it wrong. If you can’t send wanted and relevant mail then you need to reconsider your email marketing program.

What I do loathe is all those idiots that send me thousands (yes, thousands) of messages a day that I never asked for, and don’t want.

Email marketers have to contend with over zealous junk mail filters, spam crusaders that seek to destroy them and list subscribers who forgot they gave permission. It’s so much easier to ‘report as spam’ than it is to unsubscribe.

Yup, senders have to deal with the fallout from spammers just as much as recipients do. Welcome to life on the Internet where the spammers have made email worse for all of us.

I’ve used email marketing myself. I also hate spam. I will only use opt-in lists for this reason. Yet that doesn’t stop recipients of emails I’ve sent replying with torrents of abuse for daring to darken their inbox, and those are the ones I’ve heard from.

I know a lot of email marketers and those that are really sending with permission don’t get “torrents of abuse for daring to darken their inbox.” Something about this story just isn’t ringing true. I suspect that these “opt-in” lists are being purchased and the recipients are really tired of being spam victims.

Many users will just instruct whatever spam filter they use to block an email. Depending on how that spam filter works, that action gets reported and if enough people do that, the sender of the email gets blacklisted. In the case of an opt-in list this is sailing pretty close to a collective act of defamation.

Yes, if a lot of people say they don’t want mail from a particular sender, than mail from that sender is going to be blocked. That’s how spam filters work. What I can’t understand is why this person keeps doing the same thing, sending mail to these opt-in lists full of recipients that don’t actually want his mail, over and over again while expecting the results to be different.

I’m always suspicious of anyone who claims saying “this mail is spam” is defamation. It shows a deep, deep misunderstanding of  how the term “spam” is used by ISPs and end user recipients. It conflates legal concepts with colloquial terms. It is the essence of whining about how mean everyone is to you.

When users mark an email as spam, and that blacklists the sender and prevents other subscribers, who would gladly have received (and may even have been looking forward to) that email from benefiting from the content of it.

If you stop sending mail to the folks who don’t want it, then the folks who do want it can get it. If so many of your users don’t want your mail that the ISPs or spam filters can’t distinguish your mail from spam, then your mail is going to be treated as spam.

Let me repeat that: If you don’t want your mail to be treated like spam, stop sending mail that is indistinguishable from spam.

There is a solution, though it’s only partial, in the form of FBL or FeedBack Loops. Setting them up is a little complicated, though is often included in the service provided by reputable email marketing providers. I say partial because it only provides a solution for large email providers/ISPs like AOL, Comcast, Hotmail and others (a non-exhaustive list can be found here), and has to be set up with each ISP, per sending domain. An entry on the FBL for an ISP means that when one of that ISP’s customers reports your message as spam, instead of you getting blacklisted, you get a report, requiring you to unsubscribe that user. An FBL however makes no difference if the recipient of an email isn’t using their email provider’s web interface, a third party spam filter [sic]

This is a bit of a mis-interpretation of how FBLs actually work. FBLs, with the exception of the Yahoo FBL, are set up based on IP addresses, not domains. Also, companies that have FBLs set up can still be blacklisted at those ISPs. Companies with FBLs are sometimes given a bit more leeway and slightly higher thresholds before mail is blocked, but sometimes they’re not.

What is needed is a concerted effort by providers of spam filtering solutions, internet service providers (as users of those spam filters), email client developers (web and desktop) and email marketing vendors. All it would take is a recognised standard email header for ‘unsubscribe address’ and ‘unsubscribe URL’, which email client software, or the spam filter in use, would interpret and communicate with, instead of placing a black mark against the sender. The email marketing vendors (or the DIY sender) would handle the unsubscribe submissions. The list might get smaller but the deliverability goes way up.

Finally! The author of the blog post is in luck! There is, in fact a recognized standard email header for ‘unsubscribe address’ and ‘unsubscribe URL.’ At least two different ISPs (Hotmail and gmail) are using the header to provide their users simple ways to unsubscribe from mail.

Now, many email clients have not implemented an unsubscribe button, but I’m told it’s not difficult to write plugins for many common email clients (mail.app, thunderbird, outlook). I think that if a few senders should get together and write the unsubscribe plugins and make them freely available to recipients. If the tools are out there, and the recipients want them, then they’ll be used. There’s nothing stopping senders from creating the tools they want created. Hire a few developers and get it done. You’re the marketers, market the benefit to the recipients to use your tools to improve everyone’s life.

This appears to be the way Google are going with their unsubscribe option in Gmail. Criticism of this by email marketers is levelled at the wording and operation – equating unsubscribing with reporting spam. It fits with Google’s usual m.o. of trying to simplify a process as much as possible, as long as the sender does what they’re supposed to.

The criticism of Google is out there, but it doesn’t make it right. Google has also implemented the ability to unsubscribe without reporting the mail as spam. This is only being offered to senders who have good reputations at Google and who are using the RFC2369 List-Unsubscribe: mailto header.

Who loses out? People who don’t play by the rules. Everybody else wins. The spam filter providers have shorter, easier to process blacklists. Email providers and email marketing vendors spend less time processing blacklist removal requests and finally the end user who wants a mailing is guaranteed to receive it.

This is what I don’t understand. How does this do anything to punish folks who are not playing by the rules? What do they lose? In fact, what’s to stop spammers from playing by these rules? Spammers already have fake unsubscribes set up, use false headers and steal content from other senders.

I realize that sending mail is currently complicated and painful. It’s hard to not do all those things like buy lists, send lots of mail and annoy recipients. This is what separates the good marketers from the bad. Good marketers don’t have near the problems with spam filters that this blogger seems to have.