Archive for the 'Delivery Improvement' Category

Feedback loops: net benefit or net harm?

There has been a very long, ongoing discussion on one of my mailing lists about whether or not feedback loops are a net good or a net harm. I believe, overall, they are a net good, but there are people who believe they are not. The biggest objection is that the lawyer mandated redaction of the To: address combined with the fact that some users use the “this is spam” button to delete unwanted email, makes it difficult for some FBL recipients to sort out the real issues from the cruft.

Redaction can be a problem for some senders, particularly for the small mailing list hosted as a hobby or contribution to the community. In order to effectively deal with FBL emails, a sender needs to have tools on the email sending side and on the FBL receiving side. This is often more overhead than the volunteer list maintainer wants to handle. Unfortunately, these senders are a minority and therefore their issues are often not addressed by the ISPs.

Some of the objections and complaints about “broken” or “useless” FBLs come from people who do not really have any history for the FBLs, where they are, what they were designed for and who their target audience is. A bit of history may help explain why things are how they are.

The First FBL

The “this is spam” button evolved from the “notify AOL” button. This button was a way email recipients could notify AOL staff about any number of problems, including threats, viruses and other unwanted emails. As time went on, this was changed to “this is spam” to encourage users to report more spam so the AOL would have the data to make delivery decisions. Eventually, AOL made the decision to share that data with some senders and ISPs. The lawyers made the decision to redact the “To:” address, but not make any other changes to the message because they believe they should not be sharing subscriber email addresses with third parties. As some people correctly point out, the lawyers are not interested in hearing from non lawyers about changing this. It is possible that another lawyer may be able to put together a position paper and convince them this stance is overly cautious. I am pretty sure, though, that no one without a legal degree will be given any audience from them.

Given the success of the AOL FBL and the demand from both ESPs and ISPs for FBLs, other ISPs started offering FBLs as well. Many of them also redacted the To: address, either just following AOL’s lead or under advice of their own counsel.

That means, as senders, we are in a situation where we really cannot make the ISPs change what they’re doing. We can either adapt our own mailing practices to cope with them or we can forego the data provided by the FBL. One of the challenges in choosing to shun the whitelist at AOL that in order to qualify for whitelisting, you have to accept a FBL. For ISPs, who want to whitelist their outgoing MTAs, but have customers sending mail, maybe running small mailing lists, or who are forwarding mail to their ISP account, this can be a problem. However, any ISP needs some sort of abuse desk automation, and this automation should be able to handle FBLs. This can also be a problem for small ESPs or companies doing in-house email marketing. They buy something off the shelf to handle mail (or install mailman) that does not do VERP or otherwise enter the specific address in the email. When faced with a redacted email they cannot do anything with the complaint.

What does the FBL email tell the FBL recipient?

This really depends on what role the FBL recipient plays in the mail transport system. Bandwidth and network service providers use the FBL as an aggregate tool. They really only deal with FBL complaints if there is a change in complaint volume about an IP, they don’t treat each complaint as a valuable source of information. Typically what happens is that an ISP abuse desk notices a spike in complaints. After investigation, they may discover that a customer machine is compromised. They then notify the customer, the customer patches or disconnects the machine and the problem is fixed.

ESPs tend treat the FBL as an unsubscribe mechanism as well as a way to monitor customers. A few FBL complaints are not necessarily a sign that the sender is spamming, but once a threshold is reached the ESP delivery / abuse team addresses the issue. Spammers can get FBLs and often use them as a way to clean lists of complainants. Some really dirty spammers even suppress those complainants from all their lists.

Is a FBL useful?

This is really something that someone else cannot tell you. Some companies find FBLs to be extremely useful, even after they have had to make investments in software (either off the shelf or custom) to send mail that will survive the FBL redaction process and to handle the actual FBL email. Some companies find the FBLs to be more trouble than they are worth. The question, however, is really one only the sender can answer.

Overall, I think FBLs are more helpful than they are harmful. They do require investment on both sides of the transaction, but does encourage senders and receivers to cooperate with one another.

3 Comments

McColo goes offline

Last week a major player in the botnet arena was taken offline when they were shutdown by their upstream provider.  With the demise of McColo, there has been a 30 - 50% drop in the amount of spam as measured by any number of different techniques. The CBL team has posted an article about their view of the McColo disconnection, which includes links to press articles about the shutdown. Spamhaus has their own take on the shutdown and another collection of links to articles about the shutdown.

In my own mailbox, I have noticed a drastic decrease in the amount of spam over the last week. I am too jaded to expect that the change is permanent, but it is nice while it lasts.

2 Comments

Don’t do this

I recently received an email from someone I do not know. This email was welcoming me to the friends and family beta of a new website. This email got under my skin a bit and it has been one of those weeks and so I decided to reply to the email.

“Whomever sold you this email address lied to you,” says I. I did not point out all the reasons I know this, including the two @home.com addresses in the To: line next to mine, just stated that as a fact.

The sender replied telling me he did not purchase any email addresses, he just mailed the contents of his address book. At that point, I did a little poking around the web to see if I recognized the sender or we had worked together in the past or if there was a clear join between him and me. I could not find anything that triggered a memory in my mind, so I replied again. “Do you know where we met? I am not placing you.”

He finally replied, “Oh, you’re in my address book under Spamhaus. We must have interacted while you were working there.”

Please, please, dear readers, when you next launch your online business do not mail everyone in the address book you have been maintaining for the last 10 years. If you want to mail your friends and family, then do so. But just because you have an email address does not mean that the recipient wants to hear from you. And, really, mailing the folks you think work for Spamhaus? Not smart. Had I really been a Spamhaus employee, chances are his bright, shiny new company would be blocklisted before it ever had a chance.

1 Comment

AOL and DKIM

Yesterday, on an ESPC call, Mike Adkins of AOL announced upcoming changes to the AOL reputation system. As part of these changes, AOL will be checking DKIM on the inbound. Best estimates are that this will be deployed in the first half of 2009, possibly in Q1. This is something AOL has been hinting at for most of 2008.

As part of this, AOL has deployed an address where any sender can check the validity of a DKIM signature against the AOL DKIM implementation. To check a signature, send an email to any address at dkimtest.aol.com.

I have done a couple of tests, from a domain not signing with either DK or DKIM, from a domain signing with DK and from a domain signing with both DK and DKIM. In all cases, the mail is rejected by AOL. The specific rejection messages are different, however.

Unsighng domain: host dkimtest-d01.mx.aol.com[205.188.103.106]
said: 554-ERROR: No DKIM header found 554 TRANSACTION FAILED (in reply to
end of DATA command)

DK signing domain: “205.188.103.106 failed after I sent the message.
Remote host said: 554-ERROR: No DKIM header found
554 TRANSACTION FAILED”

DK/DKIM signing domain: “We tried to delivery your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 554 554-PASS: DKIM authentication verified
554 TRANSACTION FAILED (state 18).”

As you can see, in all cases mail is rejected from that address. However, when there is a valid DKIM signature, the failure message is “554-PASS.”

As I have been recommending for months now, all senders should be planning to sign with DKIM early in 2009. AOL’s announcement that they will be using DKIM signatures as part of their reputation scoring system is just one more reason to do so.

0 Comments

Two reasons not to buy a list

Ken Magill, celebrity, has two articles today that highlight the issue with buying lists from vendors. The first is yet another article about EmailAppenders selling bad data. In this case, it is not the buyers who are complaining. According to Ken EmailAppenders are sending out email advertising they can sell Internet Retailer’s list of 2008 conference attendees. Internet Retailer is disputing this and has sent EmailAppenders a cease and desist. EmailAppenders is currently dodging Ken’s attempts to get their side of the story.

The second is an article about Zoominfo, a new group in the list selling business. Zoominfo has long been harvesting information from other sites. Now, they are offering to sell their scraped and harvested list. Their only requirement is that the buyer sign an agreement to comply with CAN SPAM. And, yes, if someone is dumb enough to buy this harvested list, they should comply with CAN SPAM as sending mail to a harvested list triggers additional penalties if or when the FTC decides to go after the sender.

Not only are Zoominfo harvesting data, they are harvesting from ancient and obscure sources. They have no current information for me, but they managed to find an email address for a job I left in 1993. They have Steve listed as an employee of “postgreSQL INC” because they harvested the postgres mailing list archives. Mickey pointed out one of Zoominfo’s sources is http://free-personal-ads-wanted-sex-partner-near-hays-kansas.themasterwithin.ca/arch/4/. You do not even need to visit that site, just look at the URL!

Zoominfo’s VP and general manager claims they send emails to people regularly, offering them the chance to opt-out. First of all, I have never received one of these, have any of you? Secondly, some of the addresses are so old opt-outs are not relevant. Finally, unless they are monitoring their delivery, which I strongly doubt given their business model, anyone buying addresses from them is going to buy lots of dead addresses. And spamtraps. Lots of spamtraps.

I am sure that people who buy and sell lists regularly will tell me that these are outliers and that most companies who sell lists have higher data collection standards. My experience suggests that these are middle of the road list brokers. They are companies who are willing to sell anything with an @ sign in it and do not care about how sending to that data affects their customers.

3 Comments

Bad Idea

My mailbox and IM windows have been swamped with messages about an ISP sending out mail to participants in their FBL program. It seems this particular ISP could use some delivery consulting.

See, this ISP sent out emails with blocks of 50 - 75 email addresses in the To: line. Bad idea. Delivery wise, I do not expect that they had many delivery problems. In the copy I saw, most of the addresses started with “support” and those addresses often have fewer blocks on them than other addresses. I also do not think this ISP will see subscriber emails blocked because of this.

However, it is very, very bad practice to do what they did. And the fact that the ISP can get away with it does not mean that any other mailer can get away with it.

0 Comments

Reputation: part 2

Yesterday, I posted about reputation as a combination of measurable statistics, like bounce rates and complaint rates and spamtrap hits. But some mailers who meet those reputation numbers are still seeing some delivery problems. When they ask places, like AOL, why their mail is being put into the bulk folder or blocked they are told that the issue is their reputation. This leads to confusion on the part of those senders because, to them, their reputation is fine. Their numbers are exactly where they were a few weeks ago when their delivery was fine.

What appears to have changed is how reputation is being calculated. AOL has actually been hinting for a while that they are looking at reputation, and even published a best practices document back in April. Based on what people are saying some of that change has started to become sender visible.

We know that AOL and other ISPs look at engagement, and that they can actually measure engagement a lot more accurately than sender can. Senders rely on clicks and image loading to determine if a user opened an email. ISPs, particularly those who manage the email interface, can measure the user actively opening the email.

We also know that ISPs measure clicks. Not just “this is spam” or “this is not spam” clicks in the interface, but they know when a link in an email has been clicked as well.

I expect that both these measures are now a more formal and important part of the AOL reputation magic.

In addition to the clicks, I would speculate that AOL is now also looking at the number of dead addresses on a list. It is even possible they are doing something tricky like looking at the number of people who have a particular from address in their address book.

All ISPs know what percentage of a list is delivered to inactive accounts. After a long enough period of time of inactivity, mail to those accounts will be rejected. However for some period of time the accounts will be accepting mail. Sending a lot of mail to a lot of dead accounts is a sign of a mailer who is not paying attention to recipient engagement.

All ISPs with bulk folders have to know how many people have the from address in their address book. Otherwise, the mail would get delivered incorrectly. In this way, ISPs can monitor the “generic” recipient’s view of the email. Think of it as a similar to hitting the “this is not spam” button preemptively.

This change in reputation at the ISPs is going to force senders to change how they think of reputation, too. No longer is reputation all about complaints, it is about sending engaging and relevant email. The ISPs are now measuring engagement. They are measuring relevancy. They are measuring better than many senders are.

Senders cannot continue to accrete addresses on lists and continue sending email into the empty hole of an abandoned account while not taking a hit on their reputation. That empty hole is starting to hurt reputation much more than it helps reputation.

0 Comments

Reputation

Reputation is the buzzword in delivery these days. Everyone talks about building a good reputation and how to do it. Makes sense, the ISPs are always hammering on reputation and how critical reputation is. The more I talk with delivery folks on the ESP side of thing, the move I realize that there is a fundamental disconnect between what the ESPs mean when they say reputation and what the ISPs mean when they say reputation.

Many people handling delivery think that the bulk of reputation is wrapped up in complaint rates and bounce rates. I think they know the ISPs measure more than just complaints and bounces (spamtraps!) but really believe that most of developing a good reputation is all about keeping those complaints low.

This perspective may have been true in the past, but is becoming less true as time goes on. There are a lot of very smart people managing incoming mail at the ISPs and they are constantly looking for ways to better meet the desires of their customers. Lest we forget, their customers are not the senders, their customers are the end users. Their customers are not senders.

Part of meeting the needs of end users means actually giving them a way to provide feedback. AOL started the trend with the this-is-spam button, and other ISPs (ones that controlled the user interface at least) followed suit. For a very long time, reputation was dominated by complaint percentages, with modifiers for number of spamtrap addresses and number of non-existent users.

The problem is, these numbers were easy to game. Spammers could modify their metrics such that their email would end up in the inbox. In response, the ISPs started measuring things other than complaints, bounces and spamtraps. These other measurements are strong modifiers to complaints, such that mailers with what used to be acceptable complaint rates are seeing their mail end up bulked or even rejected.

Recently, AOL seems to have made some subtle modifications to their reputation scores. The result is mailers who have previously acceptable complaint rates are seeing delivery problems. When asked, AOL is only saying that it is a reputation issue. Lots of senders are trying to figure out what it is that is more important than complaints.

Tomorrow, I will talk about what I think AOL could be measuring.

4 Comments

Evaluating email

DJ posts the top 4 reasons an email campaign fails.

1 Comment

Constituents clog lawmaker mail servers

With the recent credit market turmoil and the proposed 700 billion dollar bail out bill many, many Americans are taking the opportunity to contact their congressional representatives. This increase in traffic has resulted in the house.gov website being slow or unresponsive, the mailservers being clogged and the phone system straining.

In response to the increased load, the CAO has put some limits on incoming email and is restricting the number of e-mails sent via the “Write Your Representative” function of the House website.

Ventura [representative of the CAO] likened the problem to a bottleneck scenario on a highway, where multiple lanes of traffic converge into a smaller set of lanes. In that situation, some cars get to move forward while others have to remain at a standstill.

“What we had to do was basically install the digital equivalent of a traffic cop,” Ventura said. “It was a question of inconveniencing everybody or inconveniencing some people some of the time, while servicing other people the other half of the time.”

This is similar to what some ISPs have to do under periods of peak load. They have to delay some senders sometimes so that they can manage the load.

1 Comment