Is their increased vigilance pissing you off? If so, your anger is misplaced. They are reacting quite sensibly to market conditions apparently imposed by Spamhaus. Ken Magill

While I agree with Ken that the ESPs are reacting to market conditions. Where we disagree is the idea that these conditions are imposed by Spamhaus. I don’t think all the uptick in ESP enforcement and compliance activity is the result of Spamhaus’ actions. I believe that many of the mass market ISPs are changing how they detect unwanted mail, and are fine tuning filters to reduce the amount of unwanted mail that shows up in the inbox.

One of the big changes is better tools for handling huge data sets. Bigger ISPs handle billions of messages a week. Even just collecting and storing the mail is a giant task. Storing it in a useable form was almost out of the question. But over the last few years there have been significant improvements in the speed and affordability of hardware to handle very, very large datasets. Likewise, there have been algorithm and software improvements in mining that data for useful correlations.

In practical terms, ISPs and filtering companies like Spamhaus don’t have to focus on complaints or trap hits or “simple” measurements. They can draw complex correlations and look at mail in a way that was simply impossible 2 or 3 years ago. This means they can better identify senders who had previously been able to slide in under the filters.

Spamhaus rolled out tools to monitor their spam feeds in a different way and have been listing a lot more “legitimate” senders because of it. ISPs are rolling out tools to better filter “greymail” and keep users inboxes full of mail that the users actually want.

One of the trends I’m noticing is that direct marketers are getting more aggressive. Whether it’s a response to the years of recession or a response to the slowly warming economy, I can’t tell. But there are a lot of direct marketers who are no longer afraid to break the law. For instance, my cell phone is getting multiple telemarketing calls a week, despite being a cell and despite being on the do not call list. My inbox is full of unsolicited email carefully engineered to get past standard filters, much of which violates CAN SPAM. I’m even getting the occasional unsolicited fax.

The increase in listings by Spamhaus are one example of the filtering screws being tightened. But it’s not just Spamhaus that’s driving this; ISPs and filtering companies are also filtering more aggressively. I’m seeing a lot more emphasis being placed on content and a good IP reputation is no longer a ticket to the inbox. Content must be clean and recipients have to want mail for it to get into the inbox.

According to the FBI

the cyber ring used a class of malware called DNSChanger to infect approximately 4 million computers in more than 100 countries. There were about 500,000 infections in the U.S., including computers belonging to individuals, businesses, and government agencies such as NASA. The thieves were able to manipulate Internet advertising to generate at least $14 million in illicit fees. In some cases, the malware had the additional effect of preventing users’ anti-virus software and operating systems from updating, thereby exposing infected machines to even more malicious software.

The FBI worked with a number of security groups around the world as part of the investigation and take down. TrendMicro was one of the groups that first identified this botnet. On their blog they discuss the information they collected during a 5 year investigation into Rove Digital. Spamhaus, too, had a large collection of information about Rove Digital in the Register of Known Spamming Operations (ROKSO).

In his report on the take down, Brian Krebs also mentions the role of the ISC in keeping the millions of infected users from losing internet access during the seizure.

Congratulations and thanks to everyone involved in the hard work it took to identify and arrest these criminals. And for the effort people put in to making the Internet safer for all of us.

Brendan is a long term spammer, who used to be in the US and moved to New Zealand in 2006. His presence in Auckland was noticed by Computerworld when a number of editors and staffers were spammed. When contacted by the paper, Brendan denied being involved in the spam and denied being the same Brendan Battles.

New Zealand anti-spam law went into effect in September 2007. The Unsolicited Electronic Messages Act 2007 prohibits any unsolicited commercial email messages with a New Zealand connection, defined as messages sent to, from or within New Zealand. It also prohibits address harvesting.

The Internal Affairs department also appears to be investigating companies that purchased services from Brendan Battles.

Internal Affairs was still investigating businesses that had bought and used IMG databases some of which had been fined for breaching the act. Senior investigator Toni Demetriou said the sender of any commercial electronic message must have the consent of the recipient before the message was sent. stuff.co.nz

Poor Brandon. He moves to a country with no anti-spam law and then a year later they enact a law prohibiting any unsolicited email marketing.

After a bench trial on the issue, the district court awarded e360 a mere $27,002, a far cry from the millions of dollars that e360 sought. Both parties have appealed. We conclude that the district court properly struck most of e360’s damages evidence, either as an appropriate discovery sanction or for proper procedural reasons, and we reject e360’s challenges to the judgment. We also agree with Spamhaus that the evidence failed to support the modest award of $27,000 in actual damages because e360 based its damage calculations on lost revenues rather than lost profits. We vacate and remand with instructions to enter judgment for e360 in the nominal amount of three dollars.  Judge Hamilton, opinion

Spamhaus posted the final judgement. The full opinion is also available.

HT: Mickey Chandler who posted a case summary and a bit more of the background on what is going on than I managed.

1. Make a URL shorter
2. Track clicks on the URL
3. Hide the destination URL

Making URLs shorter was their original role, and it’s why they’re so common in media where the raw URL is visible to the recipient – instant messaging, twitter and other microblogs, and in plain text email where the “real” URL won’t fit on a single line.

From the moment they were invented they’ve been used to trick people to click on links to pages they’d rather not visit, from musical classics to less tasteful content. And, in just the same way, spammers quickly found that they were a good way to avoid content-based filters or to hide a suspicious looking target URL.

Inevitably, URL shorteners that are persistently abused by spammers (especially those where that’s done with the support of the URL shortener operator) start to be seen as a sign of spam, and email that uses them will be treated with suspicion by content-based spam filters and often sent to the spam folder.

bit.ly is probably the highest profile URL shortener, so it’s the one you’ll most likely see people trying to use in email. What effects does that have?

Now being “totally owned” by the Canadian Pharmacy gang, thousands of URLs being spammed with very slow takedowns. Not good.SpamHaus on bit.ly

bit.ly have been on SpamHaus’s radar for quite a while. They’re listed on the SBL multiple times. They’re listed in the DBL – SpamHaus’s newish domain based blacklist, intended for content-based filtering of email. All this means that emails that contain bit.ly URLs are increasingly likely to have serious delivery problems.

This isn’t unique to bit.ly: many other URL shorteners have similar problems – j.mp, su.pr, and others. Nor is it unique to SpamHaus: many other spam filters, public and private, are starting to treat common URL shorteners with suspicion.

Naive use of URL shorteners in your email will send it to the spam folder.

More about why you shouldn’t do that – and what you can do instead – tomorrow.

Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 554 554 5.7.1 The IP address of web site www.client.com [75.101.163.44] is listed at www.spamhaus.org (state 18).

What? An SBL listing? The client hadn’t done anything wrong, certainly nothing that would provoke the wrath of Spamhaus. And… they’re not sending email from 75.101.163.44 anyway, they’re sending it out through Google Apps. And, wait, www.client.com is listed on the SBL – the SBL lists IP addresses, not hostnames.

What is going on here?

We’ve mentioned in passing before that one of the good ways to filter mail based on content is to look for suspicious URLs in the message. One way of doing this is to use hostname-based blacklists, such as SURBL, URIBL or DBL. These list domain names that have been seen in spam (and pretty much only spam), and sending email with a listed hostname in it is a quick trip to blocksville at many ISPs.

Spammers, phishers especially, often cycle through domain names quickly in order to avoid the (manually maintained) hostname-based blacklists. They often host them at the same place, though, so if you look up the IP address the hostname resolves to you can use an IP based blacklist to see if the hostname is being used for spam or phishing related email payloads, and use that information to block the email. That’ll work even if the phishers use an entirely new domain for their websites, if it’s still hosted at the same place.

The SBL blacklist is commonly used in this way. It’s manually maintained and fairly hard to get on to, and finding URLs that resolve to addresses listed on the SBL in an email corresponds pretty strongly to the mail being unwanted. The folks who run the SBL are quite aware of this, and will commonly list IP addresses that are being used to host websites advertised in spam even if they never send email.

What happened in this case was that the client was hosting their website with Heroku, a perfectly respectable cloud-based ruby-on-rails web host. But Heroku use just three IP addresses for all their customers. And one of their customers was zapt.in, a URL shortener with a serious spam problem. Zapt.in caused problems for long enough, and didn’t respond to them for long enough, that their IP address was listed on the SBL.

That meant that all of Heroku’s customers were using an IP address listed on the SBL. Which, in turn, meant that any email those customers (or their affiliates or customers or…) sent that used the customers domain would be rejected by ISPs using the SBL-as-a-hostname-blacklist trick – which is a lot of large ISPs.

What can you do to avoid this? The ideal is to not host your website on a shared IP address (or in a /24 that’s littered with spam and phishing sites).

If you can’t do that – you really can’t move your main website to a more reputable host – then your next best option is to not use your main website in any of the email you send. You don’t want to hide the connection (because you don’t want to look like a snowshoe spammer who’s obfuscating their domain ownership), but you want the hostname to be different. A good way to do that is, if your main domain name is example.com and your website is www.example.com, is to use a subdomain for URLs in emails. click.example.com, maybe.

Host that subdomain somewhere else, on an IP address you have a bit more control over – an inexpensive VPS or web hosting provider, and just run a web redirector there that simply sends an http 302 redirect for any http://click.example.com/foo/bar/baz.html to http://example.com/foo/bar/baz.html. And use the click.example.com form of the URL for everything you use in your email – not just links, but also image tags and so on.

What if setting up your own redirector isn’t something you have the resources to do? Sign up for a web redirector or URL shortener service that’ll let you use your own domain name, like bit.ly Pro. That won’t give you as much control, or protection, as running your own redirector but it’s a lot better than running your website on a shared IP address.

Two recent stories reminded me how evil some folks are. While I’ve not had any direct contact (that I know of) with any of the players on this end of things I have zero doubt that if they called me they would tell me that they were legitimate email marketers.

In one case, a members of a spam gang kidnapped the teenage daughter of someone investigating their activities. The gang held her for more than 5 years in horrific conditions. Yesterday Joseph Menn, author of “Fatal System Error” posted on Boing Boing that his friend got his daughter back. It is a heartbreaking story and incredibly sobering.

In another case, the Russian police arrested a man who ran spammit.com, a clearinghouse for viagra sellers to find spammers to send their mail. Reports say that mail volumes dropped by a fifth after the site was taken offline.

There is real evil in the email marketing industry. Sure, they’re spammers and we can all stand up and say they’re not legitimate. But, this is what the ISPs and Spamhaus and law enforcement are dealing with on a regular basis.

“SPAMHAUS BLOCKS GOOGLE!!!” the headlines scream.

My own opinion is that Google doesn’t do enough to police their network and their users, and that a SBL listing isn’t exactly a false positive or Spamhaus overreaching. In this case, though, the headlines and the original article didn’t actually get the story right.

Spamhaus blocked a range of IP addresses that are owned by Google that included the IP for www.gmail.com. This range of IP addresses did not include the gmail outgoing mailservers.

Spamhaus says

Some Google-owned server IPs hosting severe malicious spam problems – specifically Google’s “Google Docs” service – do get rightly listed in the Spamhaus SBL when Google does not take action fast enough to stop the serving of malicious sites via Google Docs. Such listings act as pointers to the abused resource but do not in any way affect Google’s Gmail service or any Google outbound mail service.

Spamhaus goes on to talk about the responsibility providers have to police their userbase and the fact that large providers who are not policing their users are cost shifting to the rest of us.

We at Spamhaus surely understand the challenges that the cloud service providers face. These problems are not easy to solve and the scale and complexity of the systems involved certainly does not make things easier. What we are puzzled by is how the rest of the internet has to keep carrying the burden of this abuse. The companies that host these services all without exception make hundreds of millions of dollars each year. They employ some of the best and brightest engineers. Surely they can spend a little of their immense resources on making the internet they rely on for their business, a better and safer place.

Unfortunately, Google doesn’t seem to see any value in policing their customers and users. If they can’t make a buck at it, then it doesn’t get done. And if Google’s costs of doing business are shifted to other companies, so much the better. Good for Spamhaus for standing up and pointedly telling Google they can’t keep supporting spam and spammers.

The gist of the story is that Mr. Luckman thinks that because it is legal to purchase lists and send mail that there is nothing anyone can do to stop him from doing so. Unfortunately for Mr. Luckman, this isn’t actually true. Simply complying with the law does not mean that spamming behaviour has to be tolerated by ISPs. What’s more, ISPs have a lot of power to stop him.

His recipients’ ISPs can stop him. Filtering companies can stop him. And his upstream can stop him. In fact, Mr. Luckman’s upstream is GoDaddy, a company that has an abuse desk that is one of the toughest on the Internet. They do not tolerate spamming at all and will disconnect customers that are spamming whether or not there is a SBL listing involved.

Sure, Mr. Luckman is complying, or says he’s complying, with CAN SPAM. But that doesn’t change the fact that he is violating his contract with GoDaddy. Given that admission, I am extremely surprised that the reporter focused so exclusively on Spamhaus’ role in this, without mentioning GoDaddy’s abuse enforcement or that Mr. Luckman has to comply with contracts he signed.

Most reputable marketers agree that sending mail to purchased email addresses is spam. Most recipients agree that mail they didn’t ask to receive is spam. Even the reporter agrees that Mr. Luckman is a spammer. Compliance with CAN SPAM doesn’t mean anyone is required to accept his mail, nor provide him with a connection to the rest of the internet.

This is a lesson Mr. Luckman is having problems learning. Instead of fixing his process so he isn’t sending spam, he contacts a reporter to plead his case in the court of public opinion. Sadly for him, most people hate spam and won’t defend a self admitted spammer against a blocking group. In fact, over 80% of the people who have voted in the “has Spamhaus gone too far” poll have said no. What’s your vote?