Is their increased vigilance pissing you off? If so, your anger is misplaced. They are reacting quite sensibly to market conditions apparently imposed by Spamhaus. Ken Magill

While I agree with Ken that the ESPs are reacting to market conditions. Where we disagree is the idea that these conditions are imposed by Spamhaus. I don’t think all the uptick in ESP enforcement and compliance activity is the result of Spamhaus’ actions. I believe that many of the mass market ISPs are changing how they detect unwanted mail, and are fine tuning filters to reduce the amount of unwanted mail that shows up in the inbox.

One of the big changes is better tools for handling huge data sets. Bigger ISPs handle billions of messages a week. Even just collecting and storing the mail is a giant task. Storing it in a useable form was almost out of the question. But over the last few years there have been significant improvements in the speed and affordability of hardware to handle very, very large datasets. Likewise, there have been algorithm and software improvements in mining that data for useful correlations.

In practical terms, ISPs and filtering companies like Spamhaus don’t have to focus on complaints or trap hits or “simple” measurements. They can draw complex correlations and look at mail in a way that was simply impossible 2 or 3 years ago. This means they can better identify senders who had previously been able to slide in under the filters.

Spamhaus rolled out tools to monitor their spam feeds in a different way and have been listing a lot more “legitimate” senders because of it. ISPs are rolling out tools to better filter “greymail” and keep users inboxes full of mail that the users actually want.

One of the trends I’m noticing is that direct marketers are getting more aggressive. Whether it’s a response to the years of recession or a response to the slowly warming economy, I can’t tell. But there are a lot of direct marketers who are no longer afraid to break the law. For instance, my cell phone is getting multiple telemarketing calls a week, despite being a cell and despite being on the do not call list. My inbox is full of unsolicited email carefully engineered to get past standard filters, much of which violates CAN SPAM. I’m even getting the occasional unsolicited fax.

The increase in listings by Spamhaus are one example of the filtering screws being tightened. But it’s not just Spamhaus that’s driving this; ISPs and filtering companies are also filtering more aggressively. I’m seeing a lot more emphasis being placed on content and a good IP reputation is no longer a ticket to the inbox. Content must be clean and recipients have to want mail for it to get into the inbox.

That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.

Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.

What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.

The URLs in the text/html parts of my message are big and ugly

Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.

(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).

The URLs in the text/plain parts of my message are big and ugly

The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.

If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.

I want to use a catchy URL shortener to enhance my brand

That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.

I want to hide the destination URL from recipients and spam filters

Then you’re probably spamming. Stop doing that.

I want to be able to track clicks on the link, using bit.ly’s neat click track reporting

Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).

So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.

I want people to have a short URL to share on twitter

Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.

If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.

All the cool kids are using bit.ly, so I need to to look cool

This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.

And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.

So… I really do need a URL shortener. Now what?

It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.

Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)

These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.

1. Make a URL shorter
2. Track clicks on the URL
3. Hide the destination URL

Making URLs shorter was their original role, and it’s why they’re so common in media where the raw URL is visible to the recipient – instant messaging, twitter and other microblogs, and in plain text email where the “real” URL won’t fit on a single line.

From the moment they were invented they’ve been used to trick people to click on links to pages they’d rather not visit, from musical classics to less tasteful content. And, in just the same way, spammers quickly found that they were a good way to avoid content-based filters or to hide a suspicious looking target URL.

Inevitably, URL shorteners that are persistently abused by spammers (especially those where that’s done with the support of the URL shortener operator) start to be seen as a sign of spam, and email that uses them will be treated with suspicion by content-based spam filters and often sent to the spam folder.

bit.ly is probably the highest profile URL shortener, so it’s the one you’ll most likely see people trying to use in email. What effects does that have?

Now being “totally owned” by the Canadian Pharmacy gang, thousands of URLs being spammed with very slow takedowns. Not good.SpamHaus on bit.ly

bit.ly have been on SpamHaus’s radar for quite a while. They’re listed on the SBL multiple times. They’re listed in the DBL – SpamHaus’s newish domain based blacklist, intended for content-based filtering of email. All this means that emails that contain bit.ly URLs are increasingly likely to have serious delivery problems.

This isn’t unique to bit.ly: many other URL shorteners have similar problems – j.mp, su.pr, and others. Nor is it unique to SpamHaus: many other spam filters, public and private, are starting to treat common URL shorteners with suspicion.

Naive use of URL shorteners in your email will send it to the spam folder.

More about why you shouldn’t do that – and what you can do instead – tomorrow.

The judge referenced a couple specific deficiencies of Holomaxx’s claims in his dismissal.

Holomaxx alleges no facts in support of its conclusory claim that Yahoo!’s filtering program is faulty, nor does it identify an objective industry standard that Yahoo! fails to meet. While it suggests that Yahoo! is “using cheap and ineffective technologies to avoid the expense of appropriately tracking and eliminating only spam email,” it offers no factual support for these allegations. (Compl. ¶ 40). Nor does Holomaxx cite any legal authority for its claim that Yahoo! has a duty to discuss in detail the particular reasons for blocking Holomaxx’s communications or to provide a remedy for such blocking.

Holomaxx did their best to identify an objective standard, they really did.

9. The objective industry standard has been set forth by the Messaging Anti-Abuse Working Group (“MAAWG”), a group consisting of the largest internet service providers on the planet. YAHOO is a member of the MAAWG and has approved of the MAAWG’s industry standard for abuse desk responses but has failed to follow the MAAWG evidencing, among other things, a lack of good faith.

10. The MAAWG sets forth an objective industry standard protocol by which its members should respond to blocked senders, such as HOLOMAXX, requesting unblocking. The MAAWG protocol states that “the simplest option would be to grant unblocking when it is requested. This alleviates the necessity of judgment calls, arguments with senders or time-consuming research. The overwhelming majority of the vote was for this approach because if the sender had not fixed the issue that caused the block it would be reinstated quickly, so damage would be theoretically minimal.” In this case, YAHOO has breached this provision of the MAAWG in that, with regard to HOLOMAXX and YAHOO’s blocking of HOLOMAXX’s emails, YAHOO refuses to  unblock HOLOMAXX’s emails.

Their filing doesn’t actually reference the document in question, so I spent some time this morning looking for which document this was. It didn’t sound like anything familiar. I eventually identified the 2007 Abuse Desk Common Practices document published by the Collaboration Committee. In the 2nd paragraph it was obvious why Holomaxx didn’t include the document as an exhibit.

The intent of this document is to present options for abuse desk administrators for addressing common problems faced by abuse desks. This is not intended to represent an absolute set of best practices. It is our belief that what is “best” is frequently determined by the particular circumstances of the network/mailboxes served by the abuse desk.

So the “objective industry standard” that Holomaxx cites explicitly states this is a document that solely discusses options and how some abuse desks currently (in 2006 and 2007) handle inquiries from senders. In most cases, the abuse desk and the postmaster desk are separate, something I’m comfortable asserting about Yahoo’s setup.

In dismissing the wiretapping claim the judge cites specific problems with Holomaxx’s complaint.

Holomaxx alleges that Yahoo! “intentionally intercepted electronic communications sent by Holomaxx, in violation of 18 U.S.C. § 2510, et seq.,” (Compl. ¶ 49), and “intentionally used and disclosed the contents of such electronic communications sent by Holomaxx, while knowing 2or having reason to know that the information was obtained through the interception of those communications in violation of 18 U.S.C. § 2511.” (Id. at ¶ 50). It claims that Yahoo!intercepted e-mails to evaluate the “spam like characteristics” of the e-mails (Id. at ¶¶ 59-60). However, while the Court must “take all factual allegations in the complaint as true,” (Ashcroft v. Iqbal, 129 S.Ct.1937, 1949 (2009)), it is not required to accept a “legal conclusion couched as a factual allegation.” Bell Atl. Corp. v. Twombly, 550 U.S. 544, 555 (2007). Here, Holomaxx’s allegations are both conclusory and devoid of factual support. Holomaxx does not explain how Yahoo! “intercepted” its communications, or how Yahoo! “used and disclosed” the contents of those communications. Additional factual detail is necessary in order to permit the Court to make a meaningful assessment of the plausibility of the allegations.

Holomaxx tried to address this deficiency.

44. Upon Information and belief, YAHOO intercepts and scans the contents of HOLOMAXX’s emails in the following manner: YAHOO uses Bayesian filtering techniques which scans the content of HOLOMAXX’s email message body and headers for words and phrases associated with alleged spam messages. YAHOO then provides the incoming and intercepted HOLOMAXX email with a score before the HOLOMAXX email reaches the YAHOO receiving server. The receiving server, armed with the score procured from the illegal interception, then decides whether or not to accept the HOLOMAXX email or reject it as spam.

45. Upon information and belief, in conjunction with the Bayesian technique, YAHOO also intercepts HOLOMAXX emails before they reach YAHOO’s servers and uses a distributed catalogue of signatures to detect spam. These signatures are generated as a result of YAHOO user submissions in the past. The HOLOMAXX emails are intercepted and the contents scanned and compared to the catalogue of past signatures. This is known as a collaborative filtration system.

46. YAHOO did not have authorization from HOLOMAXX to access the contents of HOLOMAXX’s emails in the manner alleged herein, in transit between leaving HOLOMAXX’s server and before reaching [sic] MICROSOFT’s servers.

Anyone who knows anything about email can see the flaw here. Filters don’t run on the wire. They have to go through a server somewhere, and the only way Yahoo can apply filters is to have actually received the email. Anywhere Yahoo is going to be able to filter the mail is, objectively, a server Yahoo owns or is allowed to access.  I also don’t think these three paragraphs actually explain how the mail was intercepted. It makes a lot of assertions and throws around buzzwords, but doesn’t explain how this alleged interception happened.

Holomaxx also started claiming that Yahoo sends email advertising to its users and that Y! is blocking Holomaxx to drive Holomaxx out of business and make advertisers pay higher advertising rates to Yahoo. Their proof? Screenshots of a Yahoo inbox showing banner ads on the right hand side of the screen. I think it would really help Holomaxx’s case if they could manage to not fundamentally mis-represent simple things like this. They’re ads on a webpage, they’re not sent by email.

All in all, I’m not seeing much improvement in the amended complaint. I mean, Holomaxx tried to address the deficiencies, but they really can’t support their claims. Yahoo isn’t violating the wiretapping act, they’re not violating industry standards and Holomaxx is clearly grasping at straws to try and save their company. e360 tried the same thing against Comcast and ended up bankrupting themselves. Will the same thing happen to Holomaxx?

Recently the ASRG posted a draft of a best practices document aimed at those running blocklists (draft-irtf-asrg-bcp-blacklists-07). This document has been under development for many years. The authors have used this document to share their experiences with running blocklists and their knowledge of what works and what doesn’t.

Best practices documents are never easy to write and consensus can be difficult. But I think that the authors did a good job capturing what the best practices are for blocklists. I do support the document in principle and, in fact, support many of the specific statements and practices outlined there. As with any best practices documents it’s not perfect but overall it reflects the current best practices for blocklists.

Ken Magill’s article about the BCP
Anti-Abuse buzz article about the BCP

Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 554 554 5.7.1 The IP address of web site www.client.com [75.101.163.44] is listed at www.spamhaus.org (state 18).

What? An SBL listing? The client hadn’t done anything wrong, certainly nothing that would provoke the wrath of Spamhaus. And… they’re not sending email from 75.101.163.44 anyway, they’re sending it out through Google Apps. And, wait, www.client.com is listed on the SBL – the SBL lists IP addresses, not hostnames.

What is going on here?

We’ve mentioned in passing before that one of the good ways to filter mail based on content is to look for suspicious URLs in the message. One way of doing this is to use hostname-based blacklists, such as SURBL, URIBL or DBL. These list domain names that have been seen in spam (and pretty much only spam), and sending email with a listed hostname in it is a quick trip to blocksville at many ISPs.

Spammers, phishers especially, often cycle through domain names quickly in order to avoid the (manually maintained) hostname-based blacklists. They often host them at the same place, though, so if you look up the IP address the hostname resolves to you can use an IP based blacklist to see if the hostname is being used for spam or phishing related email payloads, and use that information to block the email. That’ll work even if the phishers use an entirely new domain for their websites, if it’s still hosted at the same place.

The SBL blacklist is commonly used in this way. It’s manually maintained and fairly hard to get on to, and finding URLs that resolve to addresses listed on the SBL in an email corresponds pretty strongly to the mail being unwanted. The folks who run the SBL are quite aware of this, and will commonly list IP addresses that are being used to host websites advertised in spam even if they never send email.

What happened in this case was that the client was hosting their website with Heroku, a perfectly respectable cloud-based ruby-on-rails web host. But Heroku use just three IP addresses for all their customers. And one of their customers was zapt.in, a URL shortener with a serious spam problem. Zapt.in caused problems for long enough, and didn’t respond to them for long enough, that their IP address was listed on the SBL.

That meant that all of Heroku’s customers were using an IP address listed on the SBL. Which, in turn, meant that any email those customers (or their affiliates or customers or…) sent that used the customers domain would be rejected by ISPs using the SBL-as-a-hostname-blacklist trick – which is a lot of large ISPs.

What can you do to avoid this? The ideal is to not host your website on a shared IP address (or in a /24 that’s littered with spam and phishing sites).

If you can’t do that – you really can’t move your main website to a more reputable host – then your next best option is to not use your main website in any of the email you send. You don’t want to hide the connection (because you don’t want to look like a snowshoe spammer who’s obfuscating their domain ownership), but you want the hostname to be different. A good way to do that is, if your main domain name is example.com and your website is www.example.com, is to use a subdomain for URLs in emails. click.example.com, maybe.

Host that subdomain somewhere else, on an IP address you have a bit more control over – an inexpensive VPS or web hosting provider, and just run a web redirector there that simply sends an http 302 redirect for any http://click.example.com/foo/bar/baz.html to http://example.com/foo/bar/baz.html. And use the click.example.com form of the URL for everything you use in your email – not just links, but also image tags and so on.

What if setting up your own redirector isn’t something you have the resources to do? Sign up for a web redirector or URL shortener service that’ll let you use your own domain name, like bit.ly Pro. That won’t give you as much control, or protection, as running your own redirector but it’s a lot better than running your website on a shared IP address.

Management Summary, Redistributable Documents and Links

In the past week we’ve demonstrated that the SORBS reputation data is riddled with mistakes, poor practices, security holes and operational problems, and that the quality of the end result is really too poor to be useful.

Today I’m looking at how this information should affect your choice of spam filtering technology.

Should my spam filters use GFI/SORBS data?

Simply, NO. The quality of reputation data GFI provide is too false-positive prone to rely on in production, even as part of a scoring system.

After all, a false positive is far worse than a false negative, as far as RBL (or general filtering system) usability is concerned.@delivery_kitty

And the problem isn’t just false positives.

Because it takes a long time before a spamming IP address reliably appears on the blacklist, not much spam is stopped. SORBS appears completely unsuitable to the most common way of spamming, via botnets.abuse department at xs4all, a major EU ISP

Your ISP

If you receive mail via your ISP then you’re unlikely to have problems with SORBS blocking your mail, as very few successful ISPs will use it for blocking outright. If you’re at a smaller ISP then they may well be using spam filters such as SpamAssassin, with a dependency on GFI / SORBS data sources, though.

But it’s not worth contacting your ISP unless you find out mail is being bounced or put into a junk folder due to a SORBS listing, or if you can tell by looking at the headers of email you receive that it’s being scored against SORBS.

(If you’re concerned about use of third-party reputation sources by your ISP, you could ask them to provide – or, better, publish – a list of the data sources they use, so their customers can make well-informed decisions about their filtering.)

Your Mailserver

If you run your own inbound mailserver, make sure it is not configured to use any of the SORBS blacklists for blocking email. How to do that varies depending on the server, but for commonly used linux mailservers grepping the configuration files for the string “sorbs” is probably a good place to check.

(There are some great blacklists, with very low false positive rates, to consider using instead – for IP based reputation: spamhaus zen, spamcop, cbl – and for URL reputation: spamhaus dbl, uribl, surbl)

SpamAssassin

SpamAssassin is a widely used server-side score based spam filter. Unfortunately it seems to ship with SORBS blacklists turned on “out of the box”.

I believe that adding the following to /etc/spamassassin/local.cf will disable it – I could be wrong, and would appreciate feedback from any SpamAssassin experts out there.

score RCVD_IN_SORBS_BLOCK 0
score RCVD_IN_SORBS_DUL 0
score RCVD_IN_SORBS_HTTP 0
score RCVD_IN_SORBS_MISC 0
score RCVD_IN_SORBS_SMTP 0
score RCVD_IN_SORBS_SOCKS 0
score RCVD_IN_SORBS_WEB 0
score RCVD_IN_SORBS_ZOMBIE 0

The default SpamAssassin scores are pretty low, so it doesn’t pay that much attention to SORBS – but that a spam filter as influential as SpamAssassin uses such a poor source of data at all is a bit of a problem. Hopefully the SpamAssassin developers will look at the issue for a future release.

Commercial Products

If you’re using a commercial spam filter, check where they’re getting their reputation data from. If you have an existing commercial filter that can use external blacklists, make sure it’s use of SORBS is disabled.

If you’re considering purchasing a new commercial spam filter, there are two things you need to consider. First, if the filter supports using SORBS or other GFI-derived reputation data make sure that can be disabled. Second, if you’re considering a commercial product that uses SORBS or GFI data out of the box, despite the multi-year history of false positives and other problems, think about how solid their other product engineering decisions might be.

I for one will not be considering any products from GFI. I had budgeted and received approval for $20K worth of GFI NSM next year. I will not be making that purchase after this latest episode with SORBS.Skyhawk

Outsourced Services

Outsourced spam filtering services can be very opaque about what approaches they use to decide whether or not email is spam, and will often hide their use of external reputation services.

Some of them are more open than others. Proofpoint posted SORBS DUHL DNS Block List Causing Widespread Email Deliverability Issues Once Again (note that GFI told Proofpoint the problem was fixed on Nov 30th, which we know isn’t true), but it’s rare that a SaaS provider will be that open about how a problem is caused by their reliance on a third-party service. Kudos to Proofpoint for their openness (though they should look elsewhere for reputation data).

Edit: Proofpoint have clarified that they were discussing the problems some of their customers had sending email due to false SORBS listings – not that they were using any data from GFI themselves. Sorry, guys. So if you’re looking for a filtering appliance or outsourced service that’s GFI/SORBS-free (and also quite a nice product), Proofpoint is worth a look.

If your SaaS or outsourced spam filter provider has a clear statement in their product description or contract which third-party data sources they use, then you have the information you need. If not, you should probably contact your support representative and find out whether they use SORBS or not. If they decline to make any statement on it, assume the worst.

In the case of SORBS, this is (at least) the second major misclassification issue we’ve observed in the last 90 days. Email administrators who currently rely on SORBS should be aware of these issues and take action as necessary.Proofpoint

There’s nothing wrong with an outsourced provider using reputable third-party services but if they’re relying on poor quality data sources you may find mail to you being bounced for no good reason, at any time. If that’s the situation you’d be well advised to consider looking at alternative filtering providers.

And Finally

There’s a lot more that could be said, but I’m sure you’re interested in seeing some non-GFI/SORBS content on this blog (and there’s a limit to the amount of technical and business analysis I really want to do for someone other than a paying customer).

Laura will probably revisit the subject soon, going into some more detail about the policy problems that I just touched lightly on and looking more generally about what other companies can learn. And I know several other industry bloggers are planning on discussing GFI and SORBS in the next week or two.

I’ll be gathering links and some other information, including a PDF version of this series of articles suitable for mailing out, at http://wordtothewise.com/sorbs/ over the next day or two.

Management Summary, Redistributable Documents and Links

In the past week we’ve demonstrated that the SORBS reputation data is riddled with mistakes, poor practices, security holes and operational problems, and that the quality of the end result is really too poor to be useful.

What does this mean to you though? There are really two aspects: 1. what to do if you’re blacklisted or blocked by GFI or based on GFI/SORBS data and 2. how this information should affect your choice of spam filtering technology. We’ll be looking at the first point today, and the second tomorrow.

I’ve been blocked by SORBS! What should I do?

1. Don’t Panic

First, don’t panic. Just because you’re listed on SORBS it doesn’t mean it’s having much, if any, effect on your email. (When we last measured the impact of a SORBS listing, it was responsible for about 0.01% of mail rejected – not 0.01% of the mail sent, but of the mail that was rejected about 1 in 10,000 rejections appeared to be due to SORBS.)

Different people sending mail to different recipients will see different impact from any given blacklist. So you need to look at whether your mail is being rejected. If you’re not seeing problems with mail being rejected, the listing is not something you need to care about.

2. Check to see if you’re really listed

Next, see if you’re listed on the SORBS blacklist. Find the IP address of your outbound smarthost – perhaps it’s 10.11.12.13. Reverse the order of the numbers, and put “.dnsbl.sorbs.net” on the end to give something like “13.12.11.10.dnsbl.sorbs.net”. Open up a command prompt (on Windows do Start -> Run… and enter “command”) and use nslookup on that string:

C:\Steve>nslookup 13.12.11.10.dnsbl.sorbs.net
Server: i
Address: 192.168.80.100

i can't find 13.12.11.10.dnsbl.sorbs.net: Non-existent domain

What you’re looking for is “Non-existent domain” or “NXDOMAIN”. If you see either of those, then you’re not listed on SORBS.

If, instead, you see “timed out” or “SERVFAIL” then SORBS is broken, and you can’t tell.

If you see something near the end starting with “127.0.0.” then you probably are listed on SORBS:

C:\Steve>nslookup 13.12.11.10.dnsbl.sorbs.net
Server: i
Address: 192.168.80.100

Non-authoritative answer:
Name: 13.12.11.10.dnsbl.sorbs.net
Addresses: 127.0.0.10

You can tell which SORBS list you’re on using the table on this page. (If the SORBS website is down then the two interesting values are 127.0.0.10, which means you’re listed as a dynamically assigned address, and 127.0.0.6, which means you’re listed as a spammer).

3. See if there’s any more data on the website

Check the GFI/SORBS website to see if there’s any more information available: http://www.sorbs.net/lookup.shtml

4. Is the GFI/SORBS listing causing the blocking?

By now you know that you are having mail rejected, and you are listed on SORBS. Those two things may not be connected, though. Can you send mail to, for example, AOL, Yahoo and Gmail? None of those ISPs use SORBS, so if your mail is being rejected there, then you have some sort of problem that is not related to the SORBS listing, and need to look at that.

I’ll assume that it’s a false listing, but you should check the SORBS FAQ to see if it’s a legitimate listing.

5. Work with the ISPs that are rejecting email

This is not just a GFI problem. Many mail server admins use the SORBS Dynamic IP list in their list of RBLs, that are not GFI customers. How do we get mail server administrators to understand that SORBS is broken and to disable it?comment from yesterday

If you’re only being being blocked by a small number of recipients using SORBS then the best approach is to contact the administrators at those sites, explain that it’s a bogus listing, and ask them to whitelist your IP addresses. Maybe they’ll stop using SORBS altogether if they get too many of those requests. Sometimes, if the administrators are belligerent that you must be spammers because SORBS says so for example, there’s nothing you can do and you should just write those recipients off as incompetent to run email and not worry about it too much.

6. Work with GFI to get delisted

If you decide that the right thing to do is to get GFI/SORBS to remove the false listing then prepare yourself for a long slog. I’ve seen clearly false listings kept up for several years, and even simple delistings can take months to resolve.

The SORBS website encourages users to handle delisting requests via this link. As we’ve explained over the past few days, that’s not the best idea:

• Using that link as recommended will compromise the security of your machine by loading an untrusted SSL certification authority
• The approach SORBS use to handle inquiries is designed to punish those who ask questions about a false listing by extending the listing, not responding to queries and pushing a delisting request to the “back of the queue” any time a question is asked
• The ticket queue software is designed by the same people who designed the rest of the SORBS infrastructure so isn’t going to be any more reliable
• Some of the things that GFI employees running SORBS require to get delisted are painful and expensive to do, as well as being pointless – some of their DNS requirements in particular are the IT equivalent of dancing three times widdershins around a sacrificed goat
• Even if you do manage to get a false listing removed, it’ll just be added again the next time the database is reloaded.
• The staff handling that queue are not professional support staff, rather they are the same people who developed SORBS. Quite apart from the other problems you’re likely to have interacting with them, they’re the least likely people to be responsive to a problem caused by their own mistakes.
• There’s no record of your request in any real ticketing system, so there’s no GFI management visibility into responsiveness metrics

DEAR GFI: There is no way you could find a more incompetent set of people to run a RBL, or anything for that matter, regardless of how hard you might try.Skyhawk

GFI do have professional support staff, though, and they should be able to help with problems with their reputation products, including the SORBS blacklist. They have local contact numbers and addresses for many countries across the world listed on their contact page.

At the time of writing their US contact information is:

I’m told that the first tier GFI support folks would rather not deal with SORBS and will push callers to use the SORBS ticketing system instead, so you may need to be persistent or escalate requests.

Good luck!

More tomorrow.

Management Summary, Redistributable Documents and Links

In the last few days we’ve talked about GFI’s lack of responsiveness, the poor quality of their reputation and blacklist data, and the interesting details of their DDoS claims. Today we’re going to look at (some of) the fundamental problems with GFI’s procedures and infrastructure that cause those issues. Some of the subset of issues I’ve chosen highlight are minor, some are major, but they show a pattern of poor decisions.

SSL Certificates

When you use SSL on a web connection it brings you two benefits. The first is that it encrypts the connection between your browser and the webserver, so that it’s very difficult for anyone to watch or tamper with your interaction with that webserver. The second, more important, reason is to make sure that you’re talking to the webserver you think you’re talking to, to avoid man-in-the-middle attacks.

This security relies on you trusting the certification authority that issues the SSL certificate that the website uses. A website providing services to the public should always use an SSL certificate created by one of a small number of reputable certification authorities that are pre-loaded into all webservers as “trusted”. These SSL certificates are something that need to be be purchased, but they’re very inexpensive – less than ten dollars a year.

One more amazing thing is about the self-signed ssl certificate used by SORBS ! It look’s like a poor homeless website ! A verisign certificate can cost 500 bucks for one year, but I’ve got one inexpensive from Trustico for 16 / year and there is no trouble with all the current browsers and operating systems. It looks like SORBS was running an old 386 server in a garage, not like a worlwide operating service. This is not serious at all !Laurent Marandet

For completely private webservers used by a well-defined group of people – an enterprise web service used solely by company employees, for instance – you might instead have your IT group set up a private (“self-signed”) certification authority and have your employees configure their web browsers to use that. Doing that is a very bad idea for public webservers for three major reasons:

1. It requires the user to perform a complex setup to load the information into their browser, rather than just visiting the website and being secure.
2. It provides no real authentication that you’re visiting the site you think you are, rather than some man-in-the-middle imposter. Why? Because you’re downloading the certificate that provides that security from the what you think is the same website it’s protecting. You just need to be tricked to visit the imposter website once, or have your web session hijacked in some way, and the imposter can have you download *their* private certification authority certificate instead – and then you’ll think you’re accessing the real website, protected by SSL, when you’re really accessing the imposter.
3. Worst of all, if a user can be persuaded to load a private certification authority certificate into their browser then the people who run that private certification authority can create “fake” SSL identities for any website they like.

That last point is very scary when you think about it. If you as an end-user can be tricked into loading a certification authority into your browser then the owner of that private authority can do almost undetectable phishing or man-in-the-middle attacks against you. They can create a fake citibank.com or microsoft.com website, and fake SSL certificates to match, meaning your browser will tell you that you’re securely talking to your bank.

SORBS tries to get people to install their private certification authority certificate into their web browsers. There are all sorts of reasons a malicious website might do that, while the only reason a legitimate website would do that would be to save the $8.95 a year it would cost to use a real SSL certificate. Given that GFI uses very high-quality, expensive ($488/year), “green bar” SSL certificates on their other web properties that doesn’t seem likely.

So why is GFI/SORBS trying to get listees to install an untrusted certification authority into their browsers? The only explanation that doesn’t imply malicious intent is a deep ignorance of basic security engineering – which doesn’t inspire confidence in the safety and security of their data. (Maybe there really are hackers breaking into GFI machines and adding random false positive SORBS listings?)

Remote Hands / Remote Access

GFI regularly add huge swathes of address space to their SORBS blacklists wrongly (many millions of addresses, and a noticable fraction of the entire internet). Even when they acknowledge that it’s a bad listing they’re often not able to fix it for weeks or months due to “database problems” or “DDoS attacks”.

some people are morons “anyone can blame a mistake on a DDoS” … the problem is the mistake was corrected but the DDoS is preventing the correction from getting to the real world.GFI Statement on failure to resolve false listings

During those periods that the SORBS website is unavailable for some reason, the DNS servers that actually publish the GFI/SORBS blacklists seem to keep running with no problem.

SORBs DNS servers were responding to RBL requests during the whole time, albeit somewhat slowly. IT WOULD HAVE BEEN BETTER IF THEY HAD JUST TIMED OUT!!! Then we wouldn’t be here writing about it. But they did not time out, and worse they answered RBL requests with bad replies.Skyhawk

I’ve seen other blacklists have data problems that have caused serious false positives, and the first thing they’ve done is to remove those false positives from the data they’re publishing, even if they had to do so in a crude, broad-brush manner – in extreme cases I’ve seen them completely empty the published DNS zones until the problem was fixed. GFI isn’t doing this – and they say the reason they’re not fixing the data is that they can’t due to the DDoS or database problem.

The only way that can be true is if they have no out-of-band or remote hands access to their name servers – which is not a safe way of running mission critical internet services. Word to the Wise runs a number of servers at several locations and, while none of them are “mission critical” either to us or to our customers, they are “business critical” such that them being unavailable for more than a few hours would cause a business problem. Off the top of my head, we can access most of them in any of the following ways, even if our main office network is off the air due to a T1 cut, a power outage or even a DDoS:

1. Connect to the publicly visible IP address from anywhere there’s network
2. Ssh into bastion host, then ssh to production server over private maintenance network
3. Ssh into terminal concentrator, connect to server via serial port
4. VPN into production location from laptop at a coffee shop, use virtual machine management software to log in to production virtual servers (or reboot them, or even rebuild them from scratch)
5. VPN into production location from my iPhone, do any of the above
6. Ssh in to remote power strip and kill the power to one or more of the production servers, either to reboot them or take them off air
7. Dial in to terminal concentrator, do any of the above
8. Dial in to remote power strip, reboot or power off machines
9. ‘Phone staff at the colocation facility and have them power-off or power-cycle machines, or type commands into their keyboards
10. ‘Phone staff at the NOC, have them drop network connectivity to one of my IP addresses, to disable that server
11. Modify my DNS configuration, to point services to another machine running elsewhere
12. Modify my DNS configuration, to take a service off the air
13. Drive to the colocation facility and do whatever is needed in person (there’s someone trustworthy  within less than an hours drive of each of our facilities)

If GFI had even one piece of that infrastructure in place, they’d be able to fix problems in the data they were publishing via DNS even if their main location were DDoSed into oblivion. If we trust their assertion that they cannot do that, then they do not have enough basic infrastructure in place to run any sort of public facing internet service, let alone one that’s providing a mission-critical service to external users.

If you can’t fix your data, you need to turn off your dnsbl name servers. You are currently harming mail administrators around the world in a big way.Hans

Lack of Safeguards

The mistakes GFI make with the SORBS blacklist vary from incident to incident somewhat, but there’s a lot of repetition. Loading ancient, years-old, database dumps into the system, then publishing them without any checks seems to be one of the common failure modes.

My company is directly affected by this, for the second time in two months.anonymous commentor

People will often forgive a single mistake, but if you make the same one over and over again they lose all patience. It would be easy enough to put some automated checks into place, in the step between a new set of data being commited to the database and that data being published to the production DNS servers.

Even some of the most trivial checks, that could be implemented in a couple of hours in perl, would have prevented the (repeated) high-profile dul.dnsbl.sorbs.net errors. GFI clearly haven’t added even those most basic checks, let alone anything more sophisticated such as separate staging areas, backup databases, sanity checking against internal or external whitelists.

That either means that GFI have chosen not to invest any development resource into data and system integrity – or it means they have, but the system engineer responsible for it is not competent to do so. Either way, it casts serious doubt on their ability to maintain any level of data integrity either on their external DNSBl blacklists or their internal reputation system (which they bought out the SORBS assets to implement).

Naivete about how Internet email works

Policy problems are just as much of a problem as implementation mistakes.

I’ve concentrated mostly on the “dul” (dynamically assigned) blacklist, as it’s easier to demonstrate it’s failures. But there are similar problems in other zones.

If you’re running a mailing list, one recommended practice for signing up new users is confirmed opt-in. This is about as solid a best practice for mailing list signups as you can get, and is evangelized by legitimate anti-spam organizations such as SpamHaus or MAAWG as one of the best ways to run a mailing list.

This is the standard Best Practice for all responsible Internet mailing firms. COI ensures users are properly subscribed, from a working address, and with the address owner’s consent. [...] This simple protection means that the Bulk Email sender can not be legitimately listed on any ‘spam’ blocklist
SpamHaus on COI

Some years back Laura was working with a mailing list operator who’d been listed on the SORBS spam blacklist. That seemed strange, because they were a 100% Closed-Loop Opt-In mailing list, and so there was no way they should have been listed on any ‘spam’ blocklist. On investigating it turned out that a user, something@cox.net had tried to sign up for the mailing list, but had typoed their email address to something@cix.net – just one letter off on the keyboard. The mailing list manager sent the opt-in confirmation request to something@cix.net, nobody responded to it, so something@cix.net wasn’t subscribed to the list or sent any mail from the list. A couple of hours later the user noticed, and signed up again with their correct something@cox.net mailing address and were subscribed successfully. This is all exactly how closed-loop opt-in is supposed to work.

However, the owner of cix.net had given control over the domain to SORBS for use as a “spam-trap” domain. And SORBS used a single confirmed opt-in request to an address in that domain to blacklist the mailing list operator. That’s a very poorly thought-out way of using a spam-trap domain, and leads to serious data integrity and false-positive problems with any blacklist based on such a naive spamtrap-driven approach.

That was a few years ago, but there’s more recent behaviour that’s pretty much the same.

I’ve got some really nice data showing how SORBS lists you for hitting a seed address of theirs ONCE 24 hours after they purchased the domain which expired. The user who previously owned the address in question COI (confirmed opt-in) into a list 8 months prior and had shown opens and click throughs within a couple months of the domain transfer. This is just an example of how one of their other zones uses terrible data as the basis for a listing.mailing list operator

I’ve seen a similar thing. In November 2009 someone signed up for a mailing list – full closed-loop opt-in, everything. Mail was sent out to the mailing list regularly, and we know the recipient was reading the mail – they were clicking on links in the messages, interacting with the mail, all the sorts of things a typical happy recipient will do. Judging from the website recorded at archive.org it was just a personal domain owned by the recipient, though, and at some point they let the domain registration lapse.

Two days after the domain registration lapsed, another email was sent out to the mailing list. By then GFI had acquired access to the domain, and were using it as a spamtrap domain. The mailing list was blacklisted on the GFI/SORBS spam list due to that single, non-spam email. Even the most aggressive blacklist experts agree that you need to leave any domain that’s been previously used for legitimate email in a state where it bounces email for at least six months to a year before you can really pull much useful data out of deliveries to it.

Industry best practice is to not remove an address from a mailing list until it’s bounced at least three times, over at least a 15 day period. There’s no way at all any mailing list (or ISP smarthost or any other source of email) could avoid being listed by the SORBS spam list in this way.

Given the way the data is acquired, the quality of data in the GFI/SORBS blacklist is likely to remain poor, and to be full of false-positive listings of this type.

Database Design, Audit Trails and Broken Data

I pulled a copy of the full record for the false dul.dnsbl.sorbs.net listing of 67.194.0.0/15 on Thursday. I also have the equivalent data for a (valid) listing in the SpamHaus PBL of a Comcast cable modem pool to compare.

Spamhaus PBL listing of 98.224.0.0/12 – this is a good example of the data that needs to be tracked. It shows that the /12 is listed because Comcast, the owners of that address range, don’t want their users to send email from there directly.

Compare that with GFI/SORBS listing of 67.194.0.0/15 (this is a twenty page PDF file saved from the GFI/SORBS database lookup page)

The GFI/SORBS data is… I don’t really have any words for it. I hate to think how badly the database is designed that could even have data of this structure in it.

For those who’ve not looked at the PDF, it contains entries for six address ranges – 67.192.0.0/10, 67.192.0.0/11, 67.192.0.0/12, 67.192.0.0/13, 67.192.0.0/14 and 67.194.0.0/15. The first five of those are listed as “Unknown Status”.

The last, 67.194.0.0/15 is described as “101803411-16-IP/Netblock delisted” – suggesting that the address range has been delisted. But it’s also described as “Currently active and flagged to be published in DNS”, meaning that it’s being published in the blacklist.

There’s also an audit trail, of sorts, showing what’s been done to these records. There are something over 220 entries, all identical apart from the timestamp:

“Delisted by 10 – Comment made by [10] Michelle Sullivan at <timestamp>”

It appears Michelle has removed this data from the dul.dnsbl.sorbs.net database over 200 times, and yet it’s still “active and flagged to be published in DNS”.

I’d like to grow eloquent on all the horrible database design and data integrity implications this data dump has, but I really don’t know where to start. So I’ll stick with saying that if you have any intent of using any reputation data from GFI/SORBS for any reason, you should take a look at the PDF, show it to a friendly DBA and between you try and work out what sort of system and database setup would lead to that sort of data.

More tomorrow.

Management Summary, Redistributable Documents and Links

I’ve been stage-managing for a production of The Nutcracker this week, so musical terminology is on my mind. In opera, the intermezzo is a comedic interlude between acts of an opera series.

This comedic interlude is about the “DDoS” – a distributed denial of service attack. What is a denial of service attack?

… an attempt to make a computer resource unavailable to its intended users.

One common method of attack involves saturating the target machine with external communications requests, such that it cannot respond to legitimate traffic, or responds so slowly as to be rendered effectively unavailable.Wikipedia on DoS attacks

That’s pretty much what we’re discussing here. There are a variety of ways to mount a DoS attack but by far the most common, and the sort that’s characterized by descriptions of “gigabits per second” or “packets per second”, is simply sending high volumes of network traffic aimed at a webserver or network routers the webserver relies on. The network traffic might be pure garbage, or it might be valid web requests, but the goal of the attacker is to overwhelm either the server itself or, more commonly, the network pipe connecting the server to the internet such that it can’t provide it’s service to the public. At it’s most basic level the symptom of a DoS attack is that you can’t reach any web page on the server that’s under attack.

(What’s a distributed denial of service attack? It’s an implementation detail – the attacker uses multiple machines to attack simultaneously, enabling them to provide more attack traffic and to make that traffic a little harder to block.)

So that’s what a DoS attack can do – make a service unavailable. What can’t a DoS attack do? It can’t make any changes to the server(s) under attack. It can’t deface a web page. It can’t cause a web service to give wrong answers. It can’t corrupt information stored in a database. It can’t cause a blacklist to add false listings. That last point is fairly important – no DDoS against any blacklist infrastructure can cause it to add false listings.

How does a DDoS insert bad records into your blacklist? That is the real issue. If that would stop happening, then a DDoS would have no impact on removal of the bad records.insightful comment on Fridays post

And yet, GFI/SORBS has blamed their operational problems such as false positives, publishing of stale data, refusal to delist addresses and so on on “DDoS attacks” sufficiently often that it’s a running joke in the email industry.

Once again SORBS has reactivated old DUL-Listings, e.g. for 85.25.230.x. This happened back in october as well, and that time you also claimed DDoS.From a comment on Friday’s post

So lets look at the evolution of the latest SORBS incident. I don’t usually pay that much attention, as SORBS listings don’t affect me or my customers at all, but this time around I was researching this series of posts so I watched what was going on reasonably carefully.

On Thursday the www.sorbs.net webserver was reasonably responsive, static pages and files were returned fairly quickly, but pages that needed to access the backend database (for account creation, authentication etc.) had some problems. They were either throwing database errors (perl or PHP, apparently, and the errors looked like race conditions or invariate violations caused by page reloads) or the scgi web application process was taking too long, causing the webserver to return a “500 Internal Error” page. Reloading would (eventually) get the page. All of this behaviour will be very familiar to any web developer who’s messed up their database design to the extent that the queries needed to render a web page take too long. There was no sign of any network or webserver level problems, certainly no obvious symptoms that looked like any sort of DoS, just a database that wasn’t returning data quickly enough. If GFI were doing batch database operations against the production database, as part of trying to fix whatever was going on, that could cause the database to be sluggish, but it could also be caused by a huge number of people trying to log in (to try and get their false listings removed) or just poor database design.

I’ve been trying to create an account on Sorbs to request a delisting (from DUHL), but I keep getting an error, with perl error code echoed to screen, after waiting for minutes for the register an account to process.another satisfied commenter

On Saturday the situation changed entirely. I was unable to access the www.sorbs.net website at all (though the corporate gfi.com website was fine). That change in “DDoS” behaviour seemed very strange, and the timing (relative to GFI employees noticing that I was using the sorbs website to investigate the false listings) seemed rather convenient, so I looked in a bit more detail.

The sorbs webserver is hosted on five separate IP addresses (which one you end up at will be picked semi-randomly). That’s quite a lot – most websites, including the GFI corporate website, are hosted on a single address and even facebook.com only needs three.

None of those five addresses are in the address space allocated to GFI, rather three of them are in 111.125.160.128/26 – address space allocated to Matthew Sullivan personally while the other two are in 208.43.0.0/16 – address space assigned to softlayer technologies, most likely colocated servers or virtual servers being rented by GFI. (Note: This Matthew Sullivan, and the Michelle Sullivan who commented on Friday’s post are the same person, the GFI employee who founded SORBS originally and, to the best of my knowledge, still operates it.)

If the web server cluster were under a sustained DDoS I’d expect the five addresses to be attacked in the same way. Yet the symptoms were quite different. The three servers directly controlled by Matthew Sullivan (111.*) were completely unresponsive – no packets were being returned, 100% packet loss. The two softlayer hosted servers (208.*) were responsive at a packet level, accepting connections immediately on port 80 but not returning any results at the http level.

The behaviour of the softlayer servers is hard to explain in a DDoS related way, particularly as it was repeatable from several different networks. If all the apache sessions were occupied but there were still space available in the kernel level accept queue, that would explain the symptoms – but that would require an implausibly careful DDoS.

If the sorbs web application were broken in a way that it hung or crashed even on trivial static page requests then I’d expect the webserver to time out the app and return a 500 “I’m Broken” error, which it didn’t seem to do. That also wouldn’t explain the different behavior of the 111.* servers and the 208.* servers.

If the 208.* servers were pure proxy servers that just tunneled all web requests through to the 111.* servers that would explain what’s seen – a request to the 208.* server is accepted, then forwarded to the 111.* servers, which just hang. This would be an implausibly badly designed network architecture (it adds two servers which do nothing but add bandwidth costs, increase latency and reduce system reliability) but it’s just barely plausible. Given there’s no obvious packet level issues connecting to the 208.* servers, though it would require that the anonymous ddosers don’t bother attacking the 208.* servers as they know they’re “fake” servers and need not be attacked to take www.sorbs.net off the air.

Shouldn’t any competently-run DNSBL have plans in place for handling DDOS attacks? Why is Spamhaus stable but SORBS down relatively often?@delivery_kitty

So what else could explain the differences in behavior between the two sets of servers? The 111.* servers are in network space assigned directly to Matthew Sullivan, and he presumably has full, network level control over them, while the 208.* servers are hosted, so GFI may have a different level of access, perhaps mostly at an application or control panel level.

There is one explanation for the symptoms that explains the odd behaviour seen, and also explains the “elephant in the room” – that the degraded website behavior tends to appear soon after there’s been a rash of false data added to the database.

Or – excuse my wild speculation here – but maybe it’s not actually a DDOS attack against SORBS, but mistakes on the part of the operators?@delivery_kitty

If I were to fake up the symptoms of a DDoS where I had complete control over the network I’d pretend that I was being “packeted to death” by gigabits of traffic, and configure my router to drop all inbound packets. That would simulate reasonably accurately the effects of a massive DDoS, and also be the defensive approach you’d put in to place to defend against a real DDos. it’s exactly the behavior I see from the natively hosted sorbs web servers in 111.*.

If I only had access to the web application level (either because I was running on a hosted server, or if I didn’t have sufficient private control over the server such that I could create packet filtering without notice) the best I could do would be to make the web application hang, and possibly configure the webserver to have an extremely long timeout. That wouldn’t simulate a DDoS particularly well, but it would be good enough to convince anyone who were just using a web browser rather than looking at the lower level traffic. It’s exactly the behavior I see from the softlayer hosted sorbs web servers in 208.*.

Every time when SORBS makes just another mistake, you claim DDoS for either the problem or your inability to fix the mistake. And because you claim this every time, nobody believes you any longer.Hans

I’d be loath to suggest such a theory, even though it’s the most plausible explanation of the symptoms I’ve seen if it weren’t for the reputation sorbs has of having “convenient” DDoS attacks to explain false positive listings (which, as we explained earlier, cannot possibly be caused by any sort of DoS attack). Additionally, even though GFI were claiming to have been under a DDoS since early last week when they loaded millions of false positives into their database, the SORBS webserver had been up and basically functional, if slow. Shortly after a GFI employee – the same employee who has direct control over the 111.* webservers – commented on my blog post on Friday that explained I was looking into data inaccuracy, the “DDoS” symptoms changed to something entirely different, something that prevented me from looking at further SORBS data. Given SORBS history with respect to “DDoS attacks” I’m suspicious of both the timing and the details of the symptoms.

Fortunately, I’d actually gathered most of the data I needed for tomorrows post by Friday, so the “DDoS attack” didn’t really inconvenience me anywhere near as much as it did all the postmasters trying to investigate and resolve SORBS false listings.

the last response I got back … was that the entire /24 block was ‘inelligible’ for de-listing. The parent company sites a DDoS attack as well and says their management team is aware of the issue and working to resolve it ASAP. We’ll see what happens…anonymous commenter

GFI would benefit from some transparency about their processes, SORBS day-to-day operations and details about the mistakes they’re making, how they’re fixing them and how they’re ensuring they’re not repeated again and again. And some explicit details about exactly what sort of “DDoS” they’re seeing might help them gain some credibility. This level of communication isn’t helping with that.

More tomorrow.