Word to the Wise

Earlier this week DirectMag published an article talking about RoadRunner blocking images by default. I did talk to someone over at RoadRunner and found out a few more details about this change.

What is happening is that RR is rolling out a new web interface. This interface has both a bulk/spam filter and has images disabled by default.

I do not expect sender to notice this change in the open rates of RoadRunner addresses. Most RoadRunner customers use their own mail client (Outlook, Outlook Express, Thunderbird, etc.) and not the RR web interface. The number of uses this change touches is a very small fraction of the RR users.

No Comments »

Yahoo is aware of the recent problems and have been working feverishly to fix them. A Yahoo employee posted to a mailing list earlier today, explaining some of the recent issues. The summary is:

1) The Yahoo delays are a result of a tighter spam filtering policy. The delays are the result of the system erroneously recognizing email as spam and deferring delivery. They do believe that retrying long enough will result in all mail being delivered to Yahoo recipients.

2) They have been continually making fixes to the system over the last few days and senders should see queues start to empty over the next few hours.

3) They believe the adjustments made will resolve the deferral problems. If you continue to see problems, you can contact them through the form at http://postmaster.yahoo.com/.

4) They are working to provide more self-serve information at http://postmaster.yahoo.com/ as well as timely service updates.

Loose ends from my previous Yahoo posts:

1. The rumors of an attack were just that, rumors.
2. The Yahoo blog post about outbound servers is unrelated to the problems seen by senders recently. Outbound SMTP servers are not the same as the MX machines.

Good news all around. Thanks to the people at Yahoo for working so diligently to fix the problems.

5 Comments »

A number of people have posted to various mailing lists and made blog posts pointing to the Yahoo Mail blog post discussing recent problems Yahoo was having with mail. The general feeling seemed to be “AHA! That’s what is wrong!”

Unlike many of my peers, I do not think this explains the delivery problems senders have been seeing while attempting to deliver mail to Yahoo. The Yahoo mail blog article is talking about the Yahoo outgoing mailservers (smarthosts) for their non-webmail users. It is extremely unlikely that these are the same servers used for incoming email.

While I sympathize with everyone who had the AHA! moment and thought their delivery problems were being acknowledged and addressed by Yahoo! I do not think this is really what that blog post is saying.

I am hearing from people that Yahoo is aware of a problem with delayed incoming email, and they are working on fixing it. This does seem to be a broader problem than just bulk mailers, I am hearing from small and mid-size ISPs that they are having significant problems delivering email to Yahoo, too.

For more information about what Yahoo is doing to filter mail check out my previous post Greylisting: that which Yahoo! does not do.

2 Comments »

You may have noticed increase in delays and rejections from Yahoo. I am certainly seeing a lot of customers complaining and hearing a lot of other delivery people commenting on problems getting mail into Yahoo. I have even heard from multiple ISPs that are struggling with full queues and delayed email.

No solutions or suggestions right now, just that everyone is having problems right now. I expect it will take some time for the backlogs to dissipate, even after the underlying problem is fixed. If I hear anything more I will post it here.

8 Comments »

Last week I posted about a ESP that had a misconfiguration in their Ironport A60s that let spammers use the A60s to relay email to AOL. Earlier this week, Pat Peterson from Ironport approached me to talk about the problem and clarify what happened.

Ironport has provided me with the following explanation.

As part of a normal implementation a customer has a number of options they can choose from to setup their system.  How they handle messages on both their public and private interfaces can be easily configured based on the needs of their business.  IronPort has made sure that customers have the capability they require to configure their systems appropriately.  As with most products and services, proper usage is require for desired results.

In this case a configuration that is appropriate for some environments was viewed as a vulnerability because it was not appropriate for this specific environment.  Customers are warned about not configuring their systems to allow unauthorized domains to relay mail.  There are legitimate reasons why someone would want to allow external hosts to send mail through their IronPort, and functionality exists that allows customers to configure their systems to do that.  It should be noted that allowing any external host to relay mail through the appliance is neither encouraged nor recommended by IronPort.

In fact, the user documentation and CLI both clearly warn customers about the risk of allowing external hosts to relay mail through their IronPort.

Our customers are specifically warned during setup not to configure their systems to allow “any host to relay mail through your server.”  If someone purposely ignores these warnings and instead chooses to setup their IronPort to allow unknown domains to relay mail they can make themselves appear as an open relay, but that’s certainly not how the system was designed or intended to be used.

There is a broader message here. Anyone using any MTA needs to understand the configuration options as presented and make sure that the configuration does not allow unauthorized relay. In this case it was an Ironport MTA, but it could happen to any other MTA out there. Everyone should test their MTA after any configuration change to verify that it will not function as an open relay.

1 Comment »

JD had a comment on my Valentines day semi-fluff post, that really summed up the reality for senders. He said

Make sure your mail doesn’t look anything like spam — not just in the text and formatting, but in all of your mailing practices.

Good advice, your mail will not be blocked if it does not look like spam. What kinds of things do I mean? Here are things that spammers do, that often non-spammers do as well.

Ignore bounces. One of the absolute quickest ways to get blocked is to keep sending mail to non-existent addresses. Purge your lists, make sure you are removing those addresses that will never deliver.

Hide contact information. Do not use a domain privacy service, put your real business address in your whois records.

Fake contact information. Do not use blatantly fake information in your domain registration. Register your actual business address. Do not use 555-xxxx phone numbers.

Use free or very low cost vendors. Do not use free or advertising supported vendors for your webhosting, mail hosting, or DNS. Geocities hosted webpages, mydyndns.org hosted name servers, freemail addresses (aim, gmail, hotmail, yahoo addresses), these are ways spammers get around blocks.

Shift IP addresses. If you get an IP address blocked, for any reason, do not just start mailing from another IP. Figure out what the problem is and fix it. Skipping around blocks is what spammers do.

Mail from many different places. Do not send emails from a diverse set of IP addresses located all over the world. Spammers spread their sending out in order to dilute their spam metrics to avoid threshold based blocks. They have done this so often there is even a term for it: snowshoeing.

Use bad HELO values. Many botnets and spam infected windows machines use badly formatted or incorrect HELO values. Use a fully qualified domain name, in your domain, for a HELO value.

Use generic rDNS. Set a reverse DNS value for your IPs that does not contain the IP address or otherwise look programatically assigned.

Use incorrect HTML. Spammers hide text and use fake HTML tags in order to avoid content filters. Consequently, filters check HTML against the HTML specification.

Send different HTML and text in multipart/alternative email. In addition to using badly formatted and fake HTML, spammers put drastically different text in the text portion of HTML emails. Filters check for this and if too many differences between email parts makes mail look like spam.

Send no text part in HTML email. Spammers do this to avoid the above two filters. Do this and you look no different than they do.

Use multiple corporate identities. If you have separate divisions or brands that is one thing, but often spammers set up completely separate companies and conceal the relationship between those companies.

All of these things are spammer tactics meant to confuse, fool, deflect and avoid filtering mechanisms.

How many of them does your company do?

No Comments »

There comes an inevitable point in some of my longer term consulting gigs where my client asks me some version of the following question:

I still get spam in my inbox, so why is the email I send blocked?

So what is your best answer?

3 Comments »

It has been a rather busy day today, I do not have a full blog post. I did see a couple posts come across my RSS feeds. Both of them have content I want to talk about and discuss in a little more detail, as I think they touched on some very interesting issues.

Network World has an article interviewing Mark Risher from Yahoo. The article discusses Yahoo’s use of DomainKeys as part of their inbound mail filtering.

Mickey has an article about how to deal with ISPs when attempting to troubleshoot a blocking issue.

More details and commentary on both articles later this week.

No Comments »

Late last week I heard from someone at AOL they were seeing strange traffic from a major ESP, that looked like the ESP was an open relay. This morning I received an email from AOL detailing what happened as relayed by the ESP.

IronPort Open Relay Vulnerability

Systems Affected
IronPort A60 running software version 2.5.4-005. According to IronPort, later devices and software versions using the same filtering mechanisms are vulnerable.

Overview
In recent weeks, one or more rogue spammers have been using misconfigured IronPort A60s as open relays to send unsolicited emails for AOL users via open relay. It is important for IronPort device administrators to review their configuration to shore up any vulnerability to this web server exploit.

Diagnosis
A seemingly minor configuration mistake made years ago internally has been exploited over the last several weeks to send out massive amounts of unsolicited email to AOL users. The spam mail originated from an outside zombie server, apparently infected with remote mailing viruses (such as BackDoor.Servu.76) according to the IT contact at IP 66.139.77.16. <ESP> has a filter specifically designed to deliver email over IP ranges set for AOL only. However, it was listed before a filter designed to log and discard bounced emails coming in through the Internet-facing of the IronPort appliance.

Impact
We have received 6,500 customer complaints so far through the AOL feedback loop. As the IronPort devices are black boxes, we are unable to determine how many unsolicited emails were delivered across them. It is difficult to ascertain whether or not the rogue spammer(s) knew only AOL addresses were delivered using this exploit. It is important to note that only AOL addresses were delivered in our specific case due to the order of the filters.

Solution
The solution was simple: move the filter designed to log and drop bounce messages coming in from the Internet to the top of the filter list so it will run first, as other filters may direct the IronPort device to deliver the emails through this vulnerability.

Authors: Jake Lanza, Baigh Auvigne, Daniel Fox

Congrats to the ESP for noticing this so quickly and being on the ball to stop this leak so quickly.

The compromise was first noticed when email coming back through the AOL FBL did not match any mail sent by the ESP. Initially, the ESP contacted AOL to report a problem with the FBL, but in working with AOL employees determined the email was coming from the ESP’s IP addresses.

This highlights the need to not just process FBL emails, but also monitor them and react when there are emails in a FBL that you do not recognize.

Ironport has responded here.

3 Comments »

The great folks over at MailChimp have compiled a list of which authentication methods (DK, DKIM, SPF and SenderID) are in use at which ISPs.

Good stuff and very clear showing who is using what authentication.

3 Comments »