DMARC is working on a standard to allow senders to publish sending policies and receivers to act on those policies. Currently, senders who want receivers to not deliver unauthenticated email have to negotiate private agreements with the ISPs to make that happen. This is a way to expand the existing programs. Without a published standard, the overhead in managing individual agreements would quickly become prohibitive.

It is an anti-phishing technique built on top of current authentication processes. This is the “next step” in the process and one that most people involved in the authentication process were anticipating and planning for. I’m glad to see so many big players participating.

1. A via is presented to the user when you have a DKIM pass and the domain in the d= does not match the domain in the visible from address. In this case the interface shows via the d= domain.
2. A via is presented to the user when you have a SPF pass, no valid DKIM (either a fail or no signature at all) and the domain in the return path is different than the domain in the visible from address. In this case the interface shows via the SPF domain.

This is an issue for ESP customers who are letting ESPs sign on their behalf. If your ESP is signing with their own domain, or their own domain is present in the return path, then all your mail will be displayed with “via” in the gmail interface.

In order to not have a via showing, you need to have either the return path or the d= value within the same domain as your visible from address. There are two ways to do this. Probably the easiest is to delegate a subdomain to your ESP so they can manage the signing and keys for you. Alternatively, you can manage DKIM keys yourself and just have your ESP sign with the private key you give them.

I have heard that recipients can remove the via by replying to a message or by adding the sender to their address books. My testing did not show that either method was effective in removing the “via” from the display.

Tomorrow, the work that went into these rather simple recommendations.

So here’s some short prescriptive advice in no particular order for “how to do email authentication at an ESP well” without the long discussions of alternative approaches and justification of each piece of advice.

1. Remove every trace of DomainKeys from your email flow
2. Sign all your email with DKIM
3. Publish SPF records
4. Have your SPF records finish with ~all, not -all
5. Ignore SenderID, unless you have delivery problems at Hotmail (and even then measure results rather than just assuming it will help)
6. Don’t publish ADSP records
7. Have the customer pick a single domain to tie their reputation to – ideally their “main” domain, but if that’s not possible then a customer-specific domain owned by the ESP is the next best thing (i.e. “customer.com” is ideal, but “customer.esp.com” is much better than just “esp.com”)
   1. Use that domain as the d= field when signing email (“d=customer.com”)
   2. Use that domain, or a sub-domain of it, in the From: field of the email (“From: Someone <someone@customer.com>” or “From: Someone <someone@fooble.customer.com>”)
8. Pick a good email address to use in the From: field, don’t change it lightly
9. Use two-part selectors for DKIM, one specific to you, the ESP, on the right and one for key rotation on the left (why do this?)
10. Use just what’s needed for DKIM – don’t use i=, l=, q=, x= or z= in the signature, don’t use g= or s= in the published key
    1. dkimcore.org has more specific advice about which subset of DKIM to sign with
11. Make sure your SPF records are valid
12. TLS/SSL isn’t useful for authenticating mail (it’s great for connecting to a smarthost from a mail client, but not for connections from the smarthost)
13. None of SPF, DKIM or TLS are really designed to protect or hide the contents of a message
    1. Customers who really want that could look at S/MIME or PGP
    2. … but better to design their business model such that they don’t need to, as client support is spotty at best
14. Empirical evidence that doing something will make a customer happy trumps any of the above
    1. maybe because there’s evidence SenderID or ADSP or somesuch really helps that customer
    2. or maybe because the customer just wants it, even if there’s no evidence it would help

Mail client use of authentication data is still in flux, so it’s likely that some of this may change slightly (interaction between SPF and DKIM d= domains, for example) but it’s a pretty good base to start with today.

It’s really the next step in email authentication, showing the results to the end user.

So how does Google do this? Google is checking both SPF and DKIM. If mail is authenticated and the authentication matches the from address then they display the email as:

If we click on “details” for that message, we find more specific information.

In this case the mail went through our outgoing mailserver to gmail.

Mailed-by indicates that the message passed SPF and that the IP address is a valid source of mail from wordtothewise.com.

Signed-by shows the domain in the DKIM d=. In this case, we signed with the subdomain dt.wordtothewise.com. That’s what happens when you sign using the domain in the From address (or a subdomain of it).

For a lot of bulk senders, though, their mail is signed using their ESP’s domain instead.  In that case Gmail shows who signed the mail as well as the from address.

And when we click on “details” for that message we see:

This is an email from a sender using Madmimi as an ESP. Madmimi is handling both the SPF authentication and the DKIM authentication.

As an aside, this particular  sender has a high enough reputation that Gmail is offering me an unsubscribe option in their interface.

Gmail is distinguishing between first party and third party signatures in authentication. If the mail is authenticated, but the authentication appears to be handled by a separate entity, then Gmail is alerting recipients to that fact.

What does this mean for bulk senders?

For senders that are signing with a domain that matches their From: domain, there is no change. Recipients will not see any mention of your ESP in the headers.

However, if you are using an ESP that is signing your mail with a domain they own, then your recipients will see that information displayed in the email interface. If you don’t want this to be displayed by Gmail, then you will need to move to first party signing. Talk to your ESP about this. If they’re unsure of how to manage it, you can point them to DKIM Core for an Email Service Provider.

Gmail blogpost about the changes

Gmail help page about authentication results

Two factor authentication that uses an uncopyable physical device (such as a cellphone or a security token) as a second factor mitigates most of these threats very effectively. Weaker two factor authentication using digital certificates is a little easier to misuse (as the user can share the certificate with others, or have it copied without them noticing) but still a lot better than a password.

Security problems solved, then?

Two-factor authentication isn’t our savior. It won’t defend against phishing. It’s not going to prevent identity theft. It’s not going to secure online accounts from fraudulent transactions. It solves the security problems we had 10 years ago, not the security problems we have today.Bruce Schneier, April 2005

Password stealing attacks are still a risk – especially use of the same password on different services – but they’re not the main thrust of modern attacks, and haven’t been for years. Rather we’re seeing man-in-the-middle attacks and trojan attacks – these can be used very effectively as part of a targeted attack initiated by phishing or social engineering.

One form of a man-in-the-middle attack is to create a fake website that looks like your real website, and then to entice one of your users to go to the fake website instead of the real one. Your user then enters their password and the second factor from their securid fob, and the attacker uses that to log in to your website. Done well, the user will never notice – the attacker either gives them a fake error message and redirects them to your real login page or tunnels their transactions through to your website while also piggybacking their own transactions at the same time.

A trojan attack is similar, but the man-in-the-middle is hostile code actually running on the users computer.

Not just a theoretical attack

This isn’t just a theoretical attack. It’s fairly widespread, and probably underreported. One example from a couple of years ago is use of a trojan to steal half a million dollars from a local company, despite their banks use of one-time-password, securid style two factor authentication. Here’s another.

The accounts an ESP is protecting likely aren’t worth half a million dollars, so maybe bank-grade two factor authentication is good enough for them?

Another heavy user of two factor authentication is the online game World of Warcraft. They use a physical security fob or a smartphone app to generate one time passwords.

As we’ve mentioned before there’s a black market in stolen World of Warcraft accounts. They’re typically worth $8-$10 in bulk. And they’re being targeted by a key-logging trojan that intercepts the authentication data and passes it to the attacker, who then can take control of the account until they log out.

That means it can be cost-effective for an attacker to use a reasonably sophisticated keylogger trojan to take control of an account worth $10 for a couple of hours, which is bad news if you’re relying on your customers accounts not being that high value a target.

What value does 2FA have, then?

it won’t work for remote authentication over the Internet. I predict that banks and other financial institutions will spend millions outfitting their users with two-factor authentication tokens. Early adopters of this technology may very well experience a significant drop in fraud for a while as attackers move to easier targets, but in the end there will be a negligible drop in the amount of fraud and identity theft.Bruce Schneier, 2005

2FA is a decent way to improve password security. It’s easier and cheaper to require some form of 2FA than it is to train your users to use good passwords, and not to reuse passwords. And they can be part of a decent security approach – though the inconvenience and support overhead might exceed their value. But focusing on 2FA as a security solution won’t protect you from most current attack vectors, and can distract you and consume resources you could better spend on more effective approaches.

By concentrating on authenticating the individual rather than authenticating the transaction, banks are forced to defend against criminal tactics rather than the crime itself.Bruce Schneier, 2005

But two factor authentication is a great way to deal with some non-security related business problems, such as sharing of “flat fee” accounts by multiple users.

Two factor authentication is not a magic bullet for ESP security, and if it distracts you from implementing more effective (behaviour-based, rather than authentication based) security approaches then that narrow focus risks making your overall security worse.

Unless, that is, you’re defending solely against security threats from 1995.

In computer security terms authentication is proving who you are – when you enter a username and a password to access your email account you’re authenticating yourself to the system using a password that only you know.

Authentication (“who you are”) is the most visible part of computer access control, but it’s usually combined with two other A’s – authorization (“what you are allowed to do”) and accounting (“who did what”) to form an access control system.

And what are the two factors?

Two factor authentication means using two independent sources of evidence to demonstrate who you are. The idea behind it is that it means an attacker need to steal two quite different bits of information, with different weaknesses and attack vectors, in order to gain access. This makes the attack scenario much more complex and difficult for an attacker to carry out.

It’s important that the different factors are independent – requiring two passwords doesn’t count as 2FA, as an attack that can get the first password can just as easily get the second password. Generally 2FA requires the user to demonstrate their identity via two out of three broad ways:

1. Something the user knows – a password or a PIN
2. Something the user has – a key, an ID card, a phone number, a digital certificate or a physical token
3. Something the user is – such as a fingerprint

An everyday example of 2FA is using a cash machine or ATM. You insert your ATM card (something you have) and enter your PIN (something you know) to get access to your bank account. An attacker would have to both steal or copy your card and know your PIN to access your account. While a crooked waiter might be able to copy your card and someone could look over your shoulder to see your PIN, it’s much more difficult for an attacker to get both.

Most deployed 2FA systems work in much the same way. They require you to enter a password you know, and then to demonstrate that you have something in your possession – by having your computer present a digital certificate, or having you enter a number from a security token like those pictured above, or respond to an SMS message.

Security problems solved, then?

I’ll look at that tomorrow.

(Spoiler: No)

[W]e’re beefing up Yahoo! Mail’s SpamGuard by adding more security measures that make it much harder for phishers to get to your mailbox. We’ve teamed up with email authentication partners—namely, Authentication Metrics, eCert, Return Path, and Truedomain—to gain significant coverage to protect the prime targets of phishing attacks.

Phishing is a huge problem. I have an unprotected mailbox and get tens of dozens of phishing emails a day. But until there was a way to validate the sender of an email, rather than just the source IP, there wasn’t a good way to say that a particular email didn’t count.

SPF was one of the first attempts to solve this problem, but it didn’t do it very well. There were a number of very common uses of email that SPF didn’t accommodate.

Despite what the SPF crowd desperately wants to belive, there’s no simple way to tell what mail can legitimately be sent from what IPs. In some cases you can get pretty close, e.g., ESP spam cannon stuff, but even there plenty of people forward other accounts to gmail, which SPF doesn’t handle. — John Levine

Then there came Domain Keys and Identified Mail. Those two specs were close enough to one another that they merged into a single spec, DKIM. For the last few years significant numbers of people have been working to get DKIM stabilized and deployed.  That adoption and deployment lets companies build out products like YAMP and protect users from phishing.

Once the sender has ensured that all their email is being authenticated, they can add their domains and sub-domains to the Domain Assurance Registry list for ISPs to automatically reject all mail coming from these registered domains that fail authentication. Email senders using Domain Assurance have access to rich data reports about their email, get alerted when fraudulent emails using their domains are observed, and are provided with email intelligence on attackers and phishing URLs so they can initiate the take down of fraudulent websites.

The real situation is, unsurprisingly, a bit more complicated.

What authentication features should you look for in an ESP?

1. Awareness of email authentication, specifically SPF and DKIM
2. They encourage you to use your own domain for authentication, but don’t require it
3. They suggest you add something like “include:cust-spf.exacttarget.com” to an existing SPF record, rather than anything with “ip4:” in it
4. They suggest you add an NS record something like “espname._domainkey.yourcompany.com” to your DNS for DKIM support
5. They ask you specifically what domain you’d like to use for authentication, rather than requiring it be the same as the domain you use in your From: field
   
6. They check that any DNS changes you’ve made are correct before sending email

7. They monitor your DNS changes, to protect you from mistakes in the future

What authentication mistakes should make you wary of an ESP?

1. Any suggestion that your reputation as a sender isn’t important, rather that it’s solely the ESPs job to get good delivery rates
2. Any suggestion that the ESPs reputation is completely irrelevant, and your mail will be delivered solely based on your reputation
3. They tell you that the domain name used in your email From: has to have anything to do with authentication
4. They tell you to add a TXT record for SPF records, rather than adding an “include:” clause to your existing record
5. They tell you to add anything with “ip4:” in it to your DNS
6. They tell you to add an NS record like “_domainkey.yourcompany.com”
7. Any suggestion you use ADSP (1).
8. Any mention of DomainKeys (2)

Reputation?

When we’re talking “authentication” we’re really talking about SPF and DKIM. Both are ways to associate an identifier with the email that’s sent that receivers can use to track the reputation of your mail (so that you can get the delivery benefits of your history of sending wanted mail) and manage whitelisting and feedback loops and all sorts of other good things. See dkimcore.org for more details on why authentication can be a good thing.

Neither SPF nor DKIM require you to use the same domain you use in your From: header as you use for authentication. This has a couple of really important implications:

1. Your ESP can use a domain of their own for authentication. That means that you can get some of the advantages of authentication (feedback loops, non-portable reputation) without having to edit your DNS or even knowing anything about what your ESP is doing.
2. If you use a range of different domains in your From: field (for brand-specific mailing, maybe) you can use the a single identifier for authentication across all that mail, getting all the benefits of your history of sending wanted email for BrandA and BrandB when you launch a new campaign for BrandC.

Portable vs non-portable reputation

I used the term “non-portable reputation” there without explaining it.

Non-portable reputation is reputation that is tied to your ESP – as you send wanted mail from that ESP your reputation with ISPs receiving the mail will get better and better, and you’ll get all the delivery advantages associated with that. But if you take your business to another ESP, or start sending some mail yourself, you’ll lose all that good reputation and have to start over.

Portable reputation is reputation that is tied to you, and mostly independent of your ESP. You can build up a history of sending email that people want to receive through one ESP, then you can move to a different ESP and take all that good history with you, keeping all the delivery advantages. (Or use multiple ESPs for different campaigns and send some mail from in-house and pool your good reputation across all those sources of mail.)

If you’re a reputable sender, sending mail that people want, then building a portable reputation will significantly benefit you. It does require you to configure your domains DNS records to support it, though, which can be tricky to set up and isn’t supported well by some DNS providers. So if you’re just beginning to dangle your toes in email marketing using non-portable reputation from your ESP is a great way to get going, but you should plan on switching to using portable reputation if email is going to be important to your business.

Some ESPs don’t really like the idea of portable reputation as without it their good customers are likely to take a significant delivery rate hit if they move elsewhere – and that vendor lock-in helps them keep those customers. Better ESPs are more likely to be enthusiastic about portable reputation as while it makes it easier for customers to move to competing ESPs it will also make it easy for them to move to them from their competitors. Without that disincentive for customers to move from one ESP to another it makes it easier for good ESPs to compete based on price and quality of service.

You really want to be able to get portable reputation using both SPF and DKIM. For that you’ll want to use a consistent domain for authentication, often your main corporate domain, even if you use a range of domains in your From: field. You want to be able to do this in a way that doesn’t tie you to using a single ESP for all your mail, rather that lets you use multiple ESPs simultaneously without losing your reputation. And you want it done in a way that doesn’t require you to be involved in editing DNS records any time any of those ESPs needs to make a change, otherwise the IT overhead and the risk of mistakes will be something you’ll regret.

For SPF this can best be done with a DNS record that looks something like

yourdomain.com TXT “v=spf1 mx include:cust-spf.esp1.com include:cust-spf.esp2.com include:cust-spf.esp3.com ~all”

… possibly with a few “ip4:” fields near the beginning to specify your corporate mailservers. The use of include: to pull in the SPF configuration for each of the ESPs you use allows you to have valid SPF records for each of those ESPs, and allows them to make any changes that are needed as they rearchitect their networks without needing to bother you.

For DKIM this can best be done with a set of DNS records that looks something like

yourdomain._domainkey.yourdomain.com TXT (a DKIM public key goes here, if you want to use DKIM on mail you send yourself)
esp1._domainkey.yourdomain.com NS dkim-ns1.esp1.com
esp1._domainkey.yourdomain.com NS dkim-ns2.esp1.com
esp2._domainkey.yourdomain.com NS ns1.esp2.com
esp2._domainkey.yourdomain.com NS ns2.esp2.com
esp3._domainkey.yourdomain.com NS dkim-nameserver.esp3.com

The first record is just if you’re using DKIM (or DKIM Core) for mail you’re sending yourself. The remaining records allow the ESPs you’re using to do all the magic that’s needed to sign with DKIM – generating and rotating key pairs, publishing public keys, expiring public keys and so on – without needing you to do anything further once the NS records are set up.

If an ESP suggests that instead of delegating something like “esp1._domainkey.yourdomain.com” you instead delegate “_domainkey.yourdomain.com” then run away. If you were to do that it would make it impossible for you to use DKIM with any other ESP, or even to use it yourself.

(If you are an ESP, and you want more details on how to set up DKIM in this way, take a look at this.)

(Footnotes)

1. ADSP is an extension of DKIM that can provide some minor benefits in rare circumstances, but will cause significant delivery problems most of the time. ADSP shouldn’t come up in conversation with an ESP unless you bring it up (and you shouldn’t do that unless you’re intimately aware of the damage it will do to your delivery)
2. DomainKeys is the forerunner to DKIM. It’s obsolete now, and while a small number of ISPs still pay attention to DomainKeys signatures when there’s no DKIM signature, and some senders sign with both, there’s really no need for anyone to use DomainKeys today.

Domain Assurance [...] first audit[s] a company’s email streams to be sure authentication has been properly implemented. Then, the company’s domains are added to a registry. Participating ISPs can check the registry and block any unauthenticated emails coming from the domains found there. Return Path provides on-going checks for authentication accuracy and alerts participating companies any time their brand is phished or spoofed.

While this type of authentication won’t solve phishing, it is a tool to help protect end users from malicious mail.