DMARC is working on a standard to allow senders to publish sending policies and receivers to act on those policies. Currently, senders who want receivers to not deliver unauthenticated email have to negotiate private agreements with the ISPs to make that happen. This is a way to expand the existing programs. Without a published standard, the overhead in managing individual agreements would quickly become prohibitive.

It is an anti-phishing technique built on top of current authentication processes. This is the “next step” in the process and one that most people involved in the authentication process were anticipating and planning for. I’m glad to see so many big players participating.

J.D. Falk has a post up about the history of DKIM.

The dkim working group has completed its primary charter items, and is officially closing. The mailing list will be retained for future discussions involving dkim. The list archive will also be retained.

The dkim working group was primarily focused on DomainKeys Identified Mail (DKIM) Signatures and DomainKeys Identified Mail (DKIM) Author Domain Signing Practices (ADSP) (RFC 5617). We applaud the working group for progressing DomainKeys Identified Mail (DKIM) Signatures from proposed standard (RFC 4871) to Draft Standard (RFC 6376). The working group also produced the following Informational RFCs: Analysis of Threats Motivating DomainKeys Identified Mail (DKIM) (RFC 4868), DomainKeys Identified Mail (DKIM) Service Overview (RFC 5585), Requirements for a DomainKeys Identified Mail (DKIM) Signing Practices Protocol (RFC 5016), and DomainKeys Identified Mail (DKIM) Development, Deployment, and Operations (RFC 5863). The proposed standard DKIM And Mailing Lists (RFC 6737) was the final RFC produced by the working group.

What does this mean for DKIM-the-protocol?

DKIM as a protocol is finished and stable. It’s possible that there could be an incompatible “DKIM version 2″ in the future, but it’s fairly unlikely. It’s also possible that there’ll be a clarification of the existing spec in the future, if someone flushes out some bugs or inconsistencies while they’re implementing it, but that won’t break any existing usage.

If you want to attach an identifier to mail you send, or you want to use those identifiers to handle whitelisting or reputation-tracking or to drive domain-based feedback loops for mail you receive, DKIM is the way to do it.

What’s left to do?

DomainKeys Must Go

DomainKeys was the forerunner to DKIM. It’s been obsolete for several years and at this point nobody should be signing with it (or paying any attention to DomainKeys signatures). I’ve seen several cases recently where attempting to support both DomainKeys and DKIM lead to operational problems with the DKIM signing. It’s dead, let it go.

Good operational practices for signing with DKIM

This is the big bit of work left to do. While verifying a DKIM message and extracting the d= identifier from it is very well-defined, how best to sign with DKIM – including how best to do key management, key delegation and what DKIM options to use when signing – still hasn’t crystallized. Our suggestions for that are here: www.dkimcore.org

Configuration API support in smarthosts

While messages can be signed with DKIM anywhere in the mail pipeline they’re almost always signed at the sender smarthost. Many of the smarthosts added DKIM support by evolving the code they already had to sign with DomainKeys, which was a good engineering decision at the time. But DKIM is a much more flexible protocol than DomainKeys was, yet I’m told some of the smarthosts haven’t evolved far enough and only really support a “DomainKeys-like subset” of DKIM. Limited APIs like that reduce your flexibility in how you deploy DKIM, and will mean you can’t even try some of the smarter ways to use it.

As a minimum, your smarthost should be able to sign a message with any arbitrary private key / d= value pair rather than being tied to any email address or domain in the email headers. If it can’t do that, it’s not really supporting DKIM. If it can sign a message twice or three times efficiently – while processing the body of the message just once – that’s a DKIM feature that’s likely to be useful for ESPs.

Some sort of API to allow key management automation (deployment, delegation, rotation and invalidation) would make rich DKIM use much easier to deploy at scale, but I don’t know if anyone has looked at that sort of support in smarthosts yet. Anyone know?

Semantics

There’s still some misunderstanding about what some elements of DKIM mean.

What is the identity that DKIM conveys? (Usually the “d=” field, though the additional information in the “i=” field might sometimes be part of that too).

What does the selector mean? (Absolutely nothing! If you try and claim it does, you’re doing it wrong. It’s intended for key rotation and key delegation, and attempting to use it for anything else will make key management harder, and likely end up making the signature less secure).

What do multiple signatures mean? (That it was signed by each of those domains. None is more important than the others, and there’s no “primary” signing domain. What you do with that information is a whole other thing, though).

What about ADSP?

ADSP is effectively dead. The discussions that happened as part of it’s development, and the results (both good and bad) of limited testing with it will probably lead to something with similar goals in the near future – more limited in scope, probably, but less fundamentally flawed.

Whence SPF?

SPF isn’t dead, by any means. I expect some mail recipients will check both DKIM and SPF for the near future (though I’ve heard some interesting discussion about receivers keeping a local cache of DKIM results correlated with sending IP which may end up replacing one common SPF-based performance hack).

And what’s next?

Thanks to everyone involved in the DKIM specification process for creating this unobtrusive, robust way of attaching an identity to an email!

It’s going to be interesting to see what features people build on top of the DKIM foundation.

1. A via is presented to the user when you have a DKIM pass and the domain in the d= does not match the domain in the visible from address. In this case the interface shows via the d= domain.
2. A via is presented to the user when you have a SPF pass, no valid DKIM (either a fail or no signature at all) and the domain in the return path is different than the domain in the visible from address. In this case the interface shows via the SPF domain.

This is an issue for ESP customers who are letting ESPs sign on their behalf. If your ESP is signing with their own domain, or their own domain is present in the return path, then all your mail will be displayed with “via” in the gmail interface.

In order to not have a via showing, you need to have either the return path or the d= value within the same domain as your visible from address. There are two ways to do this. Probably the easiest is to delegate a subdomain to your ESP so they can manage the signing and keys for you. Alternatively, you can manage DKIM keys yourself and just have your ESP sign with the private key you give them.

I have heard that recipients can remove the via by replying to a message or by adding the sender to their address books. My testing did not show that either method was effective in removing the “via” from the display.

Tomorrow, the work that went into these rather simple recommendations.

It’s really the next step in email authentication, showing the results to the end user.

So how does Google do this? Google is checking both SPF and DKIM. If mail is authenticated and the authentication matches the from address then they display the email as:

If we click on “details” for that message, we find more specific information.

In this case the mail went through our outgoing mailserver to gmail.

Mailed-by indicates that the message passed SPF and that the IP address is a valid source of mail from wordtothewise.com.

Signed-by shows the domain in the DKIM d=. In this case, we signed with the subdomain dt.wordtothewise.com. That’s what happens when you sign using the domain in the From address (or a subdomain of it).

For a lot of bulk senders, though, their mail is signed using their ESP’s domain instead.  In that case Gmail shows who signed the mail as well as the from address.

And when we click on “details” for that message we see:

This is an email from a sender using Madmimi as an ESP. Madmimi is handling both the SPF authentication and the DKIM authentication.

As an aside, this particular  sender has a high enough reputation that Gmail is offering me an unsubscribe option in their interface.

Gmail is distinguishing between first party and third party signatures in authentication. If the mail is authenticated, but the authentication appears to be handled by a separate entity, then Gmail is alerting recipients to that fact.

What does this mean for bulk senders?

For senders that are signing with a domain that matches their From: domain, there is no change. Recipients will not see any mention of your ESP in the headers.

However, if you are using an ESP that is signing your mail with a domain they own, then your recipients will see that information displayed in the email interface. If you don’t want this to be displayed by Gmail, then you will need to move to first party signing. Talk to your ESP about this. If they’re unsure of how to manage it, you can point them to DKIM Core for an Email Service Provider.

Gmail blogpost about the changes

Gmail help page about authentication results

The real situation is, unsurprisingly, a bit more complicated.

What authentication features should you look for in an ESP?

1. Awareness of email authentication, specifically SPF and DKIM
2. They encourage you to use your own domain for authentication, but don’t require it
3. They suggest you add something like “include:cust-spf.exacttarget.com” to an existing SPF record, rather than anything with “ip4:” in it
4. They suggest you add an NS record something like “espname._domainkey.yourcompany.com” to your DNS for DKIM support
5. They ask you specifically what domain you’d like to use for authentication, rather than requiring it be the same as the domain you use in your From: field
   
6. They check that any DNS changes you’ve made are correct before sending email

7. They monitor your DNS changes, to protect you from mistakes in the future

What authentication mistakes should make you wary of an ESP?

1. Any suggestion that your reputation as a sender isn’t important, rather that it’s solely the ESPs job to get good delivery rates
2. Any suggestion that the ESPs reputation is completely irrelevant, and your mail will be delivered solely based on your reputation
3. They tell you that the domain name used in your email From: has to have anything to do with authentication
4. They tell you to add a TXT record for SPF records, rather than adding an “include:” clause to your existing record
5. They tell you to add anything with “ip4:” in it to your DNS
6. They tell you to add an NS record like “_domainkey.yourcompany.com”
7. Any suggestion you use ADSP (1).
8. Any mention of DomainKeys (2)

Reputation?

When we’re talking “authentication” we’re really talking about SPF and DKIM. Both are ways to associate an identifier with the email that’s sent that receivers can use to track the reputation of your mail (so that you can get the delivery benefits of your history of sending wanted mail) and manage whitelisting and feedback loops and all sorts of other good things. See dkimcore.org for more details on why authentication can be a good thing.

Neither SPF nor DKIM require you to use the same domain you use in your From: header as you use for authentication. This has a couple of really important implications:

1. Your ESP can use a domain of their own for authentication. That means that you can get some of the advantages of authentication (feedback loops, non-portable reputation) without having to edit your DNS or even knowing anything about what your ESP is doing.
2. If you use a range of different domains in your From: field (for brand-specific mailing, maybe) you can use the a single identifier for authentication across all that mail, getting all the benefits of your history of sending wanted email for BrandA and BrandB when you launch a new campaign for BrandC.

Portable vs non-portable reputation

I used the term “non-portable reputation” there without explaining it.

Non-portable reputation is reputation that is tied to your ESP – as you send wanted mail from that ESP your reputation with ISPs receiving the mail will get better and better, and you’ll get all the delivery advantages associated with that. But if you take your business to another ESP, or start sending some mail yourself, you’ll lose all that good reputation and have to start over.

Portable reputation is reputation that is tied to you, and mostly independent of your ESP. You can build up a history of sending email that people want to receive through one ESP, then you can move to a different ESP and take all that good history with you, keeping all the delivery advantages. (Or use multiple ESPs for different campaigns and send some mail from in-house and pool your good reputation across all those sources of mail.)

If you’re a reputable sender, sending mail that people want, then building a portable reputation will significantly benefit you. It does require you to configure your domains DNS records to support it, though, which can be tricky to set up and isn’t supported well by some DNS providers. So if you’re just beginning to dangle your toes in email marketing using non-portable reputation from your ESP is a great way to get going, but you should plan on switching to using portable reputation if email is going to be important to your business.

Some ESPs don’t really like the idea of portable reputation as without it their good customers are likely to take a significant delivery rate hit if they move elsewhere – and that vendor lock-in helps them keep those customers. Better ESPs are more likely to be enthusiastic about portable reputation as while it makes it easier for customers to move to competing ESPs it will also make it easy for them to move to them from their competitors. Without that disincentive for customers to move from one ESP to another it makes it easier for good ESPs to compete based on price and quality of service.

You really want to be able to get portable reputation using both SPF and DKIM. For that you’ll want to use a consistent domain for authentication, often your main corporate domain, even if you use a range of domains in your From: field. You want to be able to do this in a way that doesn’t tie you to using a single ESP for all your mail, rather that lets you use multiple ESPs simultaneously without losing your reputation. And you want it done in a way that doesn’t require you to be involved in editing DNS records any time any of those ESPs needs to make a change, otherwise the IT overhead and the risk of mistakes will be something you’ll regret.

For SPF this can best be done with a DNS record that looks something like

yourdomain.com TXT “v=spf1 mx include:cust-spf.esp1.com include:cust-spf.esp2.com include:cust-spf.esp3.com ~all”

… possibly with a few “ip4:” fields near the beginning to specify your corporate mailservers. The use of include: to pull in the SPF configuration for each of the ESPs you use allows you to have valid SPF records for each of those ESPs, and allows them to make any changes that are needed as they rearchitect their networks without needing to bother you.

For DKIM this can best be done with a set of DNS records that looks something like

yourdomain._domainkey.yourdomain.com TXT (a DKIM public key goes here, if you want to use DKIM on mail you send yourself)
esp1._domainkey.yourdomain.com NS dkim-ns1.esp1.com
esp1._domainkey.yourdomain.com NS dkim-ns2.esp1.com
esp2._domainkey.yourdomain.com NS ns1.esp2.com
esp2._domainkey.yourdomain.com NS ns2.esp2.com
esp3._domainkey.yourdomain.com NS dkim-nameserver.esp3.com

The first record is just if you’re using DKIM (or DKIM Core) for mail you’re sending yourself. The remaining records allow the ESPs you’re using to do all the magic that’s needed to sign with DKIM – generating and rotating key pairs, publishing public keys, expiring public keys and so on – without needing you to do anything further once the NS records are set up.

If an ESP suggests that instead of delegating something like “esp1._domainkey.yourdomain.com” you instead delegate “_domainkey.yourdomain.com” then run away. If you were to do that it would make it impossible for you to use DKIM with any other ESP, or even to use it yourself.

(If you are an ESP, and you want more details on how to set up DKIM in this way, take a look at this.)

(Footnotes)

1. ADSP is an extension of DKIM that can provide some minor benefits in rare circumstances, but will cause significant delivery problems most of the time. ADSP shouldn’t come up in conversation with an ESP unless you bring it up (and you shouldn’t do that unless you’re intimately aware of the damage it will do to your delivery)
2. DomainKeys is the forerunner to DKIM. It’s obsolete now, and while a small number of ISPs still pay attention to DomainKeys signatures when there’s no DKIM signature, and some senders sign with both, there’s really no need for anyone to use DomainKeys today.

1. If a user deletes a mail without reading it multiple times, Hotmail asks the user if they want to unsubscribe from the mail.
2. Users can use a the new “sweep” feature to delete or file multiple emails easily

Finally, Hotmail confirms that mail can be moved from bulk folder to inbox before the user reads it if the reputation of the sender changes.

Facebook is signing mail with DKIM, but using a very weak key that could be cracked easily. Anyone signing with DKIM should use RSA-1024 keys, nothing less.

Tagged.com is facing legal action brought by the NY AG’s office for not turning a blind eye to child porn.

Facebook’s COO announces the death of email. News at 11. I’ve been hearing announcements about the death of email since I got my first real .edu account back in ’93 or so and I will believe it when I see it. Given how much email Facebook actually sends, I can’t imagine what they’re thinking here. Facebook is the new Myspace, which is the new Geocities. Social networking may be useful for some things, but somehow I can’t imagine trying to get a customer delisted from Spamhaus by posting on a Facebook wall. Or handling receipts from online purchases or any of the other things that people use email for that don’t involve socializing with friends.

His questions were well informed for the most part. A few of them were completely out of left field, so I asked him for some of his references. One of those references was the EEC Email Authentication Whitepaper.

My client was doing the best he could to inform himself and relies on industry groups like the EEC to provide him with accurate information. In this case, their information was incomplete and incorrect.

We all have our perspectives and biases (yes, even me!) but there are objective facts that can be independently verified. For instance, the EEC Authentication whitepaper claimed that Yahoo requires DKIM signing for access to their whitelist program. This is incorrect, a sender does not have to sign with DKIM in order to apply for the Yahoo whitelist program. A bulk sender does have to sign with DKIM for a Y! FBL, but ISPs are given access to an IP based FBL by Yahoo. I am shocked that none of the experts that contributed to the document caught that error.

Independent verification is one reason I publish the Delivery Wiki. It’s a resource for everyone and a way to share my knowledge and thought processes. But other experts can “check my work” as it were and provide corrections if my information is outdated or faulty. All too often, senders end up blaming delivery problems on evil spirits, or using “dear” in the subject line or using too much pink in the design.

Delivery isn’t that esoteric or difficult if you have a clear understanding of the policy and technical decisions at a range of ESPs and ISPs, the history and reasoning behind those decisions, and enough experience to predict the implications when they collide.

Many senders do face delivery challenges and there is considerable demand for delivery experts to provide delivery facts. That niche has been filled by a mix of people, of all levels of experience, expertise and technical knowledge, leading to the difficult task of working out which of those “experts” are experts, and which of those “facts” are facts.

DKIM only answers two questions:

• Does the message have a valid signature?
• If it does, what domain signed it?

The signing domain, identified by the d= tag in the DKIM signature header, is the only part of the DKIM signature where the choices you make now will directly affect the continued deliverability of your messages. This is because d= is how you tell the receiving system who you are.

J.D. goes on to describe the different flags and how they may affect delivery of a particular signed message. It’s a good review of the flags and what they mean.