Mailing List Archive: SPF: Discuss

SPF, DKIM, and NIH

1 2 3

View All

Index | Next | Previous | View Threaded

michael at talamasca

Oct 6, 2009, 7:54 PM

Post #1 of 63 (3410 views)
Permalink

SPF, DKIM, and NIH

I hinted at the following as an aside in my Forwarder Mitigation
proposals, but no one really picked up on the idea. So I'm going to
repeat it more directly:

An ugly practical fact is that neither SPF nor DKIM-ADSP checking
policies are suitable for deployment at e-mail providers which don't
know their users well. Only vanity domains, and e-mail providers which
specialize in custom spam filtering, can use them safely.

This is because SPF checking will false-positive on traditional
forwarding, and DKIM-ADSP will false-positive on most mailing list
traffic.

But, the reason DKIM-ADSP suffers mailing list FPs is not because of any
deficiency in its cryptographic approach. It's only because it tries to
guard the header from (From:), rather than the envelope sender (MAIL
FROM:), that it has this problem. Meanwhile, its cryptographic approach
does well at avoiding traditional-forwarder FPs.

Thus, I see a niche for a protocol that, like SPF, guards the envelope
sender, but uses cryptography like DKIM. Such a protocol could be
safely deployed blindly at large ISPs.

If I understand DKIM correctly, DKIM validators are to ignore DKIM
signatures that sign what, to them, is the "wrong" identity. So, there
should be no obstacle to mailservers double-signing a message when the
envelope MAIL FROM: and the header From: are not the same.

Since only a simple flag is needed, it would make sense to piggyback
this on SPF records with a special modifier. (Such as the "fm=dkim"
from my original senderside forwarder mitigation proposal...)

Some might say that this makes all the rest of SPF pointless. To this,
I'd first point out that such sentiment betrays the ugly, irrational Not
Invented Here syndrome. This is about suppressing abusive mail, not
building empires.

And the rest of SPF can still have a point. Receivers can opt to skip
the work of DKIM-on-envelope validation if normal SPF gives a clear
pass, and they can also reject at RCPT if normal SPF gives a clear fail
*and* they are sure there are no unwhitelisted forwarders.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

stuart at bmsi

Oct 6, 2009, 8:16 PM

Post #2 of 63 (3357 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Tue, 6 Oct 2009, Michael Deutschmann wrote:

> If I understand DKIM correctly, DKIM validators are to ignore DKIM
> signatures that sign what, to them, is the "wrong" identity. So, there
> should be no obstacle to mailservers double-signing a message when the
> envelope MAIL FROM: and the header From: are not the same.
>
> Since only a simple flag is needed, it would make sense to piggyback
> this on SPF records with a special modifier. (Such as the "fm=dkim"
> from my original senderside forwarder mitigation proposal...)

This can't be verified until the entire message is received. While
using DKIM to validate Return-Path is a good idea, it is not SPF,
and is not an SMTP envelope time protocol. You should take it up
with the DKIM folks. It should be just a matter of adding a new
signed identity to the DKIM header.

As to "NIH", it is not so much that as hoping "traditional" forwarding will
become inconvenient enough to die away like open relays. It took a long
time for legit admins to realize that open relays had been rendered useless by
the abuse of spammers. The same thing has happened to
traditional forwarding, but not everyone has realized it yet.

--
Stuart D. Gathman <stuart [at] bmsi>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

scott at kitterman

Oct 6, 2009, 8:23 PM

Post #3 of 63 (3360 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Tue, 6 Oct 2009 19:54:55 -0700 (PDT) Michael Deutschmann
<michael [at] talamasca> wrote:
...
>But, the reason DKIM-ADSP suffers mailing list FPs is not because of any
>deficiency in its cryptographic approach. It's only because it tries to
>guard the header from (From:), rather than the envelope sender (MAIL
>FROM:), that it has this problem. Meanwhile, its cryptographic approach
>does well at avoiding traditional-forwarder FPs.
>...

This is exactly backwards. Body From is preserved by mailing lists. The
reason mailing lists are a problem for ADSP is most mailing lists modify
the message in ways the break the signature so the signature is lost.

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 6, 2009, 8:49 PM

Post #4 of 63 (3356 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Tue, 6 Oct 2009, Scott Kitterman wrote:
> On Tue, 6 Oct 2009 19:54:55 -0700 (PDT) Michael Deutschmann
> <michael [at] talamasca> wrote:
> ...
> >But, the reason DKIM-ADSP suffers mailing list FPs is not because of any
> >deficiency in its cryptographic approach. It's only because it tries to
> >guard the header from (From:), rather than the envelope sender (MAIL
> >FROM:), that it has this problem. Meanwhile, its cryptographic approach
> >does well at avoiding traditional-forwarder FPs.
> >...
>
> This is exactly backwards. Body From is preserved by mailing lists.

And that's the problem. The list doesn't preserve the signature, but
preserves a purported identity that requires the unbroken signature.

Mailing lists are "friendly forgery" of the header From:, and break under
DKIM/ADSP.

Traditional forwarders are "friendly forgery" of the envelope FROM:, and
break under SPF.

Neither would break under the hybrid protocol. It wouldn't care about
the friendly forgery of the mailing lists, and it would recognize most
traditional forwards as authentic because they are relayed verbatim.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

iane at sussex

Oct 7, 2009, 3:25 AM

Post #5 of 63 (3352 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

--On 6 October 2009 20:49:44 -0700 Michael Deutschmann
<michael [at] talamasca> wrote:

> On Tue, 6 Oct 2009, Scott Kitterman wrote:
>> On Tue, 6 Oct 2009 19:54:55 -0700 (PDT) Michael Deutschmann
>> <michael [at] talamasca> wrote:
>> ...
>> > But, the reason DKIM-ADSP suffers mailing list FPs is not because of
>> > any deficiency in its cryptographic approach. It's only because it
>> > tries to guard the header from (From:), rather than the envelope
>> > sender (MAIL FROM:), that it has this problem. Meanwhile, its
>> > cryptographic approach does well at avoiding traditional-forwarder FPs.
>> > ...
>>
>> This is exactly backwards. Body From is preserved by mailing lists.
>
> And that's the problem. The list doesn't preserve the signature, but
> preserves a purported identity that requires the unbroken signature.
>
> Mailing lists are "friendly forgery" of the header From:, and break under
> DKIM/ADSP.
>
> Traditional forwarders are "friendly forgery" of the envelope FROM:, and
> break under SPF.

This doesn't need a new protocol. When receiving messages, you should apply
SPF and DKIM tests, and apply reputation tests to the one that matches, if
either. What you're looking for is a token that you can reliably apply
reputation services to. An SPF fail simply means that you can't apply your
reputation service to the envelope-sender. It doesn't mean that the message
isn't good, so go ahead and see if it has a good DKIM signature.

With DKIM, if you know a message has been through a list, then it should be
re-signed. Check the new signature. If it matches, and you trust the list
sender, then check the reputation of the original poster. If you trust the
list, then you should assume that it's checking DKIM on the way in. If your
assumption is wrong, you shouldn't trust the list.

> Neither would break under the hybrid protocol. It wouldn't care about
> the friendly forgery of the mailing lists, and it would recognize most
> traditional forwards as authentic because they are relayed verbatim.
>
> ---- Michael Deutschmann <michael [at] talamasca>
>
>
> -------------------------------------------
> Sender Policy Framework: http://www.openspf.org
> Modify Your Subscription: http://www.listbox.com/member/
> Archives: https://www.listbox.com/member/archive/735/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/735/
> Powered by Listbox: http://www.listbox.com

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 7, 2009, 8:42 PM

Post #6 of 63 (3356 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Wed, 7 Oct 2009, Ian Eiloart wrote:
> This doesn't need a new protocol. When receiving messages, you should apply
> SPF and DKIM tests, and apply reputation tests to the one that matches, if
> either. What you're looking for is a token that you can reliably apply
> reputation services to. An SPF fail simply means that you can't apply your
> reputation service to the envelope-sender. It doesn't mean that the message
> isn't good, so go ahead and see if it has a good DKIM signature.

Allowing a favorable SPF result to "rescue" a message with an unfavorable
DKIM/ADSP result, is straightforward enough.

But the reverse is problematic. SPF normally rejects failures at the
RCPT TO: command. At this point, it is unknown whether the message bears
DKIM signatures, let alone whether they are valid. It's not even known
whether a signature would be required, because this depends on the header
From.

The mailserver would have to defer all SPF-related rejections until DATA,
which would be a significant cost. If the mailserver honours per-user
preferences for minimum SPF result to accept, this would then require the
use of 4xx-on-RCPT-TO tricks. This cost would be applied across all mail,
most of which will be purportedly from domains which do not publish an
ADSP.

If "DKIM/EDSP" (*envelope* domain signing practices) information was
available, on an SPF fail the mailserver could look it up based on MAIL
FROM it has seen. If there is no EDSP, the mailserver could confidently
reject at RCPT TO. If an EDSP is declared, the mailserver could go on to
accept the mail contingently, rejecting at DATA if the required signature
is missing or bogus.

Also, note that an "EDSP" protocol would take very little effort to
design. All we need to do is standardize one new SPF modifier to flag a
domain as one that always signs the envelope from. The rest is mere
coding.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

vesely at tana

Oct 9, 2009, 3:06 AM

Post #7 of 63 (3349 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

Stuart D. Gathman wrote:
> On Tue, 6 Oct 2009, Michael Deutschmann wrote:
>> If I understand DKIM correctly, DKIM validators are to ignore DKIM
>> signatures that sign what, to them, is the "wrong" identity. So, there
>> should be no obstacle to mailservers double-signing a message when the
>> envelope MAIL FROM: and the header From: are not the same.
>>
>> Since only a simple flag is needed, it would make sense to piggyback
>> this on SPF records with a special modifier. (Such as the "fm=dkim"
>> from my original senderside forwarder mitigation proposal...)
>
> This can't be verified until the entire message is received. While
> using DKIM to validate Return-Path is a good idea, it is not SPF,
> and is not an SMTP envelope time protocol. You should take it up
> with the DKIM folks. It should be just a matter of adding a new
> signed identity to the DKIM header.

I don't think it would be a good idea. To allow forwarding is the
point of DKIM. Signing the Return-Path, then would only allow that
"traditional" forwarding mentioned below.

As Scott says, the problem is that most mailing lists modify the
message in ways that break the signature. The solution to this is to
fix the implementation of the existing protocol and related stuff.

> As to "NIH", it is not so much that as hoping "traditional" forwarding will
> become inconvenient enough to die away like open relays. It took a long
> time for legit admins to realize that open relays had been rendered useless by
> the abuse of spammers. The same thing has happened to
> traditional forwarding, but not everyone has realized it yet.

Much agreed! In addition, traditional forwarding breaks the spirit
--if not the letter-- of privacy regulations. That's not for being
deferentially law-abiding, but to recognize the principle, established
long after SMTP, that an email address is the property of its owner,
who shall be in full control of it: including the ability to remove it
from .forward files straightly. I've analyzed that requirement to some
extent in http://fixforwarding.org/ and I've been surprised to learn
that implementing that principle would provide means for "fixing"
newsletters and backup MXes as well.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

me at junc

Oct 9, 2009, 7:55 PM

Post #8 of 63 (3354 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On fre 09 okt 2009 12:06:22 CEST, Alessandro Vesely wrote
> As Scott says, the problem is that most mailing lists modify the
> message in ways that break the signature. The solution to this is to
> fix the implementation of the existing protocol and related stuff.

postfix maillist works with dkim from me to i get it back unmodified,
so my dkim is still valid, but there is other maillists that modified
parts that is dkim signed, and thus break dkim, solution to that is
not to change dkim, but to tell maillists that break dkim to solve it :)

--
xpoint

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 10, 2009, 9:03 PM

Post #9 of 63 (3352 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Tue, 6 Oct 2009, Stuart D. Gathman wrote:
(regarding a DKIM flag for SPF)
> This can't be verified until the entire message is received.

If you know there are no incoming traditional forwards, you can still
reject SPF-fail messages at RCPT TO, as before.

This would just give cautious admins who would otherwise resort to Crap
Receiverside mitigation (ie: treat fail as neutral), a way to give
potentially forwarded mails a second chance, without letting through any
messages the purported sender disapproves of.

> As to "NIH", it is not so much that as hoping "traditional" forwarding will
> become inconvenient enough to die away like open relays.

To some, the end of traditional forwarding has been just over the horizon
ever since SRS was proposed. Meanwhile, this attitude has permanently
damaged SPFv1 by inspiring the use of Crap Senderside mitigation (ie:
never use "-all").

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

macquigg at ece

Oct 11, 2009, 2:53 AM

Post #10 of 63 (3350 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

Michael Deutschmann wrote:
> On Tue, 6 Oct 2009, Stuart D. Gathman wrote:
> (regarding a DKIM flag for SPF)
>
>> This can't be verified until the entire message is received.
>>
>
> If you know there are no incoming traditional forwards, you can still
> reject SPF-fail messages at RCPT TO, as before.
>
> This would just give cautious admins who would otherwise resort to Crap
> Receiverside mitigation (ie: treat fail as neutral), a way to give
> potentially forwarded mails a second chance, without letting through any
> messages the purported sender disapproves of.
>

I believe it is possible to reject as soon as you see the DKIM-Signature
header, but the problem will be the same as SPF - too many legitimate
messages still have crap authentication. Yet another chicken-and-egg
situation.

In this message:

Authentication-Results: mta399.mail.re4.yahoo.com
from=talamasca.ocis.net; domainkeys=fail (bad sig);
from=talamasca.ocis.net; dkim=neutral (no sig)

DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=2006-04-29;
d=talamasca.ocis.net;
b=sz+JhrXN7SDhsoGpgbaWnaAbsGY4yeCKiEuape/zVe8lsmTxYMQpjM5H37EZCvjY;

>> As to "NIH", it is not so much that as hoping "traditional" forwarding will
>> become inconvenient enough to die away like open relays.
>>
>
> To some, the end of traditional forwarding has been just over the horizon
> ever since SRS was proposed. Meanwhile, this attitude has permanently
> damaged SPFv1 by inspiring the use of Crap Senderside mitigation (ie:
> never use "-all").
>

Last I heard 3% of SPF records ended in "-all", and 0% of forwarders are
using SRS. Has there been any change in the last year?

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 11, 2009, 3:50 AM

Post #11 of 63 (3348 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Sun, 11 Oct 2009, David MacQuigg wrote:
> I believe it is possible to reject as soon as you see the DKIM-Signature
> header, but the problem will be the same as SPF - too many legitimate
> messages still have crap authentication. Yet another chicken-and-egg
> situation.
>
> In this message:
>
> [. results showing Michael's mail, as relayed by list, to fail DK ]

Envelope-DKIM would not fail in this way. Like SPF, it would not care
that the "From:" was forged. Only the signing policy of the MAIL FROM:
domain (which for this list is "@jeeves.archives.listbox.com") would be
enforced.

I have a hard-fail DK record, since my commonsense understanding was that
people who subscribe to mailing lists must whitelist them before arming DK to
reject messages with broken signatures, even for "o=-" domains.

However, recently on the DKIM list it was claimed that the analogous
"dkim=all" ADSP does permit validators to act without considering the mailing
list problem....

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

scott at kitterman

Oct 11, 2009, 6:39 PM

Post #12 of 63 (3356 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Sun, 11 Oct 2009 03:50:43 -0700 (PDT) Michael Deutschmann
<michael [at] talamasca> wrote:
>On Sun, 11 Oct 2009, David MacQuigg wrote:
>> I believe it is possible to reject as soon as you see the DKIM-Signature
>> header, but the problem will be the same as SPF - too many legitimate
>> messages still have crap authentication. Yet another chicken-and-egg
>> situation.
>>
>> In this message:
>>
>> [. results showing Michael's mail, as relayed by list, to fail DK ]
>
>Envelope-DKIM would not fail in this way. Like SPF, it would not care
>that the "From:" was forged. Only the signing policy of the MAIL FROM:
>domain (which for this list is "@jeeves.archives.listbox.com") would be
>enforced.

Since mailing lists use their own envelope from, I guess I'm missing
something here. What would your envolope DKIM be signing and who would
sign it?
>
>I have a hard-fail DK record, since my commonsense understanding was that
>people who subscribe to mailing lists must whitelist them before arming DK
to
>reject messages with broken signatures, even for "o=-" domains.
>
>However, recently on the DKIM list it was claimed that the analogous
>"dkim=all" ADSP does permit validators to act without considering the mailing
>list problem....

RFC's can't mandate receiver behavior. Your interpretation of the ADSP RFC is sound. What you
got on the DKIM list was speculation of what receivers might do, not what the RFC says.

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

vesely at tana

Oct 12, 2009, 3:18 AM

Post #13 of 63 (3350 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

Michael Deutschmann wrote:
> I have a hard-fail DK record, since my commonsense understanding was that
> people who subscribe to mailing lists must whitelist them before arming DK to
> reject messages with broken signatures, even for "o=-" domains.

I don't think such behavior may ever become a common practice, as it
implies keeping track of subscriptions and coordinate them with
whitelisting. Too much work, if postmasters have to do it manually.

A good DKIM signature says that the signing domain trusts the message
sender. Much like with SPF, that means nothing unless the recipient
trusts the domain.

> However, recently on the DKIM list it was claimed that the analogous
> "dkim=all" ADSP does permit validators to act without considering the mailing
> list problem....

Yeah, "dkim=all" sounds much like SPF's -all. However, they miss a
~all --which even SPF does not specify in such a way to provide for
useful tools for practical testing. That way, it's hard for a
recipient to discern whether a signature is broken for a bug in the
message's transmission rather than being an actual forgery.

I'm not DKIM signing (yet), but I think I would be content of just
signing body From, Date, Message-ID, and possibly References or
In-Reply-To. Those fields are seldom tampered with. OTOH, if authors
want to sign the body they can do it themselves with either S/MIME of
OpenPGP.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 12, 2009, 4:01 AM

Post #14 of 63 (3352 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Sun, 11 Oct 2009, Scott Kitterman wrote:
> Since mailing lists use their own envelope from, I guess I'm missing
> something here. What would your envolope DKIM be signing and who would
> sign it?

Ok, assume someone using envelope DKIM (which is *not* the same as
DKIM/ADSP) posts to the list. It arrives at the mailing-list server with an
intact envelope signature.

Now, if the mailing-list is not aware of Envelope-DKIM, it changes the MAIL
FROM: and also mucks around a bit with the body.

When an ultimate recipient receives the message, he will look for an
Envelope-DKIM policy of the *mailing list's domain* (since that's what's in
MAIL FROM:), and find none. That means that no signatures are required, so
it will accept the mail. The broken signature is irrelevant, as to
Envelope-DKIM it is now 3rd-party. (Notwithstanding that it is 1st-party to
DKIM/ADSP.)

If the mailing list is aware of Envelope-DKIM, it will take ownership of the
message, purging the old Envelope-DKIM signature. It will then put a new
signature in, using its own domain and private key.

This is precisely analogous to the way SPF avoids mailing list FPs. Mailing
lists "friendly forge" the identity DKIM/ADSP cares about, but not the one
SPF and Envelope-DKIM track.

The advatange of Envelope-DKIM is that it would also have DKIM/ADSP's
resistance to forwarder FPs. The absence of both kinds of FP would allow the
protocol to be spread faster than either SPF or DKIM/ADSP.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

scott at kitterman

Oct 12, 2009, 5:06 AM

Post #15 of 63 (3359 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009 04:01:27 -0700 (PDT) Michael Deutschmann
<michael [at] talamasca> wrote:
>On Sun, 11 Oct 2009, Scott Kitterman wrote:
>> Since mailing lists use their own envelope from, I guess I'm missing
>> something here. What would your envolope DKIM be signing and who would
>> sign it?
>
>Ok, assume someone using envelope DKIM (which is *not* the same as
>DKIM/ADSP) posts to the list. It arrives at the mailing-list server with
an
>intact envelope signature.

How does this happen? Mail from is a property of the SMTP dialogue that is
only added to the message as return path by the receiver, so I don't
understand what there would be for the sender to sign?

>Now, if the mailing-list is not aware of Envelope-DKIM, it changes the MAIL
>FROM: and also mucks around a bit with the body.
>
>When an ultimate recipient receives the message, he will look for an
>Envelope-DKIM policy of the *mailing list's domain* (since that's what's in
>MAIL FROM:), and find none. That means that no signatures are required, so
>it will accept the mail. The broken signature is irrelevant, as to
>Envelope-DKIM it is now 3rd-party. (Notwithstanding that it is 1st-party
to
>DKIM/ADSP.)
>
>If the mailing list is aware of Envelope-DKIM, it will take ownership of
the
>message, purging the old Envelope-DKIM signature. It will then put a new
>signature in, using its own domain and private key.
>
>This is precisely analogous to the way SPF avoids mailing list FPs. Mailing
>lists "friendly forge" the identity DKIM/ADSP cares about, but not the one
>SPF and Envelope-DKIM track.
>
>The advatange of Envelope-DKIM is that it would also have DKIM/ADSP's
>resistance to forwarder FPs. The absence of both kinds of FP would allow
the
>protocol to be spread faster than either SPF or DKIM/ADSP.

The problem with this scenario is that ADSP is tied to the body From domain
and so even if such a signature could be produced, it would fail an ADSP
check.

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 12, 2009, 5:36 AM

Post #16 of 63 (3350 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009, Scott Kitterman wrote:
> How does this happen? Mail from is a property of the SMTP dialogue that is
> only added to the message as return path by the receiver, so I don't
> understand what there would be for the sender to sign?

By envelope signature I don't mean a signature that *protects* the MAIL
FROM:/Return-Path: from modifications -- that is indeed impossible in RFC
4871. I mean one whose *relevance depends* on the fact that domain specified
in the signature "d=" option coincides with the right-hand-side of the MAIL
FROM: address.

Relayers are free to change the MAIL FROM:, and far from blocking them from
changing it, if they do change it this frees them to drop the signature
without consequence.

The only thing they can't do is change the MAIL FROM: to a domain they don't
have a private key for, or tamper with the body while preserving a MAIL FROM:
they don't have the private key for.

----- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

vesely at tana

Oct 12, 2009, 6:25 AM

Post #17 of 63 (3346 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

Michael Deutschmann wrote:
> By envelope signature I don't mean a signature that *protects* the MAIL
> FROM:/Return-Path: from modifications -- that is indeed impossible in RFC
> 4871. I mean one whose *relevance depends* on the fact that domain specified
> in the signature "d=" option coincides with the right-hand-side of the MAIL
> FROM: address.
>
> Relayers are free to change the MAIL FROM:, and far from blocking them from
> changing it, if they do change it this frees them to drop the signature
> without consequence.

That's very easy to forge, though. As long as spammers sign correctly,
they can relay mail apparently coming from
one [at] high-reputation without any chance for that domain to mark
--across multiple relays-- the messages that have been sent from
authenticated users. In the good cases, final recipients trust the
list even more than the original "high-reputation.domain", and so it's
ok to remove broken original signatures. However, that classifies the
protocol as specific for mailing lists with a good reputation: not the
generic forwarding-resistant solution that DKIM claims to be.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 12, 2009, 7:00 AM

Post #18 of 63 (3359 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009, Alessandro Vesely wrote:
> > Relayers are free to change the MAIL FROM:, and far from blocking them from
> > changing it, if they do change it this frees them to drop the signature
> > without consequence.
>
> That's very easy to forge, though. As long as spammers sign correctly,
> [...]
> generic forwarding-resistant solution that DKIM claims to be.

Yes, Envelope-DKIM permits the bad guys to do:

MAIL FROM: <evil [at] evil>
RCPT TO: <victim [at] victim>
DATA
DKIM-Signature: ... d=evil.example.org ...
From: First Bank of Erewhon <victims-bank [at] bank>
Subject: Urgent! Need to re-confirm your account
....

But, *so* *does* *SPF*. And it's this very property that gives SPF its
immunity to mailing list FPs.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

scott at kitterman

Oct 12, 2009, 7:07 AM

Post #19 of 63 (3349 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009 07:00:46 -0700 (PDT) Michael Deutschmann
<michael [at] talamasca> wrote:
>On Mon, 12 Oct 2009, Alessandro Vesely wrote:
>> > Relayers are free to change the MAIL FROM:, and far from blocking them
from
>> > changing it, if they do change it this frees them to drop the signature
>> > without consequence.
>>
>> That's very easy to forge, though. As long as spammers sign correctly,
>> [...]
>> generic forwarding-resistant solution that DKIM claims to be.
>
>Yes, Envelope-DKIM permits the bad guys to do:
>
>MAIL FROM: <evil [at] evil>
>RCPT TO: <victim [at] victim>
>DATA
>DKIM-Signature: ... d=evil.example.org ...
>From: First Bank of Erewhon <victims-bank [at] bank>
>Subject: Urgent! Need to re-confirm your account
>....
>
>But, *so* *does* *SPF*. And it's this very property that gives SPF its
>immunity to mailing list FPs.

Then what's the advantge over SPF?

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 12, 2009, 7:33 AM

Post #20 of 63 (3347 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009, Scott Kitterman wrote:
> >But, *so* *does* *SPF*. And it's this very property that gives SPF its
> >immunity to mailing list FPs.
>
> Then what's the advantge over SPF?

Without giving up SPF's mailing list FP immunity, it gains the same
forwarding FP immunity DKIM/ADSP has.

It would work smoother than a mere "reject mail only if DKIM/ADSP and SPF
both say Fail" policy, because the mailserver would know in advance
whether it is worthwhile to let an incoming SPF-fail transaction proceed
to DATA.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

iane at sussex

Oct 12, 2009, 7:34 AM

Post #21 of 63 (3357 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

--On 12 October 2009 10:07:54 -0400 Scott Kitterman <scott [at] kitterman>
wrote:

> On Mon, 12 Oct 2009 07:00:46 -0700 (PDT) Michael Deutschmann
> <michael [at] talamasca> wrote:
>> On Mon, 12 Oct 2009, Alessandro Vesely wrote:
>>> > Relayers are free to change the MAIL FROM:, and far from blocking
>>> > them
> from
>>> > changing it, if they do change it this frees them to drop the
>>> > signature without consequence.
>>>
>>> That's very easy to forge, though. As long as spammers sign correctly,
>>> [...]
>>> generic forwarding-resistant solution that DKIM claims to be.
>>
>> Yes, Envelope-DKIM permits the bad guys to do:
>>
>> MAIL FROM: <evil [at] evil>
>> RCPT TO: <victim [at] victim>
>> DATA
>> DKIM-Signature: ... d=evil.example.org ...
>> From: First Bank of Erewhon <victims-bank [at] bank>
>> Subject: Urgent! Need to re-confirm your account
>> ....
>>
>> But, *so* *does* *SPF*. And it's this very property that gives SPF its
>> immunity to mailing list FPs.
>
> Then what's the advantge over SPF?

The advantage is that it permits trusted traditional forwarding. Which is
what's missing with SPF.

The thing is, that there are various routes by which mail may be delivered.
SPF protects some, but not others. DKIM protects others, but not some.

What we need is a collection of sender techniques, and a collection of
recipient checks, which collectively allow the recipient to apply
reputation scores for every incoming message - except the spam, of course.

SPF neatly protects all messages except traditional forwarding.
DKIM with ADSP neatly protects all messages except mailing list messages.

If SPF and DKIM/ADSP were universally deployed, recipients would have
something they could assign reputation to for every message:

If you see an SPF pass, then assign reputation by SPF. Lists that don't
check inbound mail won't get great reputation. If there's also a DKIM
signature, you can also check that content hasn't been munged, but watch
out for list-id headers.

If SPF fails, then look for a DKIM signature. If you get a good one, you're
likely seeing traditional forwarding. Do your reputation assignment with
DKIM, don't worry too much about the SPF.

If you have an SPF soft fail, or no policy, and no DKIM signature, and no
policy, then you're stuck with IP reputation and content assessment.

>
> Scott K
>
>
> -------------------------------------------
> Sender Policy Framework: http://www.openspf.org
> Modify Your Subscription: http://www.listbox.com/member/
> Archives: https://www.listbox.com/member/archive/735/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/735/
> Powered by Listbox: http://www.listbox.com

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

macquigg at ece

Oct 12, 2009, 8:18 AM

Post #22 of 63 (3344 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

Ian Eiloart wrote:
>>
>> Then what's the advantge over SPF?
>
> The advantage is that it permits trusted traditional forwarding. Which
> is what's missing with SPF.
>
> The thing is, that there are various routes by which mail may be
> delivered. SPF protects some, but not others. DKIM protects others,
> but not some.
>
> What we need is a collection of sender techniques, and a collection of
> recipient checks, which collectively allow the recipient to apply
> reputation scores for every incoming message - except the spam, of course.
>
> SPF neatly protects all messages except traditional forwarding.
> DKIM with ADSP neatly protects all messages except mailing list messages.
>
> If SPF and DKIM/ADSP were universally deployed, recipients would have
> something they could assign reputation to for every message:
>
> If you see an SPF pass, then assign reputation by SPF. Lists that
> don't check inbound mail won't get great reputation. If there's also a
> DKIM signature, you can also check that content hasn't been munged,
> but watch out for list-id headers.

The headers aren't the problem. It's the last few lines added to the
body by the mailing list. Header checking is done only on the few
headers named in the "h=" tag.

> If SPF fails, then look for a DKIM signature. If you get a good one,
> you're likely seeing traditional forwarding.

Or forwarding by a crook. What prevents a spammer from sending a
billion ads for Viagra, all with a valid DKIM signature from a reputable
domain? All it takes is one signed message. The rest can be copies,
"forwarded" via a botnet.

The fundamental advantage of signature-based authentication (arbitrary
forwarding) is a fundamental disadvantage when the forwarder is a
crook. Signatures protect only that which is signed, i.e. the body and
a few specifically selected headers. There is *no other assurance* in a
signature. Show that Viagra ad to the original signer, and he will say
"Yup, that's our signature. We sign 500,000 messages per day. We have
per-account rate limits. We even run spam filters on new accounts.
What else do you expect us to do? "

--
************************************************************ *
* David MacQuigg, PhD email: macquigg at ece.arizona.edu * *
* Research Associate phone: USA 520-721-4583 * * *
* ECE Department, University of Arizona * * *
* 9320 East Mikelyn Lane * * *
* http://purl.net/macquigg Tucson, Arizona 85710 *
************************************************************ *

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

scott at kitterman

Oct 12, 2009, 8:25 AM

Post #23 of 63 (3347 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009 07:33:12 -0700 (PDT) Michael Deutschmann
<michael [at] talamasca> wrote:
>On Mon, 12 Oct 2009, Scott Kitterman wrote:
>> >But, *so* *does* *SPF*. And it's this very property that gives SPF its
>> >immunity to mailing list FPs.
>>
>> Then what's the advantge over SPF?
>
>Without giving up SPF's mailing list FP immunity, it gains the same
>forwarding FP immunity DKIM/ADSP has.
>
>
>It would work smoother than a mere "reject mail only if DKIM/ADSP and SPF
>both say Fail" policy, because the mailserver would know in advance
>whether it is worthwhile to let an incoming SPF-fail transaction proceed
>to DATA.
>
OK, then what is the sender signing?

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

michael at talamasca

Oct 12, 2009, 9:14 AM

Post #24 of 63 (3344 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009, Scott Kitterman wrote:
> OK, then what is the sender signing?

The signature will lock down the same things as in DKIM/ADSP -- the
entire body and the sender's choice of headers. Only the d= values of the
signatures of interest to the validator will be different.

Not that it matters. Traditional forwarders send the message verbatim, so
they don't need to care. Mailing lists can freely break the original
envelope-signature since they do not assert the original envelope.

---- Michael Deutschmann <michael [at] talamasca>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

stuart at bmsi

Oct 12, 2009, 9:20 AM

Post #25 of 63 (3344 views)
Permalink

Re: SPF, DKIM, and NIH [In reply to]

On Mon, 12 Oct 2009, Scott Kitterman wrote:

> >Without giving up SPF's mailing list FP immunity, it gains the same
> >forwarding FP immunity DKIM/ADSP has.
> >
> >
> >It would work smoother than a mere "reject mail only if DKIM/ADSP and SPF
> >both say Fail" policy, because the mailserver would know in advance
> >whether it is worthwhile to let an incoming SPF-fail transaction proceed
> >to DATA.
> >
> OK, then what is the sender signing?

The sender is signing MAIL FROM in a manner similar to SES. SES would
actually be a more efficient way to accomplish the same goal, be
evaluated at SMTP envelope time, and works with the existing SPF standard
(via exists mechanism).

The drawback? It requires the sender to supply a specialized backend to a DNS
server to handle the validation requests. (PowerDNS is an authoritative only
open source DNS server that makes plug-in backends easy - so this is not a
showstopper.)

On the other hand, the DKIMed MAIL FROM proposal puts the burden of
special software on the receiver.

--
Stuart D. Gathman <stuart [at] bmsi>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

1 2 3

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads