Mailing List Archive: SPF: SRS

More on Philosophy of SPF

Index | Next | Previous | View Threaded

mengwong at dumbo

Feb 23, 2004, 8:50 AM

Post #1 of 10 (3175 views)
Permalink

More on Philosophy of SPF

On Mon, Feb 23, 2004 at 03:31:52PM +0000, Brian Candler wrote:
|
| But what will you do next after SPF?
|

I don't like the metaphor of escalation, which is open-ended.

I prefer to think of it as playing chess, which converges to a closed result.

A common response to SPF is "oh, but spammers aren't stupid, and they can react".

While that is true, it is not, by itself, a valid criticism; the same
criticism applies to any any anti-spam proposal, and every proposal
needs to ask itself that question. I think SPF has a stronger answer
than most others.

Bayesian filtering came across as the FUSSP. What did the spammers do?
They contracted out to the army of infinite monkeys who had been working
on writing Shakespare; now they're working on writing spam. Content
filtering can be gamed. As Brightmail pointed out the only way you can
really win is by looking at the URL in the spam.

And that's just spam. Worms have different goals. They are a-life, and
need to be treated using a hygiene model. Trying to lock up virus
authors is pointless at this point; unlike spam, worms are driven by
survival imperatives, not by the profit motive which is at least tied to
the human sphere.

When you play chess, if you only think one move ahead, you lose.

Here's a view of the board a few moves ahead: http://spf.pobox.com/faq.html#churn

Accreditation also has a role there.

| I would prefer to work on *strong* solutions which have a chance of
| remaining viable over time, and which don't break things in the mean time.

There are approximately 40 technologies in the anti-spam space. Can you
identify three that are stronger than SPF, as measured on the metrics of
- saving bandwidth
- saving CPU
- stopping worms and viruses?

The reason I am concerned about saving bandwidth and CPU is very simple.

pobox.com's bandwidth expenses used to rise linearly with the number of
customers. This was back in the good old days.

But with spam, our bandwidth costs have come completely unglued from the
number of customers. The costs are now entirely dependent on the whims
of the spammers and worms out in the field. When MyDoom hit, our peak
bandwidth went up by 60%.

Who pays for this? Our customers.

I like SPF because it lets me go back from O(random) to O(N).

I have evaluated the 40 or so technologies in my head; but that is not
enough. This week I will share my analysis publically. Then all the
cards will be on the table, and it will be time to play a hand. Saying
"well, if none of them are good enough, let's just wait until something
better comes along" is not a winning strategy.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

B.Candler at pobox

Feb 23, 2004, 9:58 AM

Post #2 of 10 (3059 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

On Mon, Feb 23, 2004 at 10:50:26AM -0500, Meng Weng Wong wrote:
> On Mon, Feb 23, 2004 at 03:31:52PM +0000, Brian Candler wrote:
> |
> | But what will you do next after SPF?
> |
>
> I don't like the metaphor of escalation, which is open-ended.
>
> I prefer to think of it as playing chess, which converges to a closed result.

I guess I've not got a clear enough picture of the end-game.

> Here's a view of the board a few moves ahead: http://spf.pobox.com/faq.html#churn

As a spammer, my next moves would be:
1. send spam with a null envelope sender
2. send spam from randomuser [at] myisp
3. sign up for free dial-up accounts and send mail from dispos123 [at] myisp
4. register new domains and keep them around for a year before using them

The first three impose no more accountability than is currently present; you
still have to look at IP addresses in Received: headers to track back to the
sender.

The fourth gives a paper trail to the registrar. The quality of information
stored there is typically worse than is held at ISPs.

I don't think the spammer has been seriously inconvenienced.

> There are approximately 40 technologies in the anti-spam space. Can you
> identify three that are stronger than SPF, as measured on the metrics of
> - saving bandwidth
> - saving CPU
> - stopping worms and viruses?

Well, since I disagree that SPF will have any significant impact any of
them, then "any three of the other 39" will do :-)

Yes it may have a short-term impact. I still don't see it does anything
long-term, certainly not by itself. Spam, worms, and viruses, will be
written to have more legitimate-looking envelope senders.

For "stopping worms and viruses", content filtering goes at the top of the
list. Unfortunately, since you've already used the bandwidth in accepting
the worm so you can scan it, you don't save any. And you burn lots of CPU.

> The reason I am concerned about saving bandwidth and CPU is very simple.

And valid. I nearly cancelled my pobox.com account because of the
overwhelming amount of spam it got, which I could not auto-filter without
losing mail that I wanted to keep. Now that pobox rejects with bounces, I am
much happier to let it filter away.

As a pobox.com customer, if you implement SPF, I'm either going to have to
relay via your server for outgoing mail (SMTP AUTH), or you will publish a
per-user SPF record for me listing my ISP's netblocks - which I'll have to
keep up to date myself, if my ISP doesn't publish an SPF record you can
reference using 'include'.

If I relay through you, then you could do sender-rewriting. That would
prevent the spammer using null envelope sender, and eliminate all the
bandwidth from joe-job bounces. Even SPF won't achieve that until/unless it
is widely deployed.

Regards,

Brian.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

wayne at midwestcs

Feb 23, 2004, 11:13 AM

Post #3 of 10 (3074 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

In <20040223165803.GC2845 [at] uk> Brian Candler <B.Candler [at] pobox> writes:

> On Mon, Feb 23, 2004 at 10:50:26AM -0500, Meng Weng Wong wrote:
>> On Mon, Feb 23, 2004 at 03:31:52PM +0000, Brian Candler wrote:
>> |
>> | But what will you do next after SPF?

> I guess I've not got a clear enough picture of the end-game.
>
> As a spammer, my next moves would be:
> 1. send spam with a null envelope sender

This doesn't help the spammer too much. SPF will simply use the HELO
domain instead of the envelope-from domain. Many MTAs will check for
valid MX records in the HELO domain, which means you are basically
back to the same problems of using an envelope-from.

> 2. send spam from randomuser [at] myisp
> 3. sign up for free dial-up accounts and send mail from dispos123 [at] myisp

These are solvable by myisp.net, and if they don't solve it, SPF allows
RHSBLs to work reliably.

> 4. register new domains and keep them around for a year before using them

This is exactly what we want. Domain registration requires providing
a lot more information which can be checked. Registrars can be
subpeanaed to help trace back the spammer to the real source.

> The first three impose no more accountability than is currently present; you
> still have to look at IP addresses in Received: headers to track back to the
> sender.

No, with SPF you can look at the domain and use RHSBLs. DNSBLs will
still be very useful, but right now, RHSBLs are very useful because it
is so easy to forge domains.

> The fourth gives a paper trail to the registrar. The quality of information
> stored there is typically worse than is held at ISPs.

I can not determine much about your account information with
POBox.com. I can, however, quickly determine a lot of information
about the registration of pobox.com. In particular, I can look at who
is the registrar and make judgements based on that. If you have a
domain registered via GoDaddy, I know that either you are not a
spammer, or if you are, your domain will quickly be nuked. I can also
look at the name servers you are using. These are much harder/slower
to update and change. Tracking nameservers used by spammers is
already a very useful technique for judging a domain name.

> I don't think the spammer has been seriously inconvenienced.

On the contrary, I think the spammer will be increasingly boxed into a
corner. More importantly, with SPF and/or SRS+callbacks the spammer
will have to use someone elses domain rather than mine.

-wayne

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

B.Candler at pobox

Feb 23, 2004, 11:24 AM

Post #4 of 10 (3066 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

On Mon, Feb 23, 2004 at 12:13:53PM -0600, wayne wrote:
> > As a spammer, my next moves would be:
> > 1. send spam with a null envelope sender
>
> This doesn't help the spammer too much. SPF will simply use the HELO
> domain instead of the envelope-from domain. Many MTAs will check for
> valid MX records in the HELO domain, which means you are basically
> back to the same problems of using an envelope-from.

I don't see how validating HELO can in any way confirm that a bounce is
really a bounce, not spam. A spammer can put whatever they like in HELO,
including whatever the ISP's MTA would put in there when returning a bounce.
And if you reject bounces because you don't like what you see in HELO, you
will almost certainly lose valid bounces, and you are in violation of RFC
2821.

> > 2. send spam from randomuser [at] myisp
> > 3. sign up for free dial-up accounts and send mail from dispos123 [at] myisp
>
> These are solvable by myisp.net, and if they don't solve it, SPF allows
> RHSBLs to work reliably.

But if you're going to blacklist the entire ISP, you can just do it by IP
address anyway.

> > 4. register new domains and keep them around for a year before using them
>
> This is exactly what we want. Domain registration requires providing
> a lot more information which can be checked.

I don't think so! You will find a lot of forged, rubbish information in
there, certainly. The only thing of value will be a credit card number, and
the registrar ain't going to give that away without a fight.

> I can not determine much about your account information with
> POBox.com. I can, however, quickly determine a lot of information
> about the registration of pobox.com

Yes, because they are not spammers, and gave true information when
registering it.

Brian.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

wayne at midwestcs

Feb 23, 2004, 12:03 PM

Post #5 of 10 (3068 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

In <20040223182417.GB3599 [at] uk> Brian Candler <B.Candler [at] pobox> writes:

> On Mon, Feb 23, 2004 at 12:13:53PM -0600, wayne wrote:
>> > As a spammer, my next moves would be:
>> > 1. send spam with a null envelope sender
>>
>> This doesn't help the spammer too much. SPF will simply use the HELO
>> domain instead of the envelope-from domain. Many MTAs will check for
>> valid MX records in the HELO domain, which means you are basically
>> back to the same problems of using an envelope-from.
>
> I don't see how validating HELO can in any way confirm that a bounce is
> really a bounce, not spam.

Validating the HELO won't say if the bounce is spam or not, but then,
neither will checking the SPF record nor checking SRS.

Validating the HELO lets you know that the domain name being used to
send the bounce, and thus, you can use RHSBLs.

> A spammer can put whatever they like in HELO,
> including whatever the ISP's MTA would put in there when returning a bounce.

A spammer can't put anything they want in the HELO field and still
pass the validation.

> And if you reject bounces because you don't like what you see in HELO, you
> will almost certainly lose valid bounces, and you are in violation of RFC
> 2821.

I do not believe that rejecting email based on the HELO string is a
violation of RFC2821, as long as it is done for "policy reasons" and
not simply because the IP address of HELO domain doesn't match the
SMTP client IP address. RFC2821 is kind of muddled here. There is a
paragraph in section 4.1.4 says "However, the server MUST NOT refuse
to accept a message for this reason if the [domain/ip-address]
verification fails: [...]", however this appears to be talking about
rejecting at the HELO/EHLO command time, while SPF deals with the MAIL
FROM:. RFC2821 also talks about it being ok to issue a 550 rejection
to the HELO/EHLO for "policy reasons."

>> These are solvable by myisp.net, and if they don't solve it, SPF allows
>> RHSBLs to work reliably.
>
> But if you're going to blacklist the entire ISP, you can just do it by IP
> address anyway.

It is far easier to add a single line to a RHSBL to block an entire
domain (and their subdomains), than it is to track their constantly
changing IP address allocations.

>> > 4. register new domains and keep them around for a year before using them
>>
>> This is exactly what we want. Domain registration requires providing
>> a lot more information which can be checked.
>
> I don't think so! You will find a lot of forged, rubbish information in
> there, certainly. The only thing of value will be a credit card number, and
> the registrar ain't going to give that away without a fight.

You snipped the two things in the domain registration that are very
hard to forge: the name servers for the domain and the registrar.
IANA says that the rest of the information must also be valid and
recently registrars have started to widely enforce this rule.

-wayne

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

B.Candler at pobox

Feb 23, 2004, 12:15 PM

Post #6 of 10 (3037 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

On Mon, Feb 23, 2004 at 01:03:48PM -0600, wayne wrote:
> > I don't see how validating HELO can in any way confirm that a bounce is
> > really a bounce, not spam.
>
> Validating the HELO won't say if the bounce is spam or not, but then,
> neither will checking the SPF record nor checking SRS.

SRS will, if you accept bounces only to SRS-signed addresses.

> > A spammer can put whatever they like in HELO,
> > including whatever the ISP's MTA would put in there when returning a bounce.
>
> A spammer can't put anything they want in the HELO field and still
> pass the validation.

I meant, they can put whatever specific value is needed in HELO to make the
message get through.

Can you give an example of how SPF prevents a spammer sending spam
pretending to be a bounce?

> RFC2821 is kind of muddled here. There is a
> paragraph in section 4.1.4 says "However, the server MUST NOT refuse
> to accept a message for this reason if the [domain/ip-address]
> verification fails: [...]", however this appears to be talking about
> rejecting at the HELO/EHLO command time

it says "must not refuse to ACCEPT A MESSAGE" [i.e. at RCPT TO/DATA time],
my emphasis.

>, while SPF deals with the MAIL
> FROM:. RFC2821 also talks about it being ok to issue a 550 rejection
> to the HELO/EHLO for "policy reasons."

Which you're not proposing to do; you're proposing to reject the incoming
message itself, rather than or as well as the HELO/EHLO.

> > But if you're going to blacklist the entire ISP, you can just do it by IP
> > address anyway.
>
> It is far easier to add a single line to a RHSBL to block an entire
> domain (and their subdomains), than it is to track their constantly
> changing IP address allocations.

I say the opposite. It's much easier to block an ISP by their (very
occasional) registry allocations, as opposed to the thousands of domains
they may host, growing daily.

> You snipped the two things in the domain registration that are very
> hard to forge: the name servers for the domain and the registrar.
> IANA says that the rest of the information must also be valid and
> recently registrars have started to widely enforce this rule.

So it will be better than the information ISPs hold about their customers? I
really do doubt it. How are they enforcing it? When you sign up for a domain
on-line, what sort of checks do they do on your name and address?

The nameservers are public information in the DNS, and since the NS records
for 'vanity.com' may well point to ns1/2.vanitydomain.com, you need to
resolve to A records again. In which case you're back to IP blacklisting.

I admit there might be scaling benefits here, in that the number of
registrars might be smaller than the number of ISPs. I don't know the actual
figures. But I certainly trust registrars less than ISPs, and I trust them
less to do anti-spammer enforcement.

Regards,

Brian.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

wayne at midwestcs

Feb 23, 2004, 1:06 PM

Post #7 of 10 (3065 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

In <20040223191503.GA3846 [at] uk> Brian Candler <B.Candler [at] pobox> writes:

> On Mon, Feb 23, 2004 at 01:03:48PM -0600, wayne wrote:
>> > I don't see how validating HELO can in any way confirm that a bounce is
>> > really a bounce, not spam.
>>
>> Validating the HELO won't say if the bounce is spam or not, but then,
>> neither will checking the SPF record nor checking SRS.
>
> SRS will, if you accept bounces only to SRS-signed addresses.

No, SRS will not confirm that the bounce isn't spam. If you email a
spammer, the message you get back can be anything, inclucing spam. If
you publish or leak your SRS envelope-froms, a spammer can send spam
using it.

Validating the HELO domain, the SPF record or the SRS address do not,
in and of themselves, have anything to do with spam. The can,
however, make it harder for spammers to use invalid data to get you to
accept their spam.

> Can you give an example of how SPF prevents a spammer sending spam
> pretending to be a bounce?

SPF doesn't prevent spam, nor does it prevent non DSN from being sent
using a null MAIL FROM:. SPF helps prevent a domain name from being
forged.

>> RFC2821 is kind of muddled here. There is a
>> paragraph in section 4.1.4 says "However, the server MUST NOT refuse
>> to accept a message for this reason if the [domain/ip-address]
>> verification fails: [...]", however this appears to be talking about
>> rejecting at the HELO/EHLO command time
>
> it says "must not refuse to ACCEPT A MESSAGE" [i.e. at RCPT TO/DATA time],
> my emphasis.

Again, the MAIL FROM and RCPT TO commands allow you to reject mail for
"policy reasons". I see no reason why SPF checks based on many
things, including the HELO domain, are not valid "policy reasons".

SPF doesn't not reject a message just because the helo domain and the
client IP address don't match. These two don't even need to match in
order to pass SPF checks and SPF was developed in large part because
the IP addresses that are used to send mail are often not the same as
the A/MX records listed for the domain.

>> It is far easier to add a single line to a RHSBL to block an entire
>> domain (and their subdomains), than it is to track their constantly
>> changing IP address allocations.
>
> I say the opposite. It's much easier to block an ISP by their (very
> occasional) registry allocations, as opposed to the thousands of domains
> they may host, growing daily.

Sometimes it will be easier to squash spam via IP addresses, in which
case DNSBLs are great. There are many times which it is not so easy,
and SPF lets makes RHSBLs much more useful. I don't care how you
squash the bug as long as the bug gets squashed.

>> You snipped the two things in the domain registration that are very
>> hard to forge: the name servers for the domain and the registrar.
>> IANA says that the rest of the information must also be valid and
>> recently registrars have started to widely enforce this rule.
>
> So it will be better than the information ISPs hold about their
> really do doubt it. How are they enforcing it?

There are many free email providers out there that require almost no
information about you. The minimum amount of information needed to
register a domain is more than a very large number of ISPs.

As far as enforcement of registrar information, all you need to do is
complain to the registrar. If the registrar doesn't make sure the
information is validated, they can get in trouble with the IANA. This
enforcement is a fairly recent change, starting sometime last
summer/fall. I suspect that this enforcement will only get stronger
in the future. There is no equivalent requirement on
ISPs/companies/free-email providers enforcing valid information.

> The nameservers are public information in the DNS, and since the NS records

The name servers in the whois data is different than the name servers
in the DNS information. The whois NS info is easier to track and control.
The whois NS info can also be easily taken over by the registrar and
locked so that the domain owner can't change it.

-wayne

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

B.Candler at pobox

Feb 24, 2004, 2:54 AM

Post #8 of 10 (3032 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

On Mon, Feb 23, 2004 at 02:06:24PM -0600, wayne wrote:
> > Can you give an example of how SPF prevents a spammer sending spam
> > pretending to be a bounce?
>
> SPF doesn't prevent spam, nor does it prevent non DSN from being sent
> using a null MAIL FROM:. SPF helps prevent a domain name from being
> forged.

Right. So how does validating the HELO name help anyone then? After all,
it's not used as part of the delivery process.

OTOH, validating the HELO name risks that you lose valid bounces, just
because the HELO name does not match your "policy".

So, I can see zero benefit plus non-zero loss. Why are we doing it?

Brian.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

wayne at midwestcs

Feb 24, 2004, 4:00 AM

Post #9 of 10 (3085 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

[note: this is thread has drift away from anything to do with SRS and
back to SPF stuff. Followups have been set.)

In <20040224095420.GC4143 [at] uk> Brian Candler <B.Candler [at] pobox> writes:

> Right. So how does validating the HELO name help anyone then? After all,
> it's not used as part of the delivery process.

When a bounce is being sent via a MAIL FROM:<>, it is the MTA that is
sending the email, not a person. Therefore, SPF needs to validate the
domain name of the SMTP client, which is supplied by the HELO/EHLO
command.

> OTOH, validating the HELO name risks that you lose valid bounces, just
> because the HELO name does not match your "policy".

I guess that all depends on what you mean by a "valid bounce". If the
MTA is claiming to be some domain that it isn't, I'm not sure I
consider it to be valid.

-wayne

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

jcouzens at obscurity

Mar 1, 2004, 12:42 AM

Post #10 of 10 (3048 views)
Permalink

Re: More on Philosophy of SPF [In reply to]

On Mon, 2004-02-23 at 08:58, Brian Candler wrote:

> If I relay through you, then you could do sender-rewriting. That would
> prevent the spammer using null envelope sender, and eliminate all the
> bandwidth from joe-job bounces. Even SPF won't achieve that until/unless it
> is widely deployed.

Not to discredit your statement, but to provide an alternate opinion by
someone with a fair bit of credibility, Mark Wegman, researcher at IBM's
T.J. Watson Research Centre in Hawthorne, N.Y. had this to say of SPF in
a recent interview:

> "Even if SPF is just partially adopted, I think it will help us"

You can read the full article here:

http://news.earthweb.com/IAR/article.php/3302361

Cheers,

James

--
James Couzens,
Programmer
-----------------------------------------------------------------
http://libspf.org -- ANSI C Sender Policy Framework library
http://libsrs.org -- ANSI C Sender Rewriting Scheme library
-----------------------------------------------------------------
PGP: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBD3BF855

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=srs-discuss [at] v2

Attachments:

signature.asc (0.18 KB)

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads