Mailing List Archive: SpamAssassin: users

Spam filtering similar to SPF, less breakage

1 2

View All

Index | Next | Previous | View Threaded

Darxus at ChaosReigns

Feb 8, 2010, 7:08 PM

Post #1 of 31 (1282 views)
Permalink

Spam filtering similar to SPF, less breakage

You get an email delivered from 64.71.152.40 (last untrusted
relay). You look up the DNS A record for that IP, and get
mail.chaosreigns.com. Then you look up the DNS PTR record of
40.152.71.64.designatedsender.mail.chaosreigns.com, and if it's
127.0.0.1, it's a legit email sender and gets some negative SA score.
Otherwise it's not, and gets some positive SA score (low at first until
adoption spreads).

So it's not tied to the SMTP MAIL FROM or anything.
Forwarding doesn't break.

Eventually you reject all email from IPs without such records.

Obviously you'd need a blacklist of spammer domains that list spamming
IPs as legit senders. Not an RHSBL / MAIL FROM blacklist, but a blacklist
where, when the A record of a delivering IP is in a blacklisted domain, the
mail gets rejected.

I am not at all attached to the format of the PTR record and would
like suggestions.

Is there any way this wouldn't be very useful?

--
"Of course there's strength in numbers. But there's strength in sharp
weaponry too. Ironically, this lead to what we call 'civilization'."
- spore
http://www.ChaosReigns.com

ram at netcore

Feb 8, 2010, 11:11 PM

Post #2 of 31 (1240 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

uhlar at fantomas

Feb 9, 2010, 12:05 AM

Post #3 of 31 (1246 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On 08.02.10 22:08, Darxus [at] ChaosReigns wrote:
> You get an email delivered from 64.71.152.40 (last untrusted
> relay). You look up the DNS A record for that IP, and get

You won't look up A records for an IP, IP address do NOT have A records.
You can look up PTR (or any other record) for 40.152.71.64.in-addr.arpa.

> mail.chaosreigns.com. Then you look up the DNS PTR record of
> 40.152.71.64.designatedsender.mail.chaosreigns.com, and if it's
> 127.0.0.1, it's a legit email sender and gets some negative SA score.

Then I will look A record of mail.chaosreigns.com and see if it's
64.71.152.40.

If you compare these two, you see that I need to control the same domains no
matter which wsay I go. Just now I can do what you propose (and many
spammers can too) but that doesn't mean my mail is legitimate.

> Otherwise it's not, and gets some positive SA score (low at first until
> adoption spreads).
>
> So it's not tied to the SMTP MAIL FROM or anything.
> Forwarding doesn't break.

What do you mean by this?
Do you want to implement new way of defining which hosts are permitted to
send e-mail? There already are attempts to do this, with false positives and
negatives. Yours is a bit complicated and new which means everyone would
need to implement this (otherwise he'd get false positives on his outgoing
mail). Therefore I think it won't work this way.
If you want to implement new way of defining which hosts are permitted to
send e-mail from which domain, the same applies.

--
Matus UHLAR - fantomas, uhlar [at] fantomas ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"They say when you play that M$ CD backward you can hear satanic messages."
"That's nothing. If you play it forward it will install Windows."

per at computer

Feb 9, 2010, 12:20 AM

Post #4 of 31 (1243 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

ram wrote:

> Apparently you want to check if non mail servers are sending mails ..
> but what percentage of spams today come from non mail servers ?

The vast majority, I would say.

/Per Jessen, Zürich

cgregory at hwcn

Feb 9, 2010, 6:24 AM

Post #5 of 31 (1229 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Mon, 2010-02-08 at 22:08 -0500, Darxus [at] ChaosReigns wrote:
> You get an email delivered from 64.71.152.40 (last untrusted
> relay). You look up the DNS A record for that IP, and get
> mail.chaosreigns.com.

That's a PTR lookup, but let's press on....

> Then you look up the DNS PTR record of
> 40.152.71.64.designatedsender.mail.chaosreigns.com, and if it's
> 127.0.0.1, it's a legit email sender and gets some negative SA score.

What you describe here is functionally similar to an SPF lookup with a
'pass' result..... The server provides positive verification
that the listed IP is a legitimate sender for that domain.

> Otherwise it's not, and gets some positive SA score (low at first until
> adoption spreads).

As long as 'otherwise' is a definitive 'fail' response from an SPF (or
equivalent) server, and not merely an absence of SPF server....

> So it's not tied to the SMTP MAIL FROM or anything.

Your method would allow 'spoofing' so that a spammer who hacks a
legitimate server can use a valid return address on a different domain,
but still the mail would receive a 'passing' grade. At least, with SPF,
the spammer must forge an address on the hacked domain, which increases
the likelihood of detection....

> Forwarding doesn't break.

Ah, so you want to allow 'legitimate' forwarding, but not allow spammers
to 'forward' their mail? Good luck with that. The only way to make it work
for the legitimate sender, but not for spammers is to have a mechanism
built-in to the forwarding server that encapsulates or rewrites the
envelope 'From' address.....

> Eventually you reject all email from IPs without such records.

In a perfect world.....

> Obviously you'd need a blacklist of spammer domains that list spamming
> IPs as legit senders.

And you would be playing the same 'musical chairs' game with new domains
created by spammers on a daily basis. All the same flaws of SPF, and no
greater benefit.

> Is there any way this wouldn't be very useful?

Is there any place where SPF does not do the same job, other than mail
forwarding?

- Charles

rwmaillists at googlemail

Feb 9, 2010, 6:24 AM

Post #6 of 31 (1227 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Mon, 8 Feb 2010 22:08:10 -0500
Darxus [at] ChaosReigns wrote:

> You get an email delivered from 64.71.152.40 (last untrusted
> relay). You look up the DNS A record for that IP, and get
> mail.chaosreigns.com. Then you look up the DNS PTR record of
> 40.152.71.64.designatedsender.mail.chaosreigns.com, and if it's
> 127.0.0.1, it's a legit email sender and gets some negative SA score.
> Otherwise it's not, and gets some positive SA score (low at first
> until adoption spreads).

You've mixed-up A record and PTR record.

Checking for full-circle DNS already does most of this. What your
scheme would do is check for otherwise legitimate servers that have
been compromised and are delivering direct-to-mx.

Darxus at ChaosReigns

Feb 9, 2010, 8:42 AM

Post #7 of 31 (1221 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

I apparently need to clarify that I think this is a good idea because I am
concerned about the number of people (who control DNS records) who are very
strongly against creating SPF records specifically because of forwarding
breakage. The email I got in response to my request for my employer to
create an SPF record included the word "abomination". From a friend.
I don't entirely agree, but it is a problem for adoption.

This is entirely an attempt to replicate the functionality of SPF without
breaking forwarding, and without causing other problems that might
discourage adoption.

I set this up for my mail server (using mtx instead of designatedsender):

$ host -t PTR 64.71.152.40
40.152.71.64.in-addr.arpa domain name pointer panic.chaosreigns.com.

$ host -t A 40.152.71.64.mtx.panic.chaosreigns.com
40.152.71.64.mtx.panic.chaosreigns.com has address 127.0.0.1

All it took was creating a single record in bind:

40.152.71.64.mtx.panic.chaosreigns.com. IN A 127.0.0.1

I'll define it slightly differently:
127.0.0.1 is a pass (negative SA score).
not found is a fail (positive SA score).
127.0.0.0 is a fail (positive SA score).
Anything else is undefined (0 SA score) for future options.

I'd still appreciate feedback on the format of the A record.

On 02/09, RW wrote:
> You've mixed-up A record and PTR record.

Yes. Embarrassing.

> Checking for full-circle DNS already does most of this.

My home dynamic cablemodem address passes full-circle DNS. But not this.
So this is far more useful for checking if an IP is a legitimate sending
mail server.

> What your
> scheme would do is check for otherwise legitimate servers that have
> been compromised and are delivering direct-to-mx.

An otherwise legitimate but compromised mail server would not be detected
by this. I'm curious why you interpreted it differently.

On 02/09, Charles Gregory wrote:
> On Mon, 2010-02-08 at 22:08 -0500, Darxus [at] ChaosReigns wrote:

> What you describe here is functionally similar to an SPF lookup with a
> 'pass' result..... The server provides positive verification that the
> listed IP is a legitimate sender for that domain.

Yes.

> As long as 'otherwise' is a definitive 'fail' response from an SPF (or
> equivalent) server, and not merely an absence of SPF server....

Yes. Definitive "fail".

> Your method would allow 'spoofing' so that a spammer who hacks a
> legitimate server can use a valid return address on a different domain,
> but still the mail would receive a 'passing' grade. At least, with SPF,
> the spammer must forge an address on the hacked domain, which increases
> the likelihood of detection....

Yes. I would blacklist domains that "pass" hacked servers. Just as IPs of
hacked servers are blacklisted. They're sending spam, and need to be
fixed.

>> Forwarding doesn't break.
>
> Ah, so you want to allow 'legitimate' forwarding, but not allow spammers
> to 'forward' their mail? Good luck with that. The only way to make it
> work for the legitimate sender, but not for spammers is to have a
> mechanism built-in to the forwarding server that encapsulates or rewrites
> the envelope 'From' address.....

Encapsulating or rewriting the envelope 'From' address seems significantly
less likely to be adopted from what I've read.

>> Obviously you'd need a blacklist of spammer domains that list spamming
>> IPs as legit senders.
>
> And you would be playing the same 'musical chairs' game with new domains
> created by spammers on a daily basis. All the same flaws of SPF, and no
> greater benefit.

Same domain blacklisting issues as SPF, yes.

I am not very concerned about the throw-away domains because
I'll reject all mail from domains not at least 10 days old. 10+
day old domains are already listed as 127.0.2.3 records from
example.com.hostkarma.junkemailfilter.com.

I believe the benefit of not breaking forwarding is sufficient to make it
much more useful than SPF for spam filtering. I've come across enough
people, personally, recently, in trying to block (some = positive SA
score) emails without an SPF "pass", who are not willing to ever implement
SPF due to breaking forwarding that I believe this would be useful.

>> Is there any way this wouldn't be very useful?
>
> Is there any place where SPF does not do the same job, other than mail
> forwarding?

No. But as I said, I am concerned about the potential for wide spread
adoption of SPF due to forwarding breakage enough that I think this would
be much more useful.

On 02/09, Matus UHLAR - fantomas wrote:
> On 08.02.10 22:08, Darxus [at] ChaosReigns wrote:
> > So it's not tied to the SMTP MAIL FROM or anything.
> > Forwarding doesn't break.
>
> What do you mean by this?
> Do you want to implement new way of defining which hosts are permitted to
> send e-mail?

Yes.

> There already are attempts to do this, with false positives and
> negatives.

How is this not better than the other attempts?

> Yours is a bit complicated

How is it complicated? You need to create 1 record per sending mail
server, in the form:

<IP of mail server>.mtx.<hostname of mail server>

(And the IP needs to be reversed as in all other A records that list IPs.)

> and new which means everyone would
> need to implement this (otherwise he'd get false positives on his outgoing
> mail).

Yes.

> Therefore I think it won't work this way.

This is why I said SA scores for a fail should be small at first.

Do you see any reason this wouldn't be more quickly adopted than SPF?

--
"Life is but a walking shadow, a poor player that struts and frets his
hour upon the stage--and then is heard no more. It is a tale told by an
idiot, full of sound and fury, signifying nothing." - William Shakespeare
http://www.ChaosReigns.com

Darxus at ChaosReigns

Feb 9, 2010, 8:51 AM

Post #8 of 31 (1225 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

Another point of clarification on why I think this will work:

I'm running SA as a pre-queue content_filter from postfix, so when SA
decides an email is spam, non-forged senders get an error, without sending
backscatter to forged addresses.

When I implement this, senders will get an error asking them to ask
their administrator to create the necessary record.

And I see no reason for that administrator not to create that record.

The directions I followed to set up SA this way are half way down this
page:
http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd
(Be sure to do the --sef and X-Envelope-From bits.)

--
"I'd rather be happy than right any day."
- Slartiblartfast, The Hitchhiker's Guide to the Galaxy
http://www.ChaosReigns.com

kdeugau at vianet

Feb 9, 2010, 9:31 AM

Post #9 of 31 (1224 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

Darxus [at] ChaosReigns wrote:
> I apparently need to clarify that I think this is a good idea because I am
> concerned about the number of people (who control DNS records) who are very
> strongly against creating SPF records specifically because of forwarding
> breakage. The email I got in response to my request for my employer to
> create an SPF record included the word "abomination". From a friend.
> I don't entirely agree, but it is a problem for adoption.
>
> This is entirely an attempt to replicate the functionality of SPF without
> breaking forwarding, and without causing other problems that might
> discourage adoption.
>
>
> I set this up for my mail server (using mtx instead of designatedsender):
>
> $ host -t PTR 64.71.152.40
> 40.152.71.64.in-addr.arpa domain name pointer panic.chaosreigns.com.
>
> $ host -t A 40.152.71.64.mtx.panic.chaosreigns.com
> 40.152.71.64.mtx.panic.chaosreigns.com has address 127.0.0.1
>
> All it took was creating a single record in bind:
>
> 40.152.71.64.mtx.panic.chaosreigns.com. IN A 127.0.0.1
>
>
> I'll define it slightly differently:
> 127.0.0.1 is a pass (negative SA score).
> not found is a fail (positive SA score).
> 127.0.0.0 is a fail (positive SA score).
> Anything else is undefined (0 SA score) for future options.
>
>
> I'd still appreciate feedback on the format of the A record.

Can you be a little more detailed on exactly which pieces of information
you're looking at, and where you're getting them from?

So far as I've been able to make out from your descriptions so far, this
is a smallish subset of the functionality SPF provides, without the
disadvantage of overloading TXT records.

Consider: Spammer has control of a PC on a small(ish)-business LAN
whose NAT gateway also handles their mail. (Exchange server, for
instance - far too common a configuration IMO.)

Spammer mail originates from 10.0.0.2 (static IP assigned by ISP).

PTR record is 2.0.0.10.in-addr.arpa -> exchange.smallbusiness.com

2.0.0.10.mtx.exchange.smallbusiness.com -> 127.0.0.1 because this is the
recognized designated outbound relay for Small Business's legitimate
mail, and they've followed your proposal.

How is the spam to be *not* considered a legitimate sender in this case?
Even if the Exchange server isn't actually processing the email, its
public IP will still be the originating IP of the message.

SPF at least ties some aspect of the message itself to the relay IP in
some way. (Whether that's good or bad I'll ignore for the moment.)

-kgd

Darxus at ChaosReigns

Feb 9, 2010, 10:07 AM

Post #10 of 31 (1229 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On 02/09, Kris Deugau wrote:
> Spammer mail originates from 10.0.0.2 (static IP assigned by ISP).
>
> PTR record is 2.0.0.10.in-addr.arpa -> exchange.smallbusiness.com
>
> 2.0.0.10.mtx.exchange.smallbusiness.com -> 127.0.0.1 because this is the
> recognized designated outbound relay for Small Business's legitimate
> mail, and they've followed your proposal.
>
> How is the spam to be *not* considered a legitimate sender in this case?
> Even if the Exchange server isn't actually processing the email, its
> public IP will still be the originating IP of the message.

Blacklist the validating domain smallbusiness.com. Reject all email that
has a *.mtx.*.smallbusiness.com record. Just as you would blacklist
the sending IP for spamming. As with SPF, I expect this to be quite a lot
easier than maintaining a blacklist of spamming IPs. If I'm wrong on that
one point, this is useless.

Would you still like more detail on what pieces of information I'm looking
at, and where I'm getting them from?

--
"Anarchy is based on the observation that since few are fit to rule
themselves, even fewer are fit to rule others." -Edward Abbey
http://www.ChaosReigns.com

cgregory at hwcn

Feb 9, 2010, 10:25 AM

Post #11 of 31 (1218 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010, Darxus [at] ChaosReigns wrote:
> My home dynamic cablemodem address passes full-circle DNS. But not this.
> So this is far more useful for checking if an IP is a legitimate sending
> mail server.

So rather than mimicing SPF, you want to mimic the effect of various
IP-based blacklists to which an ISP can report all of its 'unauthorized'
IP's (typicalyl dynamic IP blocks)?

The obvious drawback is the same for your system as the already existing
dynamic-ip blackilsts, which is that it only works when domains take the
time and trouble to register either their 'authorized' (in your case) or
'unauthorized' (for the blacklist) ranges of IP's. Large companies which
frequently reorganize their IP blocks will shy away from such a system,
and smaller companies will lack the time/resources to implement anything
that isn't 'out of the box'.

> Encapsulating or rewriting the envelope 'From' address seems
> significantly less likely to be adopted from what I've read.

Unlikely in many cases, but I would quibble over 'less'. Overall, most
corporate minds seem to think that they prefer false negatives to false
positives, so they are extremely reluctant to adopt any strategy that
increases the chance of a false positive, even under such strange
conditions as a hack.... Indeed, given that there are no issues of false
positives from rewriting the envelope sender, one could argue that it is
the change *most* likely to be adopted, and therefore it has significance
that it is not being adopted by everyone....

> I am not very concerned about the throw-away domains because
> I'll reject all mail from domains not at least 10 days old.

Another feature already covered by a blacklist, and, already being
bypassed by numerous spammers who are smart enough to buy a domain name
months before they use it. (shrug)

> I believe the benefit of not breaking forwarding is sufficient to make
> it much more useful than SPF for spam filtering. I've come across
> enough people, personally, recently, in trying to block (some = positive
> SA score) emails without an SPF "pass", who are not willing to ever
> implement SPF due to breaking forwarding that I believe this would be
> useful.

But *why* do they refuse to 'fix' their forwarding? I strongly
suspect that the same reasoning would apply to their decision to
not implement SPF or your IP-based filter idea.

- C

cgregory at hwcn

Feb 9, 2010, 10:27 AM

Post #12 of 31 (1214 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010, Darxus [at] ChaosReigns wrote:
> When I implement this, senders will get an error asking them to ask
> their administrator to create the necessary record.

Those administrators will say that they do not have control over DNS,
because that's done at a higher organizational level, or that they don't
want to do something that is not 'standard' and will tell their users to
find another way to communicate with you. Same BS that is used to dodge
SPF. It's not that your idea is 'bad', it's just not that much different
from SPF in terms of the roadbblocks to its effective adoption.....

Don't blame you for trying. Actually wish it worked. :)

- Charles

Darxus at ChaosReigns

Feb 9, 2010, 11:06 AM

Post #13 of 31 (1218 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On 02/09, Darxus [at] ChaosReigns wrote:
> Anything else is undefined (0 SA score) for future options.

This won't work. Everything lacking a 127.0.0.1 record needs to be a
"fail", otherwise spammers can create undefined records to circumvent this.

On 02/09, Charles Gregory wrote:
> So rather than mimicing SPF, you want to mimic the effect of various
> IP-based blacklists to which an ISP can report all of its 'unauthorized'
> IP's (typicalyl dynamic IP blocks)?

Basically, except of course that the default, when not participating, is
effectively blacklisting (in the sense in which SA uses blacklists to
increase the spammines score of an email).

> The obvious drawback is the same for your system as the already existing
> dynamic-ip blackilsts, which is that it only works when domains take the
> time and trouble to register either their 'authorized' (in your case) or
> 'unauthorized' (for the blacklist) ranges of IP's. Large companies which
> frequently reorganize their IP blocks will shy away from such a system,
> and smaller companies will lack the time/resources to implement anything
> that isn't 'out of the box'.

I disagree. I can implement it now (in fact I expect to, which is why I'd
really like feedback on the A record format).

So I block some more spam, and I get some extra false positives, and the
senders get notified. And the senders can try to get the relevant MTX
records created if they want. And maybe along the way these MTX records
become a little more common, and I can increase the spam score of emails
without MTX records and block some more spam without further increasing
false positives. And maybe eventually, as a result of this iterative
process, enough people create MTX records that I can reject all email
without one.

Having looked over the scores of my non-spams, I'm comfortable with the
additional false positives up to an additional score of 4.5, as long as the
sender is notified and has some possibility of eliminating that additional
score.

Others might be more comfortable with an additional score of 1. Still, you
catch more spams, cause a few false positives, and the senders have the
possibility of fixing the false positives.

> Unlikely in many cases, but I would quibble over 'less'. Overall, most
> corporate minds seem to think that they prefer false negatives to false
> positives, so they are extremely reluctant to adopt any strategy that
> increases the chance of a false positive, even under such strange

I think you missed something important. Creating the records I suggest can
create no false positives. That point is critical to this idea.

Either they participate, and get their sending IPs whitelisted, or they
don't participate, and they don't get their IPs whitelisted and mail from
those IPs is more likely to be flagged as spam.

Unless they send spam, in which case they'll get blacklisted. Which would
be worse than not participating. So they should be really sure not to
whitelist any IPs that send spam. And of course just as in existing IP
blacklists, it is worth taking someone off a blacklist if they have
corrected their spam problem. Not flawless, but still way easier than
maintaining an IP blacklist.

> Another feature already covered by a blacklist, and, already being
> bypassed by numerous spammers who are smart enough to buy a domain name
> months before they use it. (shrug)

Good to know. It still doesn't concern me. This would still eliminate
spams from the vast majority of IPs for which spammers aren't delegated to
host PTR records. And if all spam has a verified paper trail (delivering
IP -> domain -> registrar who can be subpoened), I think the sending of
spam itself will be a lot easier to stop.

> But *why* do they refuse to 'fix' their forwarding? I strongly suspect

I don't know.

> that the same reasoning would apply to their decision to not implement
> SPF or your IP-based filter idea.

Except that it causes me no problems if they don't implement my
whitelisting.

On 02/09, Charles Gregory wrote:
> Those administrators will say that they do not have control over DNS,
> because that's done at a higher organizational level, or that they don't
> want to do something that is not 'standard' and will tell their users to
> find another way to communicate with you. Same BS that is used to dodge

I'm comfortable with that possibility. As I said, I expect benefit even
without people participating.

--
"The most merciful thing in the world, I think, is the inability of the
human mind to correlate all its contents."
http://www.ChaosReigns.com

rwmaillists at googlemail

Feb 9, 2010, 11:07 AM

Post #14 of 31 (1219 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010 11:42:07 -0500
Darxus [at] ChaosReigns wrote:

> On 02/09, RW wrote:

> > What your
> > scheme would do is check for otherwise legitimate servers that have
> > been compromised and are delivering direct-to-mx.
>
> An otherwise legitimate but compromised mail server would not be
> detected by this. I'm curious why you interpreted it differently.

A compromised webserver with full-circle DNS would be caught by this.
My point is that is the only class of spam that this could help with

Every thing else is either handled by full-circle and no DNS tests,
or can be easily worked around by spammers setting their own dns.

Darxus at ChaosReigns

Feb 9, 2010, 11:15 AM

Post #15 of 31 (1218 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On 02/09, RW wrote:
> A compromised webserver with full-circle DNS would be caught by this.
> My point is that is the only class of spam that this could help with

Ah, sorry, I thought you meant mail server.

Still, I don't understand why you're saying this.

It would also block, for example, spam from a dynamic cablemodem IP.

And everything that it didn't block could be blocked by blacklisting
domains which have MTX records for spamming IPs.

> Every thing else is either handled by full-circle and no DNS tests,

Full circle DNS tests don't block spam from quite a lot of IPs.

> or can be easily worked around by spammers setting their own dns.

Spammers can't create DNS records for hostnames for IPs they don't own
(don't have PTR authority delegated to them for).

--
"Life is but a walking shadow, a poor player that struts and frets his
hour upon the stage--and then is heard no more. It is a tale told by an
idiot, full of sound and fury, signifying nothing." - William Shakespeare
http://www.ChaosReigns.com

cgregory at hwcn

Feb 9, 2010, 11:45 AM

Post #16 of 31 (1215 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010, Darxus [at] ChaosReigns wrote:
>> So rather than mimicing SPF, you want to mimic the effect of various
>> IP-based blacklists to which an ISP can report all of its 'unauthorized'
>> IP's (typicalyl dynamic IP blocks)?
> Basically, except of course that the default, when not participating, is
> effectively blacklisting (in the sense in which SA uses blacklists to
> increase the spammines score of an email).

LOL.... Good luck with that. The first time that an important e-mail
correspondent (money!) is blocked by such a 'default' setting, the
sysadmins will be rushing to cripple this default action. You will never
succeed in introducing ANY spam filtering system that blocks mail based
upon an 'undecided' or 'neutral' status.

>> ..... Large companies which frequently reorganize their IP blocks will
>> shy away from such a system, and smaller companies will lack the
>> time/resources to implement anything that isn't 'out of the box'.
>
> I disagree. I can implement it now (in fact I expect to...

For your own use, sure. But that's just like SPF. A bunch of people will
use it, and a bunch, including ones that you still *really* want to
communicate with, will NOT. Have you figured out how you are going to sell
'hotmail' and 'gmail' on your idea? Or are you just going to block all
mail from them? Your choice. But if you have multiple users, well, you had
better choose conservatively..... (grin)

> So I block some more spam, and I get some extra false positives, and the
> senders get notified.

And every hotmail user will be writing to the you complaining they
have no way to talk hotmail into adopting your system, and begging you
to *remove* the 'block'. And if you stick to your guns, those are people
who, by no choice of their own, you will not hear from again. And if you
have users who stop hearing from good friends and relatives, they won't be
yuour users much longer.

As I said before, 'in a perfect world....' your idea would work.
But sadly, not in THIS one....

> I think you missed something important. Creating the records I suggest
> can create no false positives. That point is critical to this idea.

The FP's would occur on the systems *looking* for those records, and
scoring positively in SA for simply not finding them. Your argument, and
all of mine here, are not about the simplistic task of creating a DNS
record, but about the battle to have the scoring/testing protocol
implemented to make those records 'useful'.

> Either they participate, and get their sending IPs whitelisted, or they
> don't participate, and they don't get their IPs whitelisted and mail from
> those IPs is more likely to be flagged as spam.

Expressed as a 'whitelist only' mechanism that at least opens the door to
a possibility of avoiding the fears of implementation, but again, you face
the same lethargy and fear that keeps SPF from benig widely adopted....

> Good to know. It still doesn't concern me. This would still eliminate
> spams from the vast majority of IPs for which spammers aren't delegated to
> host PTR records. And if all spam has a verified paper trail (delivering
> IP -> domain -> registrar who can be subpoened), I think the sending of
> spam itself will be a lot easier to stop.

If ISP's just blocked port 25 access for their DSL customers a great deal
of spam would be stopped. And that is SO easy to do. Think about that. If
you can't even talk ISP's into such a simple approach, truly free of any
false positive, then how can you expect 'reason' regarding a system that
you admit has to have a bigger chance of false postitives....?

> Except that it causes me no problems if they don't implement my
> whitelisting.

Then it will cuase you no problem at all if they just don't implement
anything at all....?

>> Those administrators will say that they do not have control over DNS,
>> because that's done at a higher organizational level, or that they don't
>> want to do something that is not 'standard' and will tell their users to
>> find another way to communicate with you.
> I'm comfortable with that possibility. As I said, I expect benefit even
> without people participating.

Is this 'all about you'? If so, then just whitelist your own
correspondents and save us all the trouble of debating a DNS PTR system
that benefits no one but you. If you intend to introduce a new idea for
broad adoption, then it needs to benefit everyone. And 'everyone' is not
giong to be 'comfortable with that possibility' that you are.... :)

- C

kdeugau at vianet

Feb 9, 2010, 11:59 AM

Post #17 of 31 (1222 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

Darxus [at] ChaosReigns wrote:
> On 02/09, Kris Deugau wrote:
>> Spammer mail originates from 10.0.0.2 (static IP assigned by ISP).
>>
>> PTR record is 2.0.0.10.in-addr.arpa -> exchange.smallbusiness.com
>>
>> 2.0.0.10.mtx.exchange.smallbusiness.com -> 127.0.0.1 because this is the
>> recognized designated outbound relay for Small Business's legitimate
>> mail, and they've followed your proposal.
>>
>> How is the spam to be *not* considered a legitimate sender in this case?
>> Even if the Exchange server isn't actually processing the email, its
>> public IP will still be the originating IP of the message.
>
> Blacklist the validating domain smallbusiness.com. Reject all email that
> has a *.mtx.*.smallbusiness.com record. Just as you would blacklist
> the sending IP for spamming. As with SPF, I expect this to be quite a lot
> easier than maintaining a blacklist of spamming IPs. If I'm wrong on that
> one point, this is useless.

I'm still not seeing the whole picture; maybe you can explain the
difference between these two cases:

1) Legitimate sender, uses the NAT machine as the legitimate, designated
relayhost.

So far as a remote system is concerned, the mail originates from an IP
(10.0.0.2) with a certain rDNS (exchange.smallbusiness.com).

Remote system takes the IP, gets the PTR, and looks up
2.0.0.10.mtx.exchange.smallbusiness.com, and gets 127.0.0.1. Yay! Good
mail, let's let it through!

2) Spam, from an infected machine on the same LAN, either via relay
through the NAT box, or using its own SMTP engine to do direct-to-MX

So far as a remote system is concerned, the mail originates from an IP
(10.0.0.2) with a certain rDNS (exchange.smallbusiness.com).

Remote system takes the IP, gets the PTR, and looks up
2.0.0.10.mtx.exchange.smallbusiness.com, and gets 127.0.0.1. Yay! Good
mail, let's let it through!

> Would you still like more detail on what pieces of information I'm looking
> at, and where I'm getting them from?

Obviously I've missed *something* about what you've been trying to
describe, but I haven't seen any indication that you're working with
**ANYTHING** other than the PTR record for the (apparent) originating IP
(which, for a small business on a single static-IP connection, may or
may not even have anything to do with the business's own domain(s) at
all), and some arbitrary (sub^n)domain A record based on the PTR.

About all your scheme seems to do is identify IPs which may emit
legitimate email, generally; it's certainly nothing I'd score at
anything more than an advisory -0.001 in SA.

Consider the case of a legitimate ISP's outbound relay - most of the
mail is perfectly legitimate, but sooner or later *someone* on an IP
controlled by that ISP (and therefore allowed to relay through that
outbound relay host) will have their machine infected or someone with an
account with that ISP will have their password stolen, and then that
infected machine will spew out junk via the relay, or a machine
somewhere else will use that stolen password to send SMTP AUTH mail
through that relay....

We regularly see both of those cases here (medium-sided ISP).

The more I think about it and the more I read of what you're describing,
the more most of it seems like a reasonable component of any blacklist
operation, not a whole FUSSP[1] in its own right.

[1] http://www.claws-and-paws.com/fussp.html, among other references

-kgd

Darxus at ChaosReigns

Feb 9, 2010, 1:17 PM

Post #18 of 31 (1213 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

Charles: Thanks, I clearly need to lay out implementation sequence.

1) People who are sufficiently entertained by the subject create MTX
records for our servers, and collect blacklists of domains which
create MTX records for spamming IPs.

2) New SA capability is implemented:
A) Blacklisting by the domain of the PTR record of the sending IP (MTX_BL).
B) Verifying IPs against MTX records (MTX_PASS / MTX_FAIL).

3) Conservative people use these new tests to reduce false positives:
score MTX_BL 1
score MTX_PASS -1 # Only hit if MTX_BL wasn't.
score MTX_FAIL 0.001

People with less need to be conservative, perhaps with SA configured
to report false positives to non-forged senders, use more aggressive
scores:
score MTX_BL 2
score MTX_PASS -2
score MTX_FAIL 1

4) In order to reduce false positives, more people create MTX records.

5) Due to 4, MTX_FAIL scores can safely be increased slightly without
increasing false positives.

6) Go to 4.

One day, due to steps 4-6, enough people have created MTX records that all
email without one can reasonably be rejected by the MTA.

Are you willing to, right now, create a <IP (reversed)>.mtx.<hostname> DNS
record for all your transmitting mail servers? If not, why not?

Any thoughts on the format of that record yet?

On 02/09, Charles Gregory wrote:
> LOL.... Good luck with that. The first time that an important e-mail
> correspondent (money!) is blocked by such a 'default' setting, the
> sysadmins will be rushing to cripple this default action. You will never
> succeed in introducing ANY spam filtering system that blocks mail based
> upon an 'undecided' or 'neutral' status.

This is a good point. This is also why I suggested starting out with a
low score, for most people.

Of course I'll use something around 4 to 4.5.

>> I disagree. I can implement it now (in fact I expect to...
>
> For your own use, sure. But that's just like SPF. A bunch of people will
> use it, and a bunch, including ones that you still *really* want to
> communicate with, will NOT. Have you figured out how you are going to
> sell 'hotmail' and 'gmail' on your idea? Or are you just going to block
> all mail from them? Your choice. But if you have multiple users, well,
> you had better choose conservatively..... (grin)

Did my implementation sequence above sufficiently cover this?

Even with a MTX_FAIL score of 4.5, it still won't block the vast majority
of email I get from gmail. And I certainly wouldn't switch to blocking all
email without an MTX record without gmail adopting them. And I do think
there is a decent possibility of gmail eventually adopting MTX records due
to the sequence I described.

> And every hotmail user will be writing to the you complaining they have
> no way to talk hotmail into adopting your system, and begging you

The vast majority of legit mail will still get through even with my high
MTX_FAIL score. And if the complaints get annoying, I'll lower it. And
really, everybody that's currently getting a score of 0.5+ score from SA,
I'm happy to never hear from again.

And even if nobody gives MTX_FAIL a positive score, I still expect MTX
records to be adopted to reduce false positives, making it reasonable to
gradually use slightly higher MTX_FAIL scores.

>> I think you missed something important. Creating the records I suggest
>> can create no false positives. That point is critical to this idea.
>
> The FP's would occur on the systems *looking* for those records, and
> scoring positively in SA for simply not finding them. Your argument, and
> all of mine here, are not about the simplistic task of creating a DNS
> record, but about the battle to have the scoring/testing protocol
> implemented to make those records 'useful'.

No, actually, I was arguing for the simplicity of creating the DNS records.
I realize it'll be a while before major email providers start giving
MTX_FAIL significant positive values. The records are immediately useful
for whitelisting to reduce false positives.

--
"We will be dead soon. Is this how we want to live?"
http://www.ChaosReigns.com

Darxus at ChaosReigns

Feb 9, 2010, 1:31 PM

Post #19 of 31 (1216 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On 02/09, Kris Deugau wrote:
> I'm still not seeing the whole picture; maybe you can explain the
> difference between these two cases:
>
> 1) Legitimate sender, uses the NAT machine as the legitimate, designated

> (10.0.0.2) with a certain rDNS (exchange.smallbusiness.com).
> 2) Spam, from an infected machine on the same LAN, either via relay

> (10.0.0.2) with a certain rDNS (exchange.smallbusiness.com).

The IP is sending spam, so it gets blacklisted (by a blacklist of domains
which have MTX records for spamming IPs).

> Obviously I've missed *something* about what you've been trying to
> describe, but I haven't seen any indication that you're working with
> **ANYTHING** other than the PTR record

Yes.

> for the (apparent) originating IP

Nothing apparent about it. The delivering IP (last untrusted relay) is
the only thing in an email that can't be forged.

> (which, for a small business on a single static-IP connection, may or
> may not even have anything to do with the business's own domain(s) at
> all),

Indeed, so they would need to get whoever is delegated the authority to
provide PTR records for that IP to create the necessary records.

Two options where smallbusiness.com doesn't have the ability to define its
own PTR records. For example, the PTR record is defined by isp.com.

1) isp.com sets the PTR record to exchange.smallbusiness.isp.com, and
creates the MTX record for it (2.0.0.10.mtx.smallbusiness.isp.com
with a value of 127.0.0.1). If 10.0.0.2 sends spam, isp.com gets
blacklisted.

2) isp.com sets the PTR record to exchange.smallbusiness.com, and
smallbusiness.com creates their own MTX record
(2.0.0.10.mtx.smallbusiness.com = 127.0.0.1). If 10.0.0.2 sends spam,
smallbusiness.com gets blacklisted.

> and some arbitrary (sub^n)domain A record based on the PTR.

Yes. That's all. What format should this arbitrary A record be?

> About all your scheme seems to do is identify IPs which may emit
> legitimate email, generally; it's certainly nothing I'd score at
> anything more than an advisory -0.001 in SA.

Of course, unless you use a blacklist of domains which have MTX records for
spamming domains.

> Consider the case of a legitimate ISP's outbound relay - most of the
> mail is perfectly legitimate, but sooner or later *someone* on an IP
> controlled by that ISP (and therefore allowed to relay through that
> outbound relay host) will have their machine infected or someone with an
> account with that ISP will have their password stolen, and then that
> infected machine will spew out junk via the relay, or a machine
> somewhere else will use that stolen password to send SMTP AUTH mail
> through that relay....
>
> We regularly see both of those cases here (medium-sided ISP).

It's an issue of blacklisting. What is involved in keeping your ISP off of
IP blacklists?

> The more I think about it and the more I read of what you're describing,
> the more most of it seems like a reasonable component of any blacklist
> operation, not a whole FUSSP[1] in its own right.
>
> [1] http://www.claws-and-paws.com/fussp.html, among other references

I have been directed to that url frequently in the last few days :)

--
"For every complex problem, there is a solution that is simple, neat,
and wrong." - H. L. Mencken
http://www.ChaosReigns.com

Bowie_Bailey at BUC

Feb 9, 2010, 1:53 PM

Post #20 of 31 (1212 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

Darxus [at] ChaosReigns wrote:
> "For every complex problem, there is a solution that is simple, neat,
> and wrong." - H. L. Mencken

I think your auto-quote program is trying to tell you something... :)

--
Bowie

rwmaillists at googlemail

Feb 9, 2010, 2:29 PM

Post #21 of 31 (1212 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010 14:15:37 -0500
Darxus [at] ChaosReigns wrote:

> On 02/09, RW wrote:
> > A compromised webserver with full-circle DNS would be caught by
> > this. My point is that is the only class of spam that this could
> > help with
>
> Ah, sorry, I thought you meant mail server.
>
> Still, I don't understand why you're saying this.
>
> It would also block, for example, spam from a dynamic cablemodem IP.

Aside from a few corner cases, I don't see any advantage over checking
for full circle DNS

> And everything that it didn't block could be blocked by blacklisting
> domains which have MTX records for spamming IPs.

The same thing applies to full circle DNS

> > Every thing else is either handled by full-circle and no DNS tests,
>
> Full circle DNS tests don't block spam from quite a lot of IPs.

But how many of those are neither mail-servers, nor spammer controlled
ip ranges. My guess is that the kind of spam your scheme would
identify is mostly caught by other means.

The chief problem is that there is no way to use this scheme until it
has *very* high adoption, below 95% it wouldn't even be worth scoring
at 0.5 in Spamassassin. With SPF you at least know the difference
between a fail and a non-adopter. Whilst you could identify compliant
servers there's no way that that would warrant anthing more than a
nominal negative score. SPF_PASS scores -0.001

> > or can be easily worked around by spammers setting their own dns.
>
> Spammers can't create DNS records for hostnames for IPs they don't own
> (don't have PTR authority delegated to them for).

I was referring to IP ranges that they do control

cgregory at hwcn

Feb 9, 2010, 2:54 PM

Post #22 of 31 (1216 views)
Permalink

Re: [sa] Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010, Darxus [at] ChaosReigns wrote:
> Charles: Thanks, I clearly need to lay out implementation sequence.

Sorry to be wasting your time, but I am smart enough to have grasped it
from your previous e-mails. You just WANT your solution to be magically
adopted by everyone and you seem to be completely dismissing/ignoring the
real and practical improbability of any widespread implementation....

> Are you willing to, right now, create a <IP (reversed)>.mtx.<hostname> DNS
> record for all your transmitting mail servers? If not, why not?

Because it does not benefit me or my users. You have failed to convince me
that any significant widespread implementation will occur, and have not
demonstrated any signficant new benefits that make your idea any more
appealing than existing SPF and white/blacklists..... For me, it is
simpler to forward mail in an SPF-compatible fashion, and require SMTP
Auth through one server designated with SPF-PASS. Haven't done that yet,
for similar reasons. :)

> Any thoughts on the format of that record yet?

Take a look at the format used by SPF. You are eliminating the
cross-reference to sender address, but otherwise, all the subtle nuances
are the same. Which still leads me back to 'why should I bother to
participate in reinventing the wheel'....

Anyways, this is my fin du conversation....

- C

kdeugau at vianet

Feb 9, 2010, 3:26 PM

Post #23 of 31 (1214 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

Darxus [at] ChaosReigns wrote:
> On 02/09, Kris Deugau wrote:
>> I'm still not seeing the whole picture; maybe you can explain the
>> difference between these two cases:
>>
>> 1) Legitimate sender, uses the NAT machine as the legitimate, designated
>
>> (10.0.0.2) with a certain rDNS (exchange.smallbusiness.com).
>> 2) Spam, from an infected machine on the same LAN, either via relay
>
>> (10.0.0.2) with a certain rDNS (exchange.smallbusiness.com).
>
> The IP is sending spam, so it gets blacklisted (by a blacklist of domains
> which have MTX records for spamming IPs).

... So what incentive do I as: (pick any one or more hats below)

ISP mail/DNS admin
Colo/self-hosted domain admin
Inbound mail admin

have to either set up this extra A record, or check for it, that
existing blacklists don't provide? Chicken<->egg.

Where do you draw the lines for adding someone to a/the blacklist?
0.0001% spam? 0.01%? 1%?

How would you publish the blacklist? Globally? Rely on individual
local DNS operators running their own blacklist? (Good luck getting
*off* ten million local blacklists...)

> Two options where smallbusiness.com doesn't have the ability to define its
> own PTR records. For example, the PTR record is defined by isp.com.
>
> 1) isp.com sets the PTR record to exchange.smallbusiness.isp.com, and
> creates the MTX record for it (2.0.0.10.mtx.smallbusiness.isp.com
> with a value of 127.0.0.1). If 10.0.0.2 sends spam, isp.com gets
> blacklisted.

Immediately? Would you *really* blacklist eg AOL, Hotmail, or Gmail if
you received *one* spam from any of their networks?

If you have paying customers using your mail systems, you'd be out of
business in a matter of months.

> 2) isp.com sets the PTR record to exchange.smallbusiness.com, and
> smallbusiness.com creates their own MTX record
> (2.0.0.10.mtx.smallbusiness.com = 127.0.0.1). If 10.0.0.2 sends spam,
> smallbusiness.com gets blacklisted.
>
>> and some arbitrary (sub^n)domain A record based on the PTR.
>
> Yes. That's all. What format should this arbitrary A record be?

Aside from considerations above, what you've posted is fine,
structurally. From a pure "how to publish data" perspective, a new DNS
RR type would be slightly better, but reversed IPs concatenated to a
subdomain is a well-established way to publish this type of data.

>> About all your scheme seems to do is identify IPs which may emit
>> legitimate email, generally; it's certainly nothing I'd score at
>> anything more than an advisory -0.001 in SA.
>
> Of course, unless you use a blacklist of domains which have MTX records for
> spamming domains.

Well, aside from getting tougher on customers with infected systems, how
do you propose to actually stop the spam from being generated? If you
can't stop that, you-as-ISP *CAN NOT* fully prevent spam from being
relayed through your servers unless you unplug the network and power
cables...

>> Consider the case of a legitimate ISP's outbound relay - most of the
>> mail is perfectly legitimate, but sooner or later *someone* on an IP
>> controlled by that ISP (and therefore allowed to relay through that
>> outbound relay host) will have their machine infected or someone with an
>> account with that ISP will have their password stolen, and then that
>> infected machine will spew out junk via the relay, or a machine
>> somewhere else will use that stolen password to send SMTP AUTH mail
>> through that relay....
>>
>> We regularly see both of those cases here (medium-sided ISP).
>
> It's an issue of blacklisting. What is involved in keeping your ISP off of
> IP blacklists?

Blocking outbound connections to port 25 from residential IP blocks,
responding to reports (cutting of residential customers found to be
infected, warning and eventually cutting off static-IP business
customers; notifying users whose passwords seem to have been cracked or
stolen - among other standard measures). It helps, and we've signed up
for the feedback loop widgets with a number of places (AOL, Comcast),
but there is NO WAY we can absolutely stop all spam from emitting from
IP space under our control, short of turning off our core routers.

This is not exactly unusual.

>> The more I think about it and the more I read of what you're describing,
>> the more most of it seems like a reasonable component of any blacklist
>> operation, not a whole FUSSP[1] in its own right.
>>
>> [1] http://www.claws-and-paws.com/fussp.html, among other references
>
> I have been directed to that url frequently in the last few days :)

<g> The form may be a bit tongue-in-cheek, but taking it at face value
is helpful in seeing if you're really trying something new, or if you're
just putting a new coat of paint on something that's already in practice
as a small subset of a larger operation. (Or trying to resurrect a dead
horse that was already beaten to a thin paste ten years ago. I don't
think you've gotten *that* far yet.)

TBH, what you seem to be trying to do seems far better suited to a local
system of some kind that sees where mail is coming from, and whether
it's spam or not, and applies some degree of filter white bias to sites
generating less spam over time. (.... AWL anyone?) I'd exclude
Hotmail, Gmail, and Yahoo from it though; it's far too easy to pick up
an account, spew out a bunch of garbage and then drop the account.

It's looking like effort (admittedly, not much) for no real return,
given that *any* scheme like this needs a significant fraction of the
Internet to be using it to see it behaving properly.

-kgd

rwmaillists at googlemail

Feb 9, 2010, 4:40 PM

Post #24 of 31 (1190 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 9 Feb 2010 18:02:09 -0500
Darxus [at] ChaosReigns wrote:

> On 02/09, RW wrote:
> > > And everything that it didn't block could be blocked by
> > > blacklisting domains which have MTX records for spamming IPs.
> >
> > The same thing applies to full circle DNS
>
> So you want to blacklist all domains that have *any* PTR records for
> IPs that send spam?

Perhaps you should google full-circle dns if you don't understand what
it means. Essentially there's little difference, unless you are happy
to ignore spammers that get blacklisted and then remove the MTX
record. If you are, then there's little point to listing them in the
first place.

> > > > Every thing else is either handled by full-circle and no DNS
> > > > tests,
> > >
> > > Full circle DNS tests don't block spam from quite a lot of IPs.
> >
> > But how many of those are neither mail-servers, nor spammer
> > controlled ip ranges.
>
> A lot? Spam bot nets.

Spambots don't have full-circle DNS.

> > The chief problem is that there is no way to use this scheme until
> > it has *very* high adoption, below 95% it wouldn't even be worth
> > scoring
>
> As I recently posted, it can be used for whitelisting (after
> blacklisting) immediately. And that use could increase creation of
> the relevant records gradually until they're wide spread.

How could your scheme be used for whitelisting when a huge amount of
spam comes through mail-servers that send a mixture of spam and ham?
Would you really want to whitelist hotmail? This kind of argument was
made about SPF - a pass still scores -0.001.

SPF has a limited amount of enlightened self-interest about it. If you
publish SPF records you get less backscatter - your scheme doesn't do
that. There's nothing in your scheme to encourage anyone to use it.

> > at 0.5 in Spamassassin. With SPF you at least know the difference
> > between a fail and a non-adopter. Whilst you could identify
> > compliant servers there's no way that that would warrant anthing
> > more than a nominal negative score. SPF_PASS scores -0.001
>
> Only with a domain blacklist.

So why would anyone use it and risk being blacklisted, what's the point
of it existing if you don't punish domains that don't join - any that
get on the blacklist could just drop-out of the scheme.

ram at netcore

Feb 9, 2010, 10:15 PM

Post #25 of 31 (1175 views)
Permalink

Re: Spam filtering similar to SPF, less breakage [In reply to]

On Tue, 2010-02-09 at 11:42 -0500, Darxus [at] ChaosReigns wrote:
> I apparently need to clarify that I think this is a good idea because I am
> concerned about the number of people (who control DNS records) who are very
> strongly against creating SPF records specifically because of forwarding
> breakage. The email I got in response to my request for my employer to
> create an SPF record included the word "abomination". From a friend.
> I don't entirely agree, but it is a problem for adoption.
>
> This is entirely an attempt to replicate the functionality of SPF without
> breaking forwarding, and without causing other problems that might
> discourage adoption.
>
>
How does this idea authenticate mail from domain ? SPF is aimed at doing
that.

IMHO the SPF-breaks-forwarding argument is misplaced
What about SRS. If SRS implementation is not going to be easy because
mailservers have been there too long for adopting anything new then can
your be sure MailServer IP validation will be adopted ?

Anyway I block spams from almost all non-mailservers by using RBL's
I dont see any value add in implementing this

Thanks
Ram

> I set this up for my mail server (using mtx instead of designatedsender):
>
> $ host -t PTR 64.71.152.40
> 40.152.71.64.in-addr.arpa domain name pointer panic.chaosreigns.com.
>
> $ host -t A 40.152.71.64.mtx.panic.chaosreigns.com
> 40.152.71.64.mtx.panic.chaosreigns.com has address 127.0.0.1
>
> All it took was creating a single record in bind:
>
> 40.152.71.64.mtx.panic.chaosreigns.com. IN A 127.0.0.1
>
>
> I'll define it slightly differently:
> 127.0.0.1 is a pass (negative SA score).
> not found is a fail (positive SA score).
> 127.0.0.0 is a fail (positive SA score).
> Anything else is undefined (0 SA score) for future options.
>
>
> I'd still appreciate feedback on the format of the A record.
>
>
> On 02/09, RW wrote:
> > You've mixed-up A record and PTR record.
>
> Yes. Embarrassing.
>
> > Checking for full-circle DNS already does most of this.
>
> My home dynamic cablemodem address passes full-circle DNS. But not this.
> So this is far more useful for checking if an IP is a legitimate sending
> mail server.
>
> > What your
> > scheme would do is check for otherwise legitimate servers that have
> > been compromised and are delivering direct-to-mx.
>
> An otherwise legitimate but compromised mail server would not be detected
> by this. I'm curious why you interpreted it differently.
>
>
> On 02/09, Charles Gregory wrote:
> > On Mon, 2010-02-08 at 22:08 -0500, Darxus [at] ChaosReigns wrote:
>
> > What you describe here is functionally similar to an SPF lookup with a
> > 'pass' result..... The server provides positive verification that the
> > listed IP is a legitimate sender for that domain.
>
> Yes.
>
> > As long as 'otherwise' is a definitive 'fail' response from an SPF (or
> > equivalent) server, and not merely an absence of SPF server....
>
> Yes. Definitive "fail".
>
> > Your method would allow 'spoofing' so that a spammer who hacks a
> > legitimate server can use a valid return address on a different domain,
> > but still the mail would receive a 'passing' grade. At least, with SPF,
> > the spammer must forge an address on the hacked domain, which increases
> > the likelihood of detection....
>
> Yes. I would blacklist domains that "pass" hacked servers. Just as IPs of
> hacked servers are blacklisted. They're sending spam, and need to be
> fixed.
>
> >> Forwarding doesn't break.
> >
> > Ah, so you want to allow 'legitimate' forwarding, but not allow spammers
> > to 'forward' their mail? Good luck with that. The only way to make it
> > work for the legitimate sender, but not for spammers is to have a
> > mechanism built-in to the forwarding server that encapsulates or rewrites
> > the envelope 'From' address.....
>
> Encapsulating or rewriting the envelope 'From' address seems significantly
> less likely to be adopted from what I've read.
>
> >> Obviously you'd need a blacklist of spammer domains that list spamming
> >> IPs as legit senders.
> >
> > And you would be playing the same 'musical chairs' game with new domains
> > created by spammers on a daily basis. All the same flaws of SPF, and no
> > greater benefit.
>
> Same domain blacklisting issues as SPF, yes.
>
> I am not very concerned about the throw-away domains because
> I'll reject all mail from domains not at least 10 days old. 10+
> day old domains are already listed as 127.0.2.3 records from
> example.com.hostkarma.junkemailfilter.com.
>
> I believe the benefit of not breaking forwarding is sufficient to make it
> much more useful than SPF for spam filtering. I've come across enough
> people, personally, recently, in trying to block (some = positive SA
> score) emails without an SPF "pass", who are not willing to ever implement
> SPF due to breaking forwarding that I believe this would be useful.
>
> >> Is there any way this wouldn't be very useful?
> >
> > Is there any place where SPF does not do the same job, other than mail
> > forwarding?
>
> No. But as I said, I am concerned about the potential for wide spread
> adoption of SPF due to forwarding breakage enough that I think this would
> be much more useful.
>
>
> On 02/09, Matus UHLAR - fantomas wrote:
> > On 08.02.10 22:08, Darxus [at] ChaosReigns wrote:
> > > So it's not tied to the SMTP MAIL FROM or anything.
> > > Forwarding doesn't break.
> >
> > What do you mean by this?
> > Do you want to implement new way of defining which hosts are permitted to
> > send e-mail?
>
> Yes.
>
> > There already are attempts to do this, with false positives and
> > negatives.
>
> How is this not better than the other attempts?
>
> > Yours is a bit complicated
>
> How is it complicated? You need to create 1 record per sending mail
> server, in the form:
>
> <IP of mail server>.mtx.<hostname of mail server>
>
> (And the IP needs to be reversed as in all other A records that list IPs.)
>
> > and new which means everyone would
> > need to implement this (otherwise he'd get false positives on his outgoing
> > mail).
>
> Yes.
>
> > Therefore I think it won't work this way.
>
> This is why I said SA scores for a fail should be small at first.
>
>
> Do you see any reason this wouldn't be more quickly adopted than SPF?
>

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads