Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SPF: Discuss

The .forward problem

 

 

SPF discuss RSS feed   Index | Next | Previous | View Threaded


mengwong at dumbo

Oct 7, 2003, 10:10 AM

Post #1 of 6 (347 views)
Permalink
The .forward problem

On Tue, Oct 07, 2003 at 01:09:09PM +0200, Justo Alonso wrote:
|
| Meng, please... can you write it ?!?!
|

The new draft is being written in committee fashion together with the
authors of RMX/DMP/DRIP/etc, which is why it's taking so long.

Right now I'm worrying over the .forward problem. I want to be sure it
can be reasonably solved; if it can't be solved, many people are going
to complain, and SPF will be mired.

So far I see three classes of solutions.

Scenario:

Phase 1: alpha [at] yahoo sends mail to beta [at] pobox
Phase 2: beta [at] pobox forwards to gamma [at] hp

Problem:

Phase 1: Pobox tests SPF. Result: pass.
Phase 2: HP tests SPF. Result: fail.

Solutions:

1) SRS: change the envelope sender.
2) per-recipient custom whitelisting.
3) change SMTP, introducing a COOKIE as Saez has suggested.

* * *

Solution 1: SRS

Code:
use Mail::SRS;
Mail::SRS::set_secret("my secret");
my $sender = q"alpha [at] yahoo";
my $alias = q"beta [at] pobox";
my @rcpts = [q"gamma [at] yahoo"];
my $newsndr = Mail::SRS::forward(sender => $sender, alias => $alias, rcpts => \@rcpts);
print $newsndr;

Result:
bounce-alpha#yahoo.com-laNsR7A9Bb*sU@Bڛ  Ȉ

This is not vaporware: Mail::SRS has been written and runs.

Solution 2: per-recipient custom whitelisting.

This solution was suggested by Ted Cabeen.

We can assume gamma [at] hp knows that beta [at] pobox is forwarding to
him.

If he wants to get beta [at] pobox mail, he can tell HP's MTA "if you
get mail for me, if it comes from pobox.com's servers, let it
through." So hp.com's MTA would have to do two sets of SPF checks:
one on the envelope sender domain, and another on the domains
specified by the user. In this case, gamma [at] hp would tell HP.com's
MTA to also SPF-allow pobox.com.

This assumes pobox.com publishes SPF. If pobox.com does not publish
SPF, HP's MTA would return "unknown" and it would have to accept the
mail.

Solution 3: add some sort of call-back-to-sender cookie to the MAIL FROM command.

This solution was suggested by David Saez.
http://archives.listbox.com/spf-discuss [at] v2/200310/0049.html

It reflects ideas suggested by mjd
http://archives.listbox.com/spf-discuss [at] v2/200309/0017.html

And can be seen as an upside-down version of djb's IM2000.

This seems like a good idea but of course requires patching MTAs, and
at first glance relies on end-to-end support.

* * *

Evaluation of Solutions

1) pobox.com's MTA has to change: it has to do SRS.
2) hp.com's MTA has to change: per-user whitelisting is rare and
cutting-edge. To my knowledge only Colander supports it.
3) yahoo, pobox, and hp all have to change: cookies would be new to SMTP.

Possibility of Abuse

1) if a spammer gets hold of an SRS-encoded path, it can forge bounce
messages with MAIL FROM:<> that would go back to the original sender.

spammers can do this in a number of ways: SRS-encoded return paths
could be exposed on web archives. Or a spammer could sign up for a
pobox account, forge a message from alpha [at] yahoo, receive it, and
then spam via pobox.

To solve this, SRS cookies --- the laNsR7A9Bb*sU part of the new
sender --- can be expired over time (eg. after one week) or by the
number of times they've been used.

Expiring SRS cookies is fine if the rewritten sender addresses are
only used as a return path. However, some overzealous mailing list
software use envelope senders as subscription addresses.

2) less open to abuse.

3) less open to abuse.

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname@#Mo\HU;֤͵


matthew at syrah

Oct 7, 2003, 11:06 AM

Post #2 of 6 (339 views)
Permalink
Re: The .forward problem [In reply to]

On Tue, Oct 07, 2003 at 01:10:26PM -0400, Meng Weng Wong wrote:

> Right now I'm worrying over the .forward problem.

> 1) SRS: change the envelope sender.
> 2) per-recipient custom whitelisting.
> 3) change SMTP, introducing a COOKIE as Saez has suggested.

I'm in favor of per-recipient custom whitelisting. (I need to
think more about 1 and 3, as they seem complicated.) It looks like
the solutions are not mutually exclusive, so more than one could be
used.

I might have some further improvements on the idea of
per-recipient custom whitelisting:

> Solution 2: per-recipient custom whitelisting.

> This assumes pobox.com publishes SPF. If pobox.com does not
> publish SPF, HP's MTA would return "unknown" and it would have to
> accept the mail.

If pobox.com does not publish SPF, couldn't HP's MTA fall back
on pobox.com's MX records? For example: HP's MTA knows that
gamma [at] hp receives forwarding from pobox.com, HP's MTA looks up
pobox.com's SPF records (which do not exists), so HP's MTA looks up
pobox.com's MX records and uses them as a standin for the SPF records.
Hopefully, the pobox.com MTA has an MX record. In the case of
.forward files, I think it usually will. This isn't perfect. But it
might help in many cases.
But I may be wrong about this: how many MTA's have multiple IP
addresses and use non MXed IP addresses to send outgoing mail? I do
not know how common this is.

> Evaluation of Solutions
>
> 1) pobox.com's MTA has to change: it has to do SRS.

> 2) hp.com's MTA has to change: per-user whitelisting is rare and
> cutting-edge. To my knowledge only Colander supports it.

I believe per-user whitelisting can be done using courier-mta
and maildrop. (Or courier-mta + any appropriate filter, but maildrop
sprang to mind.)

But does the whitelisting really need to be done "on the wire"
during the SMTP transaction? Couldn't it be done later inside some
filter? The SPF layer could then just add headers that the filter
could look at to evaluate the credibitily of the message.

-Matthew.
______________________________________________________________________
matthew [at] syrah

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname@#Mo\HU;֤͵


secabeen at pobox

Oct 7, 2003, 12:23 PM

Post #3 of 6 (339 views)
Permalink
Re: The .forward problem [In reply to]

Meng Weng Wong <mengwong [at] dumbo> writes:
> Solution 2: per-recipient custom whitelisting.
>
> This solution was suggested by Ted Cabeen.
>
> We can assume gamma [at] hp knows that beta [at] pobox is forwarding to
> him.
>
> If he wants to get beta [at] pobox mail, he can tell HP's MTA "if you
> get mail for me, if it comes from pobox.com's servers, let it
> through." So hp.com's MTA would have to do two sets of SPF checks:
> one on the envelope sender domain, and another on the domains
> specified by the user. In this case, gamma [at] hp would tell HP.com's
> MTA to also SPF-allow pobox.com.
>
> This assumes pobox.com publishes SPF. If pobox.com does not publish
> SPF, HP's MTA would return "unknown" and it would have to accept the
> mail.

We don't necessarily need per-user whitelisting here. We can do it on
a larger scale, but there would need to be some accountability for it.
For example, the hp.com admin could have a whitelist that lists the
entire pobox.com domain as okay for forwarding, since pobox.com is a
trustworthy source. Theoretically, hp could do whitelisting for all
of the forwarding services that their users use. However, in such a
system, we'd probably want to also create DNS-based whitelists that
list all reputable mailing-list and forwarding services that use SPF
to reduce the workload on the SPF-enabled server admins. Perhaps in
order to get on such a list, the forwarder owner would have to
register directly with the whitelist so that email abuse through the
forwarding system could be tracked. This sort of thing would also fix
the mailing list problem, as they could register with the whitelists
as well.

--
Ted Cabeen http://www.pobox.com/~secabeen ted [at] impulse
Check Website or Keyserver for PGP/GPG Key BA0349D2 secabeen [at] pobox
"I have taken all knowledge to be my province." -F. Bacon secabeen [at] cabeen
"Human kind cannot bear very much reality."-T.S.Eliot cabeen [at] netcom

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname@#Mo\HU;֤͵


nwp at nz

Oct 9, 2003, 2:23 AM

Post #4 of 6 (338 views)
Permalink
Re: The .forward problem [In reply to]

On Tue, Oct 07, 2003 at 12:23:47PM -0700, Ted Cabeen wrote:
> Meng Weng Wong <mengwong [at] dumbo> writes:
> > Solution 2: per-recipient custom whitelisting.

I think that rather than just trying to come up with "something that might
work", it is a good idea to consider what is happening at a slightly more
abstract level (and so ensuring that we have a model that is consistent).
I don't want to see this turning into an ugly great hack.

So...

When a mail is forwarded, what is happening? Is the forwarded mail from the
original sender? Is it a new mail from the forwarder which happens to be
pretty much identical to the original? Is it always the same, or does it
vary?

It seems to me that there are three distinct possible situations -- where
the relay or forward is imposed by the originator, where it is imposed by
the recipient, and (most awkwardly) where it is imposed by a third party
(e.g. there is no direct connectivity between sender and recipient and a
gateway must be used).

If the originator is responsible for the relay then they should also be
responsible for ensuring that the sender data is valid when it leaves the
relay. Example: I send mail from home via a dialup to random_isp, and
the mail is relayed by my smarthost at work. In this situation I must ensure
that either the smarthost is listed in SPF as an emitter for my personal
domain, or that the smarthost rewrites the envelope sender to be from another
domain for which it is listed.

If the recipient is responsible for the relay then they should also be
responsible for ensuring the safe delivery from the point where the relay
receives it (and verifies the SPF). Example: I have a .debian.org address
which forwards to my personal address. I must either rewrite the sender so
that my MX will accept it (perhaps I have no control over the MX), or I
must configure the MX to trust debian's servers to send on some category
of mail.

The third situation is the most awkward.

Once upon a time it was common for messages to be relayed by random third
parties -- to mail from JANET to the NSFNet I would have had to mail
to "someone%nsf.net.address [at] uk". In that situation, it would
be impossible for the final receiving server to verify that the sender was
who they claimed to be, unless nsfnet-relay gave them some indication that
it was sure who the sender was, and they also trusted nsfnet-relay.

Pretty awkward. And pretty much the same situation we have now, although now
it is less common.

If you are sure that the relay knows and is honest about the original sender,
then you can trust all sender data that that relay sends. This may be
possible, but there may be an awful lot of relays out there that you don't
yet know about. If it turns out not to be possible, then you require some
form of communication with the originating server, some of which will have
to be outside the message in question.

We have to add some kind of authentication information to the message itself,
which can be verified against some information we got from somewhere else.
The simplest example of this would be to GPG-sign the message individually, but
there are all sorts of possibilities involving the emitting server (listed
in SPF as a valid sender for the alleged sender domain) either using a
shared secret or some form of PKI to sign the message. So the shared secret
or public keys would have to be distributed somehow.

Whether this is done with cookies of some kind, or by including server public
keys in SPF data, *shrug*...


I think it boils down to a problem that has been sitting in the "too hard"
basket for a very long time. It is going to be inconvenient no matter what,
but it has to be solved (being unable to verify senders is even more
inconvenient).

We can get away with the first two cases a lot of the time (I think), but the
third one is almost certain to crop up, and if it can be solved then it will
also conveniently cover the first two as well (and avoid the need for all
kinds of inconvenient responsibility-taking by lots of lusers).

> For example, the hp.com admin could have a whitelist that lists the
> entire pobox.com domain as okay for forwarding, since pobox.com is a
> trustworthy source. Theoretically, hp could do whitelisting for all
> of the forwarding services that their users use.

This is all well and good for "big boys", but does not scale. If we can, we
should solve the hard problem.

> However, in such a
> system, we'd probably want to also create DNS-based whitelists that
> list all reputable mailing-list and forwarding services that use SPF
> to reduce the workload on the SPF-enabled server admins. Perhaps in
> order to get on such a list, the forwarder owner would have to
> register directly with the whitelist so that email abuse through the
> forwarding system could be tracked. This sort of thing would also fix
> the mailing list problem, as they could register with the whitelists
> as well.

Whitelisting by IP? Not so great for people with dynamic IPs. How would you
run registration? How would you know that the person registering the server
was actually the server owner/operator?


If we do solve the hard problem, is SPF as currently conceived still useful?
A little? A lot? I'm not sure.

If not, then we'd do best to either solve the hard problem damn quick, or
just acknowledge that SPF can't be perfect and be rather more ruthless about
chucking out objections that can't be overcome (yet).



Cheers,


Nick

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname@#Mo\HU;֤͵


wayne at midwestcs

Oct 10, 2003, 12:49 PM

Post #5 of 6 (338 views)
Permalink
Re: The .forward problem [In reply to]

In <20031007171026.GI2345 [at] dumbo> Meng Weng Wong <mengwong [at] dumbo> writes:

> 1) SRS: change the envelope sender.
> 2) per-recipient custom whitelisting.
> 3) change SMTP, introducing a COOKIE as Saez has suggested.

I've been thinking about this for a while.

Unless I'm missing something, these solutions are not mutualy
exlusive.

I think it is inevitable that those domains that use SPF (or similar
systems) are going to be the ones that will have to implement a
solution.

So, in the case of alpha [at] yahoo -> beta [at] pobox ->
gamma [at] hp, we have:

If hp.com wants to check SPF systems *and* is willing to accept email
that has been forwarded, it *MUST* do some sort of whitelisting.
pobox.com, after all, may not want to do anything. This same thing is
going to turn up for many mailing lists that do not put their own
domain in the envelope-from, but instead put the email address of the
person posting to the mailing list.

I don't see much way around this.


If pobox.com wants to allow forwarding of email through their system
or run a mailing list, they *SHOULD* implment SRS, whether or not they
implement SPF.


-wayne

-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname@#Mo\HU;֤͵


wayne at midwestcs

Oct 10, 2003, 7:23 PM

Post #6 of 6 (339 views)
Permalink
Re: The .forward problem [In reply to]

In <x4r81kho0j.fsf [at] footbone> wayne <wayne [at] midwestcs> writes:

> In <20031007171026.GI2345 [at] dumbo> Meng Weng Wong <mengwong [at] dumbo> writes:
>
>> 1) SRS: change the envelope sender.
>> 2) per-recipient custom whitelisting.
>> 3) change SMTP, introducing a COOKIE as Saez has suggested.
>
> I think it is inevitable that those domains that use SPF (or similar
> systems) are going to be the ones that will have to implement a
> solution.
>
> So, in the case of alpha [at] yahoo -> beta [at] pobox ->
> gamma [at] hp, we have:
>
> [ ... ]
>
> If pobox.com wants to allow forwarding of email through their system
> or run a mailing list, they *SHOULD* implment SRS, whether or not they
> implement SPF.

I need to add to this.

If pobox.com uses SPF and it allows forwarding email through their
system, then it *MUST* also use SRS. This is so that the next hop
can correctly verify the envelope-from.


So, systems that use SPF, *MUST* do both SRS and whitelisting for
those systems that don't do SRS or have "bad" mailing list software.

Systems that do fowarding *SHOULD* do SRS, but are not required to.

Similarly, systems that run mailing lists *SHOULD* put their email
address in the envelope-from rather than use the email address of the
poster. Any mailing list software that doesn't do this already will
cause bounces to be sent to the poster, which is really annoying.


-wayne


-------
Sender Permitted From: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname@#Mo\HU;֤͵

SPF discuss RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.