Mailing List Archive: SPF: Discuss

C/R Pros and Cons

Index | Next | Previous | View Threaded

dmquigg-spf at yahoo

Oct 14, 2008, 1:35 PM

Post #1 of 15 (2539 views)
Permalink

C/R Pros and Cons

At 10:39 AM 10/14/2008 -0700, Michael Deutschmann wrote:

>On Mon, 13 Oct 2008, David MacQuigg wrote:
>> I didn't see any response to my suggestion of sending a challenge to
>> the author via SMTP reject. That still seem like the better alternative,
>> and one that doesn't have the "weak antibiotic" problem.
>
>Uh, of all the anti-spam methods, C/R is probably the poster child for
>the antibiotic problem.

We need to define what we mean by "weak antibiotic". I would say that an anti-spam tactic has this problem when it is so weak that it encourages the development of better spamware to nullify the tactic. Is it likely that spamware will attempt to nullify C/R systems?

>When a single person uses a homebuilt C/R, it's extremely effective, with
>the only false negatives being when the backscattering variant is used and
>angry backscatter victims deliberately confirm forged e-mail to punish the
>C/R user.

I would not call a C/R system with backscatter a "variant". A better word would be "mistake", or "bad design".

>But if it's widely deployed, many people will be using identical C/R
>software, and spamware will be extended to handshake with it. Captcha is
>not an impregnable obstacle.

There are many ways to implement a challenge. Even with CAPTCHA, you can dial up the difficulty enough to make it unprofitable for spammers, but not too difficult for individual authors with important messages. I would sure like to see some data on this, but my guess is that even 30 seconds of effort to find a webpage would be enough. Add a paragraph of instructions that change from day to day, and spammers will not be able to automate a response.

The important point is the variability of the challenge, even if every Receiver in the world is using identical C/R software. Spamware developers will not even try, if they know that it is easy for Receivers to change the challenge. Think about how you would write a challenge with endless variations only a human would understand. It's not difficult.

>We just haven't seen this because most people competent enough to
>implement C/R reject it out of hand due to the backscatter problem, which
>has clamped usage to the point that spammers haven't had to adapt.

I have seen discussions like this go on for days, with the anti-C/R folks assuming that all C/R systems have backscatter, and not even acknowledging statements to the contrary.

>C/R in the 550 message, as you propose, avoids the backscatter problem

I'm glad to see you are not one of those folks.

>but not the antibiotic problem. It also has the issue that some senders
>may not ever see the 550 text.

Let's drop the word "sender", since we now need a little more precision. If the Transmitter does not handle the text of the SMTP reject message properly, and by that I would assume simply relay it to the original Author of the message, if that is the problem, it is not the Receiver's responsibility. The Receiver has done his job by sending a clearly-worded reject message.

If both the Receiver and the Transmitter do their job, and the problem is that the Author does not know what to do, he needs to ask his mail-system admin. Receivers cannot be held responsible for these kind of problems on the sending side.

Has anyone read Erickson's paper on whitelisting? http://www.ceas.cc/ Would you agree that this is a good use of C/R?

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

sandy at cypressintegrated

Oct 14, 2008, 2:20 PM

Post #2 of 15 (2461 views)
Permalink

Re: C/R Pros and Cons [In reply to]

> Let's drop the word "sender", since we now need a little more
> precision. If the Transmitter does not handle the text of the SMTP
> reject message properly, and by that I would assume simply relay it
> to the original Author of the message, if that is the problem, it is
> not the Receiver's responsibility. The Receiver has done his job by
> sending a clearly-worded reject message.

Your claim that a "clearly-worded" (512-character, given non-global
support for multi-line responses) response message is your only
responsibility is ridiculous. Though classic FUSSP.

Try doing all your tech support in 512-character phrases from now on.
Guess what: it's not like the developers of the original C/R concept
had never heard of the SMTP protocol. They just knew their idea would
be _even lamer_ if they tried to express it using such an, ahem,
concise end user interface.

> Receivers cannot be held responsible for these kind of problems on
> the sending side.

The age-old "problem" of not being able to send mail because the
receiver set a policy that stopped being feasible in around 1983, or
whenever the first non-techie end user used SMTP.

--Sandy

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

dmquigg-spf at yahoo

Oct 14, 2008, 3:36 PM

Post #3 of 15 (2454 views)
Permalink

Re: C/R Pros and Cons [In reply to]

At 05:20 PM 10/14/2008 -0400, Sanford Whiteman wrote:

>> Let's drop the word "sender", since we now need a little more
>> precision. If the Transmitter does not handle the text of the SMTP
>> reject message properly, and by that I would assume simply relay it
>> to the original Author of the message, if that is the problem, it is
>> not the Receiver's responsibility. The Receiver has done his job by
>> sending a clearly-worded reject message.
>
>Your claim that a "clearly-worded" (512-character, given non-global
>support for multi-line responses) response message is your only
>responsibility is ridiculous. Though classic FUSSP.
>
>Try doing all your tech support in 512-character phrases from now on.
>Guess what: it's not like the developers of the original C/R concept
>had never heard of the SMTP protocol. They just knew their idea would
>be _even lamer_ if they tried to express it using such an, ahem,
>concise end user interface.

Here are some sample reject messages that I would consider clearly worded. Notice we are not trying to do tech support in the message itself.

Sorry! We cannot guarantee delivery of this message.
- <your domain> does not offer sufficient authentication to prevent forgery.
- <your domain> does not have sufficient reputation for <recipient>.
We will run it through our spam filter, and keep it in our quarantine, but the recipient may not read it.
See http://open-mail.org/rejects for help.

>> Receivers cannot be held responsible for these kind of problems on
>> the sending side.
>
>The age-old "problem" of not being able to send mail because the
>receiver set a policy that stopped being feasible in around 1983, or
>whenever the first non-techie end user used SMTP.

There seems to be a lot of unstated assumptions here. I'm not quite sure what is behind this statement, so I can't offer a complete response. I'll just say that I believe an intelligent reject policy can motivate senders to fix a problem without adding to problems on the receiver side.

Maybe a little less sarcasm would help. I am interested in what you have to say.

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

stuart at bmsi

Oct 14, 2008, 5:15 PM

Post #4 of 15 (2454 views)
Permalink

Re: C/R Pros and Cons [In reply to]

On Tue, 14 Oct 2008, Sanford Whiteman wrote:

> Try doing all your tech support in 512-character phrases from now on.

Been there. Done that. Lots of ISPs like AOL do it. Just include an URL
on the first line. The web page (customized by a transaction id
parameter) has lots of information. If the MTA reports the reject in a DSN to
the sender, it is even clickable in most email clients.

> Guess what: it's not like the developers of the original C/R concept
> had never heard of the SMTP protocol. They just knew their idea would
> be _even lamer_ if they tried to express it using such an, ahem,
> concise end user interface.

I have no problem with C/R - as long the C/R system checks SPF and maybe
DKIM before responding - or at least send the C/R as a real DSN (null
MAIL FROM) so I can reject it via SRS or other VERP. As for those that don't -
I hadn't thought of confirming the spammers as revenge. I guess I'm not evil
enough. Tempting though ....

> The age-old "problem" of not being able to send mail because the
> receiver set a policy that stopped being feasible in around 1983, or
> whenever the first non-techie end user used SMTP.

Since I pay to receive the mail, and it's my domain, I'll reject anyone I like,
thank you very much. You are thinking in terms of a large free email provider
like gmail. Spam may eventually drive large email providers out of business.

--
Stuart D. Gathman <stuart [at] bmsi>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

sandy at cypressintegrated

Oct 14, 2008, 7:14 PM

Post #5 of 15 (2445 views)
Permalink

Re[2]: C/R Pros and Cons [In reply to]

> Been there. Done that. Lots of ISPs like AOL do it. Just include an
> URL on the first line.

I question anyone's desire to join the ranks of AOL's mail team --
famously non-responsive and frustrating to real mail admins and
useless to end users experiencing protocol-level problems.

Anyway, you know we don't control the "first line" of a DSN. Are you
only considering the fantasy DSN that will exist after we overhaul the
SMTP protocol to put calmer language first? No MTA will ever print
your "No, no, we still want your mail message if you go to our website
first" message BEFORE they print "Permanent failure. Your message
could not be delivered." That is what a 5xx code means, and every
2821-compliant MTA is completely correct in being as firm as they want
in stating the message could not, would not, be accepted. You can't
arbitrarily decide that some 5xx codes are merely advisory. No MTA
vendor will, or should, listen.

Maybe if you wrote an SMTP extension RFC and tried to go through the
actual process. But you can see how easy that was for SPF. And all for
the hacked-together concept of HTTP-before-SMTP? Please.

> The web page (customized by a transaction id parameter) has lots of
> information. If the MTA reports the reject in a DSN to the sender,
> it is even clickable in most email clients.

As you know, real-world experience proves over and over again that
users will not read and follow "lots of information" in a DSN. We are
talking about humans who have plenty of other things to do -- *zero*
of their responsibilities are technical, and they are not your
children or parents. Experienced admins know that users will call the
help desk when they get a DSN that clearly states "The recipient's
mailbox is full": "Is this on our end or theirs?" they'll ask. When a
DSN says "The recipient domain misspelled.com does not exist", they
call and say "Our website is down."

And the protocol-level C/R concept suggests adding to this support
burden an additional PERMANENT FAILURE that, well, isn't reeeeeeally a
failure. "Permanent failure. No, temporary failure. No, um, permanent
for now, but not the next one. Well, unless your company has outbound
gateways on multiple networks and the next one comes from somewhere
else." Epic fail.

> Since I pay to receive the mail, and it's my domain, I'll reject
> anyone I like, thank you very much. You are thinking in terms of a
> large free email provider like gmail.

Look, the tough-guy domain admin thing is itself a marker of FUSSP and
pretty boring. Nobody cares what you do with your personal domain. I
refer to any messaging system whose users have the ability to get you
removed from your job. There are outliers, of course, companies whose
users inexplicably kowtow to the lab rat attitudes and condescension
of their IT staff, and these places may be happy to implement ideas
like 5xx C/R. Otherwise, it is the stuff of home networks and
family-name domains.

> Spam may eventually drive large email providers out of business.

I am sure people are not champing at the bit for a hobbyist hosting
provider using draconian C/R. I certainly do not think that Earthlink,
C/R "pioneers", have nabbed a net increase from GMail or Yahoo lately.

--Sandy

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

stuart at bmsi

Oct 14, 2008, 8:03 PM

Post #6 of 15 (2440 views)
Permalink

Re[2]: C/R Pros and Cons [In reply to]

On Tue, 14 Oct 2008, Sanford Whiteman wrote:

> in stating the message could not, would not, be accepted. You can't
> arbitrarily decide that some 5xx codes are merely advisory. No MTA
> vendor will, or should, listen.

I don't need to. Some 5xx codes, like 551, *are* already advisory in
rfc2821. It just means "can't deliver it that way, but try this other
way". 551 has been proposed as an alternative to SRS.

> Maybe if you wrote an SMTP extension RFC and tried to go through the
> actual process. But you can see how easy that was for SPF. And all for
> the hacked-together concept of HTTP-before-SMTP? Please.

Hey, I don't use C/R - and I have never responded to C/R (email wasn't
important enough to take the trouble). I'm just saying I don't mind
it as long as there is no backscatter. And a 5xx code is a reasonable
way to say "can't deliver it that way - but do this instead".

> Look, the tough-guy domain admin thing is itself a marker of FUSSP and
> pretty boring. Nobody cares what you do with your personal domain. I
> refer to any messaging system whose users have the ability to get you
> removed from your job. There are outliers, of course, companies whose
> users inexplicably kowtow to the lab rat attitudes and condescension
> of their IT staff, and these places may be happy to implement ideas
> like 5xx C/R. Otherwise, it is the stuff of home networks and
> family-name domains.

The domains I manage are for real customers who have to receive timely
emails from partners who all too often are completely clueless about
maintaining their email systems properly (as in rfc compliant). That is
why I don't use C/R. But it is a legitimate option (minus the backscatter)
for those who don't want email unless the sender cares enough to jump
through the hoop. If I was Donald Knuth, I'd use C/R.

> > Spam may eventually drive large email providers out of business.
>
> I am sure people are not champing at the bit for a hobbyist hosting
> provider using draconian C/R. I certainly do not think that Earthlink,
> C/R "pioneers", have nabbed a net increase from GMail or Yahoo lately.

Look, I don't use C/R. But I *do* do a much better job of spam abatement
with my "hobbiest" software than the big email providers. The key
is customizability - which requires company (or family) specific domains.
A large part of the success is "draconian" rules (like requiring a valid
HELO, a valid PTR, or an SPF Pass or Neutral), which are relaxed and
customized for specific business partners with braindead email. The
system learns about business partners based on who the local users send
email to - plus I can add manual rules for braindead mailing lists.

--
Stuart D. Gathman <stuart [at] bmsi>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

sandy at cypressintegrated

Oct 14, 2008, 9:47 PM

Post #7 of 15 (2464 views)
Permalink

Re[3]: C/R Pros and Cons [In reply to]

> I don't need to. Some 5xx codes, like 551, *are* already advisory in
> rfc2821. It just means "can't deliver it that way, but try this other
> way". 551 has been proposed as an alternative to SRS.

Nowhere in 2821 is 551 treated as anything but a permanent error. It
continues to be a "Permanent Negative Completion reply" in the 5yz
range. 551 is noted as more correctable -- *via a new message
envelope* -- than other permanent errors. By definition, any such
"correction" cannot occur in real-time: even if the error might not
exist minutes later, the DSN resulting from a 551 _must never
represent that the submitted message could be delivered_. Therefore,
it will always appear to the user as a permanent failure alongside
everything else they call the help desk about.

Unless the majority of MTAa specifically change the format of their
DSNs for 551s (or for some reserved enhanced status code) to hide the
"permanent" part and show a happy, clickable URL instead, the idea of
using any 5xx code for C/R is dead in the water. IMO, one would be
better off lobbying for a new 4xx code that instructs "keep retrying
with this exact message object, but *also* send a transient DSN with
the following happy text" -- which covers more bases. But I wouldn't
spend my time lobbying for either one.

> Look, I don't use C/R. But I *do* do a much better job of spam
> abatement with my "hobbiest" software than the big email providers.

I believe that. I have many colleagues that do boutique anti-abuse
hosting that do a better job than the big guys as well. But none of
them, including you, use C/R to do it. And you wouldn't be any more
likely to be able to get away with C/R for your customer base if you
used this phantasmic envelope-time rejection idea.

--Sandy

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

dmquigg-spf at yahoo

Oct 15, 2008, 3:19 AM

Post #8 of 15 (2444 views)
Permalink

Re: C/R Pros and Cons [In reply to]

At 08:15 PM 10/14/2008 -0400, Stuart D. Gathman wrote:

>On Tue, 14 Oct 2008, Sanford Whiteman wrote:
>
>> The age-old "problem" of not being able to send mail because the
>> receiver set a policy that stopped being feasible in around 1983, or
>> whenever the first non-techie end user used SMTP.
>
>Since I pay to receive the mail, and it's my domain, I'll reject anyone I like,
>thank you very much. You are thinking in terms of a large free email provider
>like gmail. Spam may eventually drive large email providers out of business.

Recent trends (and current business strategy at some large ESPs) suggest the opposite. Large ESPs are growing, while smaller ESPs and ISPs that provide email as a sideline, are going out of business. The thinking at the large ESPs (and large spam appliance companies) seems to be that effective spam blocking is their competitive advantage, and it takes a really large company to do it right.

Part of the large-company strategy, unfortunately, is to hide problems with lost mail. Given current conditions, I use yahoo.com for my Transmitter, and my own "hobbyist" MTA for receiving.

As for reject policy, it should be the Recipient that decides whether to send a challenge, a normal reject, or use a quarantine. There should be a sensible default set by the Receiver for recipients that don't know or don't care.

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

vulcan at photonforge

Oct 15, 2008, 4:41 AM

Post #9 of 15 (2439 views)
Permalink

Re: C/R Pros and Cons [In reply to]

Don't forget that much legitimate email is sent by automated systems:
mailing lists, shopping receipts, financial institution notifications,
eBay notifications, system log monitors, and the like. No one is there
to respond to the challenge. Send enough rejection notices to the
software that runs a mailing list and you will be dropped from the
mailing list.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

stuart at bmsi

Oct 15, 2008, 7:48 AM

Post #10 of 15 (2442 views)
Permalink

Re[3]: C/R Pros and Cons [In reply to]

On Wed, 15 Oct 2008, Sanford Whiteman wrote:

> > I don't need to. Some 5xx codes, like 551, *are* already advisory in
> > rfc2821. It just means "can't deliver it that way, but try this other
> > way". 551 has been proposed as an alternative to SRS.
>
> Nowhere in 2821 is 551 treated as anything but a permanent error. It
> continues to be a "Permanent Negative Completion reply" in the 5yz
> range. 551 is noted as more correctable -- *via a new message
> envelope* -- than other permanent errors. By definition, any such

As it says above your response it is only permanent *for that recipient*.
A new message envelope - or a web query - is exactly what we are talking
about. With C/R (which I don't use - I'm just objecting to your rants)
the 5xx (or DSN) is in some sense a lie: the message is stored someplace. It
just won't be delivered until some event unrelated to SMTP takes place.
551 says delivery to that recipient is dead in the water - but suggests
another recipient to use instead.

> I believe that. I have many colleagues that do boutique anti-abuse
> hosting that do a better job than the big guys as well. But none of
> them, including you, use C/R to do it. And you wouldn't be any more
> likely to be able to get away with C/R for your customer base if you
> used this phantasmic envelope-time rejection idea.

I would never use C/R for a customer base. It is an option for
individuals who want to get only quality emails. But that is why
the current implementations are so bad. To send a proper DSN or
5xx requires hooks at the MTA level. Most people have some procmail
style script that sends a reply - aaaarrrrgggghhh! Now if MTAs would
only *support* (optionally) C/R schemes, then C/R backscatter would
not be such a problem.

--
Stuart D. Gathman <stuart [at] bmsi>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

dmquigg-spf at yahoo

Oct 15, 2008, 8:23 AM

Post #11 of 15 (2432 views)
Permalink

Re: C/R Pros and Cons [In reply to]

At 06:41 AM 10/15/2008 -0500, Roman Zimmermann wrote:

>Don't forget that much legitimate email is sent by automated systems: mailing lists, shopping receipts, financial institution notifications, eBay notifications, system log monitors, and the like. No one is there to respond to the challenge. Send enough rejection notices to the software that runs a mailing list and you will be dropped from the mailing list.

The automated system should stop retries after the *first* reject. Code 5xy means there is a problem on the sender side. The sender is supposed to fix the problem before trying again.

A properly designed C/R system should *not* expect all senders to respond. It should *not* increase the probability of a losing a legitimate message. The message is going to the quarantine anyway. A response will move it from the quarantine to the inbox. No response just leaves it where it is.

I like Stuart's observation that 5xy "reject" in this case (a hypothetical C/R system) is in some sense a lie: the message is actually received and stored in a quarantine. It just won't be "delivered" to the inbox until some event unrelated to SMTP takes place. This event could be the sender responding to the challenge, or the recipient noticing a valid message in his quarantine, and clicking "Not Spam".

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

vulcan at photonforge

Oct 15, 2008, 1:24 PM

Post #12 of 15 (2436 views)
Permalink

Re: C/R Pros and Cons [In reply to]

David MacQuigg wrote:
> The automated system should stop retries after the *first* reject. Code 5xy means there is a problem on the sender side. The sender is supposed to fix the problem before trying again.
>

When a sending MTA receives a permanent rejection reply code, it will
usually generate a bounce message. What the sending software does upon
receipt of this bounce varies. In the case Mailman, a mailing list
manager, it takes bounced emails spread across five days for it to drop
a subscriber.

5.x.y SMTP reply codes are permanent errors. It might be a permanent
error because a downed MTA was brought back up before its configuration
was fully debugged. The reply code might be generated due to the
receiving MTA's use of an RBL that is now blacklisting all queries.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

sandy at cypressintegrated

Oct 15, 2008, 4:37 PM

Post #13 of 15 (2434 views)
Permalink

Re[2]: C/R Pros and Cons [In reply to]

> A properly designed C/R system should *not* expect all senders to
> respond. It should *not* increase the probability of a losing a
> legitimate message.

And that is why this design exists only in your mind, until you
reinvent SMTP.

Your design dictates that some permanent non-deliverable messages be
transformed into "out-of-band action required for delivery" messages.
You suggest using standard SMTP permanent error codes to implement
this nuance. [.Never mind the fact that the final disposition of the
message, if indeed the message object is stored on an intermediate
server instead of deleted, is closer (if anything) to a new type of
transient error.] Implementing your design would require an end-to-end
revisiting of how major MTAs deal with permanent error codes.

I don't mean to be entirely dismissive. It is always tempting to
brainstorm "human intervention" solutions that occur during the SMTP
envelope, instead of creating post-acceptance DSNs and inevitably
creating backscatter. Yet you seem unwilling to see your design as
more fitting a new (E)SMTP _extension_, insisting that it is in some
way workable now. That is why it fits with standard kookery.

Why don't you rethink and propose your idea in terms of a new SMTP
extension (i.e., SMTP verb/s)? Your basic notion is to expose a
requeueing mechanism to the end user using some standard notation.
Most MTAs, for example, give the admin the ability to "Send now" a
specific message object from the retry queue, though this
functionality is rarely exposed to the end user due to obvious
security and stability concerns. Perhaps what you want is to tag some
messages a with a "source-requested retry allowed" and/or
"source-requested retry only" flag. (I would advise you to abandon
your idea of the _remote_ quarantine; that leaps over a security
barrier that doesn't make this feasible as a proposed extension, and
it also means that all messages will be accepted in entirety by the
remote server, which is a tremendous waste of bandwidth.) Messages
with such flags will be kept in the queue for the standard queue
lifecycle, but your extension will instruct that a context-specific
DSN be sent to the end user when both sides support the extension. A
receiver-supplied challenge based on transaction ID would be
presented. (URL to a CAPTCHA, whatever). The response would be added
to the message envelope and transmitted along with the message object
upon the subsequent retry.

By proposing a new SMTP extension, you would lessen skeptical
responses based on real-world experience. An SMTP extension would need
to be announced and supported on both sides in order to work, allowing
more graceful degradation based on mutual understanding of both sides'
capabilities. A receiver would decide whether to accept mail in
sessions during which extension support was not announced (as public
MXs implicitly decide to accept mail from senders who did not
authenticate, MIME bitness is negotiated, PIPELINING is agreed upon,
etc.). A mandatory spam weight could be given to messages submitted
without extension support, or they would be rejected. And so on.

> The message is going to the quarantine anyway. A response will move
> it from the quarantine to the inbox. No response just leaves it
> where it is.

It will "just" be in the quarantine, which at its *most* user-friendly
is linked in real-time to the user's Junk Mail folder. And we know
that JM folders are practical peers to people's Inboxes and sorting
rules -- there is no devaluation of JM folders, right? Well, no. This
fails to persuade or distract from the essentially imaginary quality
of the design.

--Sandy

------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
e-mail: sandy [at] cypressintegrated
------------------------------------

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

sandy at cypressintegrated

Oct 15, 2008, 4:37 PM

Post #14 of 15 (2435 views)
Permalink

Re[2]: C/R Pros and Cons [In reply to]

> 5.x.y SMTP reply codes are permanent errors. It might be a permanent
> error because a downed MTA was brought back up before its
> configuration was fully debugged. The reply code might be generated
> due to the receiving MTA's use of an RBL that is now blacklisting
> all queries.

Roman, thank you for taking the time to revisit some of the settled
truths about how automated senders and how they deal with mail
rejection.

--Sandy

------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
e-mail: sandy [at] cypressintegrated
------------------------------------

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

sandy at cypressintegrated

Oct 15, 2008, 4:40 PM

Post #15 of 15 (2435 views)
Permalink

Re[4]: C/R Pros and Cons [In reply to]

> As it says above your response it is only permanent *for that
> recipient*. A new message envelope - or a web query - is exactly
> what we are talking about. With C/R (which I don't use - I'm just
> objecting to your rants)

The case against C/R was settled years ago, and it's an embarrassment
to have to debate an even more primitive version than the one that
(briefly) saw the light of day.

The "old new" C/R that is being considered here only works in an
imaginary world where you get to publish a Standard RFC whenever you
want. Envelope-time C/R was already considered and dismissed. There is
nothing newly standardized in SMTP that makes it any more feasible
now.

> the 5xx (or DSN) is in some sense a lie: the message is stored
> someplace. It just won't be delivered until some event unrelated to
> SMTP takes place. 551 says delivery to that recipient is dead in the
> water - but suggests another recipient to use instead.

I fully understand the imaginary C/R implementation.

Unlike the real-world C/R implementations, it requires an
across-the-board rewrite of the way SMTP senders deal with an existing
class of permanent errors: hiding the permanence of the error and
substituting friendly language that directs the user to use an
out-of-band method of requeueing/unquarantining a stored message
object.

Like the real-world C/R implementations, it suffers from a plethora of
guaranteed deal-killing problems because listserves, et al. do not
read responses, whether they are true DSNs, massaged DSNs, or crafted
procmail responses.

Like the real-world C/R implementations, it permanently adds HTTP into
SMTP as if that's just a small step that will not at all influence the
willingness/ability of even *human* senders to pursue the final
delivery of the e-mail.

Unlike the real-world C/R implementations, it does not create
backscatter. Yet since the imaginary implementation isn't possible, it
hardly seems necessary to note this "pro".

And to believe in the imaginary C/R implementation apparently also
requires a suspension of disbelief that quarantines are *by their very
definition* outside of the people's day-to-day workflow.

--Sandy

------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
e-mail: sandy [at] cypressintegrated
------------------------------------

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/
Powered by Listbox: http://www.listbox.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads