Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: exim: dev

Re: SMTP PRDR

 

 

exim dev RSS feed   Index | Next | Previous | View Threaded


jgh at wizmail

Mar 19, 2012, 10:47 AM

Post #1 of 12 (1017 views)
Permalink
Re: SMTP PRDR

I'm in no sense an authority, but here's my take:

On 2012-03-19 16:19, Todd Lyons wrote:
> MAIL FROM:<user [at] example> PRDR
>
> In the code which parses args to mail and rcpt commands, it requires
> key=value pairs (for example, BODY=149833),

Actually, despite the comment on extract_option() I don't see it being
called at RCPT. Maybe I'm missing something, or maybe the
comment needs correcting.


> So the extract_option requires a key=value pair. At this point, the
> has not yet verified that the options are valid/invalid, it's just
> splitting them from the email address. With contemporary exim design
> standards in mind:
>
> 1) would it better to pass a variable to extract_option() to indicate
> that "=" is not required?

Given the limited number of callers I'd be happy with such a mod.
The problem is that it's still required for the current options,
and you've not determined what the option is yet.

> 2) would it be better to make a duplicate function that basically just
> does #1 (i.e. not require "=")
> 3) something else?

I'd like a code refactor from the if/elseif chain into a table-driven
approach, using a static data table of acceptable option names
each with a "supplies argument" indicator. This would be in line
with (eg.) the parsing in acl_verify().

Admission: I've been a-refactoring in verify myself (bug 1184).


Good luck; it looks like a worthwhile project.
--
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


tlyons at ivenue

Mar 19, 2012, 11:22 AM

Post #2 of 12 (983 views)
Permalink
Re: SMTP PRDR [In reply to]

On Mon, Mar 19, 2012 at 10:47 AM, Jeremy Harris <jgh [at] wizmail> wrote:
> Actually, despite the comment on extract_option() I don't see it being
> called at RCPT.  Maybe I'm missing something, or maybe the
> comment needs correcting.

I took it at face value, but it appears to need correcting since it's
not called in the RCPT_CMD case.

>> So the extract_option requires a key=value pair.  At this point, the
>> has not yet verified that the options are valid/invalid, it's just
>> splitting them from the email address.  With contemporary exim design
>> standards in mind:
>> 1) would it better to pass a variable to extract_option() to indicate
>> that "=" is not required?
>
> Given the limited number of callers I'd be happy with such a mod.
> The problem is that it's still required for the current options,
> and you've not determined what the option is yet.
>
> I'd like a code refactor from the if/elseif chain into a table-driven
> approach, using a static data table of acceptable option names
> each with a "supplies argument" indicator.  This would be in line
> with (eg.) the parsing in acl_verify().

...that makes a lot of sense. Until now, everything that exim
supported required an argument, so bailing on the absence of an "="
was sufficient for a quick sanity check. To support non-argument
options, each option will have to be sanity checked and validated at
the same time. Doing this in a table would be, theoretically, better
able to do both at once.

> Good luck; it looks like a worthwhile project.

Thanks for the encouragement.

...Todd
--
Always code as if the guy who ends up maintaining your code will be a
violent psychopath who knows where you live. -- Martin Golding

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


jgh at wizmail

Jan 24, 2013, 1:31 PM

Post #3 of 12 (873 views)
Permalink
Re: SMTP PRDR [In reply to]

> On 2012-03-19 16:19, Todd Lyons wrote:
>> MAIL FROM:<user [at] example> PRDR

I've pushed a working version of PRDR in Exim to

http://git.exim.org/users/jgh/exim.git/log/refs/heads/todd_prdr

Todd gets credit for kicking off the effort and doing the server side.
I did the client side. Undoubtedly there are bugs and missing
features; please speak up if you're interested.

I'll wait a little while but at some point I'd like to merge a squashed
version back into the Exim mainline.

Here's the spiel from experimental-spec.txt :

-------------

Per-Recipient Data Response is an SMTP extension proposed by Eric Hall
in a (now-expired) IETF draft from 2007. It's not hit mainstream
use, but has apparently been implemented in the META1 MTA.

There is mention at http://mail.aegee.org/intern/sendmail.html
of a patch to sendmail "to make it PRDR capable".

ref: http://www.eric-a-hall.com/specs/draft-hall-prdr-00.txt

If Exim is built with EXPERIMENTAL_PRDR there is a new config
boolean "prdr_enable" while controls whether PRDR is advertised
as part of an EHLO response, a new "acl_data_smtp_prdr" ACL
(called for each recipient, after data arrives but before the
data ACL), and a new smtp transport option "hosts_try_prdr".

PRDR may be used to support per-user content filtering. Without it
one must defer any recipient after the first that has a different
content-filter configuration. With PRDR, the RCPT-time check
for this can be disabled when the MAIL-time $smtp_command included
"PRDR". Any required difference in behaviour of the main DATA-time
ACL should however depend on the PRDR-time ACL having run, as Exim
will avoid doing so in some situations (eg. single-recipient mails).

--
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


jgh at wizmail

Jan 27, 2013, 6:31 AM

Post #4 of 12 (842 views)
Permalink
Re: SMTP PRDR [In reply to]

On 01/24/2013 09:31 PM, Jeremy Harris wrote:
> I'd like to merge a squashed
> version back into the Exim mainline.

Now done.
--
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


tlyons at ivenue

Feb 5, 2013, 5:53 PM

Post #5 of 12 (840 views)
Permalink
Re: SMTP PRDR [In reply to]

On Sun, Jan 27, 2013 at 6:31 AM, Jeremy Harris <jgh [at] wizmail> wrote:
> On 01/24/2013 09:31 PM, Jeremy Harris wrote:
>>
>> I'd like to merge a squashed
>> version [of PRDR] back into the Exim mainline.
> Now done.

It's now built and running on m.lm.ivenue.com. It's jgh's prdr
branch, which is not very far behind current master.

[todd [at] tlyon ~/projects/exim_jgh (prdr)]$ git remote -v
origin git://git.exim.org/users/jgh/exim.git (fetch)
origin git://git.exim.org/users/jgh/exim.git (push)

I still have to create some test users and rules to apply to them, but
it's a live server and capable of testing (once I get the users
created). There is at least one live (to the public internet) domain
that I will be able to create these test users on.

[root [at] ivhkwww5 ~]# telnet m.lm.ivenue.com 25
Trying 208.89.138.62...
Connected to m.lm.ivenue.com.
Escape character is '^]'.
220-m.lm.ivenue.com, ESMTP Exim 4.81_RC1+libopendmarc, Wed, 06 Feb 2013
220 01:38:53 +0000
ehlo m.ivenue.com
250-m.lm.ivenue.com Hello m.ivenue.com [120.136.44.7]
250-SIZE 52428800
250-8BITMIME
250-ETRN
250-EXPN
250-AUTH PLAIN LOGIN
250-STARTTLS
250-PRDR
250 HELP
quit
221 m.lm.ivenue.com closing connection
Connection closed by foreign host.

Jeremy, hit me up in IRC when you see me logged in and we can do some
remote to remote testing. But it may take some wiggle room on your
part. My work has been slammed lately with a lot of things that came
up at once, so my available time to work on this is limited until I
get through the backlog.

...Todd
--
The total budget at all receivers for solving senders' problems is $0.
If you want them to accept your mail and manage it the way you want,
send it the way the spec says to. --John Levine

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


tlyons at ivenue

Feb 6, 2013, 10:53 AM

Post #6 of 12 (857 views)
Permalink
Re: SMTP PRDR [In reply to]

On Tue, Feb 5, 2013 at 5:53 PM, Todd Lyons <tlyons [at] ivenue> wrote:
> On Sun, Jan 27, 2013 at 6:31 AM, Jeremy Harris <jgh [at] wizmail> wrote:
>> On 01/24/2013 09:31 PM, Jeremy Harris wrote:
>>>
>>> I'd like to merge a squashed
>>> version [of PRDR] back into the Exim mainline.
>> Now done.
>
> It's now built and running on m.lm.ivenue.com. It's jgh's prdr
> branch, which is not very far behind current master.
>
> [todd [at] tlyon ~/projects/exim_jgh (prdr)]$ git remote -v
> origin git://git.exim.org/users/jgh/exim.git (fetch)
> origin git://git.exim.org/users/jgh/exim.git (push)
>
> I still have to create some test users and rules to apply to them, but
> it's a live server and capable of testing (once I get the users
> created). There is at least one live (to the public internet) domain
> that I will be able to create these test users on.

Users are created and available for testing:
todd [at] ivtestdomain
todddefer [at] ivtestdomain (will always be deferred)
toddreject [at] ivtestdomain (do I have to describe it?)


I do have a question about behavior. Witness the following SMTP conversation:

[todd [at] tlyon ~]$ telnet m.lm.ivenue.com 25
Trying 208.89.138.62...
Connected to m.lm.ivenue.com.
Escape character is '^]'.
220-m.lm.ivenue.com, ESMTP Exim 4.81_RC1+libopendmarc, Wed, 06 Feb 2013
220 18:31:38 +0000
EHLO tlyons.ivenue.net
250-m.lm.ivenue.com Hello tlyons.ivenue.net [192.168.1.16]
250-SIZE 52428800
250-8BITMIME
250-ETRN
250-EXPN
250-AUTH PLAIN LOGIN
250-STARTTLS
250-PRDR
250 HELP
MAIL FROM:<todd [at] mrball> PRDR
250 OK, PRDR Requested
RCPT TO:<toddreject [at] ivtestdomain>
250 Accepted
RCPT TO:<todddefer [at] ivtestdomain>
250 Accepted
RCPT TO:<todd [at] ivtestdomain>
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
<snip valid message>
.
353 PRDR content analysis beginning
550 PRDR R=<toddreject [at] ivtestdomain> refusal
450 PRDR R=<todddefer [at] ivtestdomain> temporary refusal
250 PRDR R=<todd [at] ivtestdomain> acceptance
550 Detected dedicated spamware generated email
quit
221 m.lm.ivenue.com closing connection

It seems like the 250 PRDR response should not get issued until after
the message passes the actual DATA acl, no? You can't issue a 250 and
then turn around and reject it AFAIK.

...5 minutes later...

WRONG. According to the draft:

4.5.3. The Final Response Code

After the server has transmitted all of the recipient-specific
response lines, a final response that indicates whether or not
the server will accept responsibility for the email message as
a whole MUST be returned.j

So in this case, it is doing exactly as it is spec'd. My initial
trial shows that it works, for this narrow test case. Anybody else
from the outside want to test PRDR against my test server, go for it.
Give me some return mailboxes that I can use to test the client
implementation as well.

...Todd
--
The total budget at all receivers for solving senders' problems is $0.
If you want them to accept your mail and manage it the way you want,
send it the way the spec says to. --John Levine

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


jgh at wizmail

Feb 6, 2013, 12:21 PM

Post #7 of 12 (831 views)
Permalink
Re: SMTP PRDR [In reply to]

On 02/06/2013 06:53 PM, Todd Lyons wrote:
> 353 PRDR content analysis beginning
> 550 PRDR R=<toddreject [at] ivtestdomain> refusal
> 450 PRDR R=<todddefer [at] ivtestdomain> temporary refusal
> 250 PRDR R=<todd [at] ivtestdomain> acceptance
> 550 Detected dedicated spamware generated email
> quit
> 221 m.lm.ivenue.com closing connection
[...]
> After the server has transmitted all of the recipient-specific
> response lines, a final response that indicates whether or not
> the server will accept responsibility for the email message as
> a whole MUST be returned.j
>
> So in this case, it is doing exactly as it is spec'd.

That's how I read the draft, at least :)

What human-readable message ought to go with the final is open
to argument though.
--
Cheers,
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


pdp at exim

Feb 6, 2013, 1:33 PM

Post #8 of 12 (839 views)
Permalink
Re: SMTP PRDR [In reply to]

On 2013-02-06 at 10:53 -0800, Todd Lyons wrote:
> .
> 353 PRDR content analysis beginning
> 550 PRDR R=<toddreject [at] ivtestdomain> refusal
> 450 PRDR R=<todddefer [at] ivtestdomain> temporary refusal
> 250 PRDR R=<todd [at] ivtestdomain> acceptance
> 550 Detected dedicated spamware generated email


> It seems like the 250 PRDR response should not get issued until after
> the message passes the actual DATA acl, no? You can't issue a 250 and
> then turn around and reject it AFAIK.
>
> ...5 minutes later...
>
> WRONG. According to the draft:
>
> 4.5.3. The Final Response Code
>
> After the server has transmitted all of the recipient-specific
> response lines, a final response that indicates whether or not
> the server will accept responsibility for the email message as
> a whole MUST be returned.j

I think the interpretation you are placing upon that is not the
interpretation expected, as evidenced by the example in section 5.1 of
partial failure.

Do remember that you're working from a _first_ draft of an RFC, so
it's not as polished on these issues as we'd normally expect from a
modern full RFC.

----------------------------8< cut here >8------------------------------
5.1. One Failure, One Success

The following dialog illustrates a message with two recipients,
one of which refuses the content, the other of which accepts:
[...]
C: DATA
S: 354 go ahead
C: <sends data>
C: <crlf>.<crlf>
S: 353 content analysis has started
S: 550 5.6.0 fighter [at] example refuses the content
S: 250 2.1.5 lover [at] example accepts the content
S: 250 data accepted by some recipients

Since the message was acceptable to one of the recipients, and
since the server is able to accept the message for subsequent
processing, the final response indicates success. At this point,
the SMTP client would need to clear the message from its queue,
and then generate a disposition or error message on behalf of the
failing recipient.
----------------------------8< cut here >8------------------------------

So the 250 means that the message was accepted for some recipient, the
message identified by that Message-ID header now exists on the receiving
server, but only for those recipients that were individually accepted by
PRDR. The sender has to generate bounces for the other recipients.

So, if _any_ of the recipients gave a 250 response, then overall a 250
response must be given.

Further, I'd expect that to be the case so that the local system's
queue-id (Exim $message_id) can be given in the final 250 response, as
per normal, so that the sender's logs can record that queue-id, for
correlation in tracing message flow as folks try to debug what happened
to a particular mail.

If Exim sends mail with PRDR and the receiving system takes it for one
recipient, what ends up in the sending-Exim's log-lines in the C="..."
field? Without PRDR, it's the one 250 line, which will contain the
queue-id. With PRDR, which of the two makes it in, and why? Or both?
It seems that both make some sense, but we definitely do need the final
response, to get the queue-id.

-Phil

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


jgh at wizmail

Feb 6, 2013, 1:45 PM

Post #9 of 12 (835 views)
Permalink
Re: SMTP PRDR [In reply to]

On 02/06/2013 09:33 PM, Phil Pennock wrote:
> So, if _any_ of the recipients gave a 250 response, then overall a 250
> response must be given.

Unless the system decided to overall-reject the mail, yes? I.e. in a
later checking phase than the per-recipient ones.

--
Jeremy



--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


tlyons at ivenue

Feb 6, 2013, 3:19 PM

Post #10 of 12 (825 views)
Permalink
Re: SMTP PRDR [In reply to]

On Wed, Feb 6, 2013 at 1:45 PM, Jeremy Harris <jgh [at] wizmail> wrote:
> On 02/06/2013 09:33 PM, Phil Pennock wrote:
>>
>> So, if _any_ of the recipients gave a 250 response, then overall a 250
>> response must be given.
> Unless the system decided to overall-reject the mail, yes? I.e. in a
> later checking phase than the per-recipient ones.

I think this is why Phil would prefer the DATA ACL to run first, then
deliver final answers as each recipient is run through the PRDR ACL.
Otherwise, you have to delay each PRDR answer given to the sending
system until you complete the DATA ACL (and possibly adjust that final
answer based on prior PRDR results...ugh it can get twisty).

...Todd
--
The total budget at all receivers for solving senders' problems is $0.
If you want them to accept your mail and manage it the way you want,
send it the way the spec says to. --John Levine

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


pdp at exim

Feb 6, 2013, 3:52 PM

Post #11 of 12 (825 views)
Permalink
Re: SMTP PRDR [In reply to]

On 2013-02-06 at 21:45 +0000, Jeremy Harris wrote:
> On 02/06/2013 09:33 PM, Phil Pennock wrote:
> > So, if _any_ of the recipients gave a 250 response, then overall a 250
> > response must be given.
>
> Unless the system decided to overall-reject the mail, yes? I.e. in a
> later checking phase than the per-recipient ones.

True, sorry.

So a recipient getting a 250 response can still be overruled by a 550 at
the end. Hrm.

How does this play with 4xx deferred responses?

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##


jgh at wizmail

Feb 7, 2013, 4:27 AM

Post #12 of 12 (822 views)
Permalink
Re: SMTP PRDR [In reply to]

On 06/02/2013 23:52, Phil Pennock wrote:
> On 2013-02-06 at 21:45 +0000, Jeremy Harris wrote:
>> On 02/06/2013 09:33 PM, Phil Pennock wrote:
>>> So, if _any_ of the recipients gave a 250 response, then overall a 250
>>> response must be given.
>>
>> Unless the system decided to overall-reject the mail, yes? I.e. in a
>> later checking phase than the per-recipient ones.
>
> True, sorry.
>
> So a recipient getting a 250 response can still be overruled by a 550 at
> the end. Hrm.
>
> How does this play with 4xx deferred responses?
>

As far as I can see, the most restrictive response takes effect - 5xx
overrides 4xx overrides 2xx.
--
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

exim dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.