Mailing List Archive: exim: dev

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO

Index | Next | Previous | View Threaded

bugzilla at logicalsolutns

Jun 20, 2009, 1:38 PM

Post #1 of 13 (2058 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO

bugzilla at logicalsolutns

Jun 20, 2009, 1:42 PM

Post #2 of 13 (1977 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #1 from bugzilla[at]logicalsolutns.com 2009-06-20 21:42:49 ---
This is actually the relevant section of the docs, though it shows essentially
the same thing:

http://www.exim.org/exim-html-4.50/doc/html/spec_39.html

39.31 Callout verification

...

For a sender callout check, Exim makes SMTP connections to the remote hosts, to
test whether a bounce message could be delivered to the sender address. The
following SMTP commands are sent:

HELO <smtp active host name>
MAIL FROM:<>
RCPT TO:<the address to be tested>
QUIT

******************************************

VRFY should be used, not RCPT TO:

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

nigel at exim

Jun 21, 2009, 12:47 AM

Post #3 of 13 (1978 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

Nigel Metheringham <nigel[at]exim.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |INVALID

--- Comment #2 from Nigel Metheringham <nigel[at]exim.org> 2009-06-21 08:47:50 ---
VRFY tests only for the apparent existence of an address (plus many sites block
it).

A FROM <>/RCPT TO pair tests the deliverability of bounce messages, including
whether the site accepts null senders (some idiots still think these should
be rejected), and whether the address itself is apparently valid.
This is an entirely different test to VRFY.

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

bugzilla at logicalsolutns

Jun 21, 2009, 1:55 AM

Post #4 of 13 (1976 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

bugzilla[at]logicalsolutns.com <bugzilla[at]logicalsolutns.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |bugzilla[at]logicalsolutns.com
Status|RESOLVED |REOPENED
Resolution|INVALID |

--- Comment #3 from bugzilla[at]logicalsolutns.com 2009-06-21 09:55:29 ---
Nigel,

RFC 821 requires VRFY and defines it as what we need for our purposes.
http://www.ietf.org/rfc/rfc821.txt

RFC 5321 requires VRFY and defines it as what we need for our purposes.
http://www.ietf.org/rfc/rfc5321.txt

>Comment #2 From Nigel Metheringham 2009-06-21 08:47:50 [reply] -------
>
>VRFY tests only for the apparent existence of an address (plus many sites block
it).
>
>A FROM <>/RCPT TO pair tests the deliverability of bounce messages, including whether the site accepts null senders (some idiots still think these should be rejected), and whether the address itself is apparently valid.
>
>This is an entirely different test to VRFY

I'd rather not get in the middle of your pissing war with the "idiots" you
describe.

However, because of the misuse of RCPT TO, in order to VRFY addresses, my IP is
now listed on a DNSBL @ backscatter.org

http://www.backscatterer.org/index.php?target=sendercallouts

If you would please review my original writeup and give it more careful
consideration.

If the RFC calls for VRFY to be enabled, and sites choose to remove VRFY from
their MTA command response set, then I have no problem 'dropping' their inbound
mail as "unverifiable".

However, it is wrong headed (and two wrongs don't make a right) to abuse RCPT
TO, in order to "get around" the admin's who have disabled VRFY.

Bottom line: If you refuse to consider making EXIM work within the RFC's and
use VRFY instead of RCPT TO, then please consider adding a NEW feature that
uses RFC specified VRFY.

Otherwise, users of EXIM will either find themselves BLACK LISTED or be forced
to stop using acl_smtp_vrfy

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

graeme at graemef

Jun 21, 2009, 3:02 AM

Post #5 of 13 (1974 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

Graeme Fowler <graeme[at]graemef.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |graeme[at]graemef.net

--- Comment #4 from Graeme Fowler <graeme[at]graemef.net> 2009-06-21 11:02:02 ---
(In reply to comment #3)
> RFC 821 requires VRFY and defines it as what we need for our purposes.
> http://www.ietf.org/rfc/rfc821.txt
>
> RFC 5321 requires VRFY and defines it as what we need for our purposes.
> http://www.ietf.org/rfc/rfc5321.txt

Indeed. And because of well documented problems with nefarious types using VRFY
as a dictionary address harvester, most sites switch it off. It's been
practically irrelevant for years, but is still implemented within many MTA
applications because it's mandated in the RFC.

All that said:

> However, because of the misuse of RCPT TO, in order to VRFY addresses, my IP is
> now listed on a DNSBL @ backscatter.org

You would likely be listed elsewhere if you continually did VRFY requests, too,
as they're not seen as being particularly friendly these days. I can't recall
the last time I saw a site with VRFY switched *on*.

> However, it is wrong headed (and two wrongs don't make a right) to abuse RCPT
> TO, in order to "get around" the admin's who have disabled VRFY.

The issue here is whether to do callout verification of *any* type to validate
an incoming email. A Joe Job would cause just as much pain using VRFY as using
RCPT TO.

IMO the documentation should be changed, not the code, to include a warning as
follows:

============================================================================
WARNING
============================================================================

Global, blanket usage of callout verification to validate message _senders_
against arbitrary external hosts is frequently considered abusive. Callout
verification of any sort (sender or recipient) should be restricted to networks
or hosts which are either under the same organisational control, or with which
a trust relationship exists.

Arbitrary usage of callouts to verify the existence of sender addresses may
lead to the calling host being added to several DNSBLs which will cause
problems for messages traversing the affected system.

============================================================================

There are many cases (as described above) where this type of verification is
perfectly valid (or indeed VRFY is). Switching, however, to VRFY which is
largely obsolete nowadays, would be a mistake.

Graeme

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

bugzilla at logicalsolutns

Jun 21, 2009, 3:27 AM

Post #6 of 13 (1975 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #5 from bugzilla[at]logicalsolutns.com 2009-06-21 11:27:27 ---
(In reply to comment #4)
> (In reply to comment #3)

> There are many cases (as described above) where this type of verification is
> perfectly valid (or indeed VRFY is). Switching, however, to VRFY which is
> largely obsolete nowadays, would be a mistake.
>
> Graeme
>

Thank you.

My counter would be:

If VRFY has been disabled by many sites, and if both RCPT TO and VRFY get you
DNSBL'd for doing either of the verifications with foreign sources...

That leaves only 'internal' handshaking between 'friendly' servers. And, we're
back to VRFY, which was designed for that purpose.

Even if, as you suggest, VRFY would get us listed, clearly RCPT TO already gets
us listed and is, therefore, useless as coded.

-john

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

graeme at graemef

Jun 21, 2009, 3:44 AM

Post #7 of 13 (1976 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #6 from Graeme Fowler <graeme[at]graemef.net> 2009-06-21 11:44:49 ---
(In reply to comment #5)
> My counter would be:
>
> If VRFY has been disabled by many sites, and if both RCPT TO and VRFY get you
> DNSBL'd for doing either of the verifications with foreign sources...

Quite the Catch-22. Is that you, Yosarian?

> That leaves only 'internal' handshaking between 'friendly' servers. And, we're
> back to VRFY, which was designed for that purpose.

But - VRFY gives a simple binary answer without allowing additional operations
which the use of RCPT TO does on the "server end" - consider the run through
Exim's RCPT ACL, which can't be done using VRFY in the same way (I don't
think!).

> Even if, as you suggest, VRFY would get us listed, clearly RCPT TO already gets
> us listed and is, therefore, useless as coded.

Not exactly. There are many cases (like in my dayjob where there are a small
number of "external", to use, systems to which we deliver mail over which we
have no control and all of whch have VRFY switched off) where a recipient
callout using RCPT TO is the Right Thing To Do. Also, consider that the
response to RCPT TO can vary according to the MAIL FROM part of the transaction
- none of that degree of variance is available using VRFY, which is a simple
yes/no answer in response to a simple question.

Graeme

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

peter at bowyer

Jun 21, 2009, 4:24 AM

Post #8 of 13 (1976 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

Peter Bowyer <peter[at]bowyer.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |peter[at]bowyer.org

--- Comment #7 from Peter Bowyer <peter[at]bowyer.org> 2009-06-21 12:24:18 ---
(In reply to comment #3)
>
> However, because of the misuse of RCPT TO, in order to VRFY addresses, my IP is
> now listed on a DNSBL @ backscatter.org

Then stop mis-using it. Don't suggest the removal of a feature many find
useful.

Graeme's documentation suggestion in comment #4 seems sensible, however, to
avoid accidental mis-use. Note also that the default config does not (and never
has) enable sender callouts so new users have to deliberately turn it on and do
not need protecting.

Exim is a very stable system, and backwardly-incompatible changes must be
avoided. If you'd care to enter a suggestion to add access to 'VRFY' from the
ACLs in addition to the existing callout mechanism, further consideration would
likely be given. But the devs would probably need convincing as to how widely
it could be used given the widespread disabling of VRFY in the real world.

-1 for code change, +1 for doc change

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

bugzilla at logicalsolutns

Jun 21, 2009, 7:15 AM

Post #9 of 13 (1972 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #8 from bugzilla[at]logicalsolutns.com 2009-06-21 15:15:39 ---
(In reply to comment #7)
> (In reply to comment #3)
> >
> > However, because of the misuse of RCPT TO, in order to VRFY addresses, my IP is
> > now listed on a DNSBL @ backscatter.org
>
> Then stop mis-using it. Don't suggest the removal of a feature many find
> useful.

I use Sendmail for all but one of my servers. That server has CPanel and EXIM.

The CPanel config has "sender callout verification" as a checkmark option.

The CPanel technician indicated that Exim would use VRFY

Only _after_ our IP got listed by backscatter.org did I become aware of the
flaws in the system.

There was _nothing_ from CPanel to indicate that there was any 'mis-use'.
There was _nothing_ from Exim to indicate that there was any 'mis-use'.

> Graeme's documentation suggestion in comment #4 seems sensible, however, to
> avoid accidental mis-use. Note also that the default config does not (and never
> has) enable sender callouts so new users have to deliberately turn it on and do
> not need protecting.

With no warnings from Exim, and no warnings from CPanel, and no indication that
enabling it was 'mis-use', I think placing blame on new users is a bit odd.

> Exim is a very stable system, and backwardly-incompatible changes must be
> avoided. If you'd care to enter a suggestion to add access to 'VRFY' from the
> ACLs in addition to the existing callout mechanism, further consideration would
> likely be given. But the devs would probably need convincing as to how widely
> it could be used given the widespread disabling of VRFY in the real world.
>
> -1 for code change, +1 for doc change
>

I support document change, both in Exim and in Cpanel.

I was trying to use something that looked like it would be a useful tool, with
no indication that the "bridge is out ahead".

Now I'm attempting to help other uninformed users by getting "something"
changed.

Thank you.

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

peter at bowyer

Jun 21, 2009, 7:36 AM

Post #10 of 13 (1971 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #9 from Peter Bowyer <peter[at]bowyer.org> 2009-06-21 15:36:13 ---
(In reply to comment #8)

>
> The CPanel config has "sender callout verification" as a checkmark option.
>
> The CPanel technician indicated that Exim would use VRFY

It should be fairly easy to identify the problematic component, then.

>
>
> With no warnings from Exim, and no warnings from CPanel, and no indication that
> enabling it was 'mis-use', I think placing blame on new users is a bit odd.

An incompatible change might be proposed on the basis that the default
configuration is dangerous in the hands of new users. I was simply discounting
this as a factor in this case. Of course, CPanel may or may not use the default
configuration or even be derived from it.

>
>
> > Exim is a very stable system, and backwardly-incompatible changes must be
> > avoided. If you'd care to enter a suggestion to add access to 'VRFY' from the
> > ACLs in addition to the existing callout mechanism, further consideration would
> > likely be given. But the devs would probably need convincing as to how widely
> > it could be used given the widespread disabling of VRFY in the real world.
> >
> > -1 for code change, +1 for doc change
> >
>
> I support document change, both in Exim and in Cpanel.

CPanel users are unlikely to see the Exim documentation, unfortunately. You'll
need to lobby the CPanel maintainers.

>
> I was trying to use something that looked like it would be a useful tool, with
> no indication that the "bridge is out ahead".
>
> Now I'm attempting to help other uninformed users by getting "something"
> changed.

Fair do's, no problem with that.

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

graeme at graemef

Jun 21, 2009, 7:50 AM

Post #11 of 13 (1973 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #10 from Graeme Fowler <graeme[at]graemef.net> 2009-06-21 15:50:03 ---
On Sun, 2009-06-21 at 15:15 +0100, bugzilla[at]logicalsolutns.com wrote:
> I use Sendmail for all but one of my servers. That server has CPanel and EXIM.
>
> The CPanel config has "sender callout verification" as a checkmark option.
>
> The CPanel technician indicated that Exim would use VRFY

Aha, it all becomes clear. This seems to be a fundamental
misunderstanding and perhaps documentation "bug" on the part of cPanel.
They are clearly making the option available without, perhaps, fully
understanding how the option works.

> There was _nothing_ from CPanel to indicate that there was any 'mis-use'.
> There was _nothing_ from Exim to indicate that there was any 'mis-use'.

I would not expect there to be. cPanel is a "self-contained" hosting
platform which makes very heavy customisations to a lot of packages
(Apache httpd, Exim, PHP and so on) but rarely goes upstream to discuss
things (in my experience).

> With no warnings from Exim, and no warnings from CPanel, and no indication that
> enabling it was 'mis-use', I think placing blame on new users is a bit odd.

That could have been avoided if you'd mentioned cPanel in your initial
report. I can guarantee, absolutely, that they won't be providing the
default config file - their raison d'etre is to provide a buffer between
you, the user, and the underlying components so you can manage your
hosting system via a web interface.

Unfortunately we spend a lot of time answering questions regarding
various Exim switches and options where it becomes very obvious that the
person raising the question has not read the docs (which is poor form).
This is similar to the "don't press this button" psychological
experiment; in your case you aren't really to blame, as such, because
you couldn't see the button you were pressing until it was too late. You
then lifted the covers (cPanel) to see what you were leaning on (the
Exim config)...

I'll quit with the poor analogy now :)

> I was trying to use something that looked like it would be a useful tool, with
> no indication that the "bridge is out ahead".

I think you can take it back to the cPanel maintainers, in that case.
They will need to improve their docs. Feel free to pass on the proposed
warning I posted earlier.

Graeme

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

bugzilla at logicalsolutns

Jun 21, 2009, 8:16 AM

Post #12 of 13 (1973 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

--- Comment #11 from bugzilla[at]logicalsolutns.com 2009-06-21 16:16:44 ---
> > I was trying to use something that looked like it would be a useful tool, with
> > no indication that the "bridge is out ahead".
>
> I think you can take it back to the cPanel maintainers, in that case.
> They will need to improve their docs. Feel free to pass on the proposed
> warning I posted earlier.
>
> Graeme
>

Graeme,

You can be assured, I have done that.

Hopefully you still consider your suggested changes to the Exim documentation
to be relevant.

The CPanel developers might have been more 'in touch' with the situation, had
your warning been in the documentation. And, I'm not sure its reasonable to
expect them to change their documentation or control panel, if Exim doesn't
change theirs.

Thank you again, on record, for being very responsive and conversational.

It was, indeed, welcome relief, after being tar'd and feather'd by the admin
newsgroup.

-john

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

grin at grin

Jun 26, 2009, 3:33 AM

Post #13 of 13 (1832 views)
Permalink

[Bug 855] Sender-callout-Verification should use VRFY not RCPT TO [In reply to]

------- You are receiving this mail because: -------
You are on the CC list for the bug.

http://bugs.exim.org/show_bug.cgi?id=855

Peter Gervai <grin[at]grin.hu> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |grin[at]grin.hu
Severity|bug |wishlist

--- Comment #12 from Peter Gervai <grin[at]grin.hu> 2009-06-26 11:33:45 ---
There is possible merits in implementing a switch to be able to use VRFY
additional to the current method.

The positive side is:
- possibility to follow other people's advices about it ;)
- try to be nicer to others
- prevent religious anti-antispam people to blacklist it in backstreet RBLs

Negative side is:
- it would be close to useless as most sites disable VRFY altogether (which is
stupid since RCPT TO reveals the same, if not more, information)
- ..thus most people wouldn't use it anyway
- require some new code to be written

And to make it spelled out: we *do* need RCPT checks since it's the only method
working in today's environment. We cannot move over to VRFY unless it's
compulsory to properly support, and even then we can expect some people to
violate that and result fake responses.

So this is not a bug, but a wishlist item.

--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com