Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Qmail: users

Patch to redirect bounce messages

 

 

Qmail users RSS feed   Index | Next | Previous | View Threaded


p.a.terry.lists at gmail

Apr 17, 2009, 3:47 AM

Post #1 of 9 (2091 views)
Permalink
Patch to redirect bounce messages

Hi

I've written a patch to redirect bounce messages to a host specified in
/var/qmail/control/bounceredirect that can be found here:

http://homepage.eircom.net/~qmailpatch/

Comments or suggestions for improvement would be very welcome.

Thanks,

Paul


amb-1166294724 at bradfords

Apr 19, 2009, 3:45 PM

Post #2 of 9 (1951 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

Thus said Paul Terry on Fri, 17 Apr 2009 11:47:10 BST:

> I've written a patch to redirect bounce messages to a host specified
> in /var/qmail/control/bounceredirect that can be found here:

Without looking at your patch I'm not exactly sure what it does, but if
you're taking bounces that would normally be sent to the original sender
and redirecting them somewhere else, this seems like a bad idea. This
will make your mail system look much more unreliable to anyone sending
email to/from your server.

Assuming I'm using your mail server, and I send an email to someone but
I accidentally mistype the email address, I would expect the bounce
message to be sent to me. Otherwise, it seems like the mail was
successful (from the end user perspective) and I am not aware that I
send it to the wrong address because the bounce was sent elsewhere.

Now, if this system isn't going to be used at all by users (or for
receiving email by non-users) then I suppose you might have a good
reason for it (the patch), in which case you don't likely need
suggestions except maybe for the correctness of the code.

Andy
--
[-----------[system uptime]--------------------------------------------]
4:45pm up 40 min, 1 user, load average: 1.01, 1.15, 1.08


p.a.terry.lists at gmail

Apr 20, 2009, 4:14 AM

Post #3 of 9 (1965 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

The idea behind the patch is to have a dedicated host(s) for handling the
delivery of bounce messages.

This should have the following benefits:

a) Reduced queue sizes on customer facing mail servers.
b) Easier queue cleaning.

These would be particularly relevant where backscatter from spam is causing
problems.

Paul


johnl at iecc

Apr 20, 2009, 6:28 AM

Post #4 of 9 (1937 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

>These would be particularly relevant where backscatter from spam is
>causing problems.

Wouldn't it be a better idea to apply Andrew Richards' qmail-verify
patch to do SMTP rejections rather than bounces? Back in 1998 it was
reasonable for qmail to accept and bounce, but on today's net where
95% of mail is spam and most of that is to nonexistent addresses, it's
not.

If you're sending vast numbers of bounces, you need to stop sending
them, not figure out a faster way to send them. Another good option
is Bruce Guenter's mailfront, which includes plugins to do SMTP-time
address checking and lots more useful stuff.

R's,
John


p.a.terry.lists at gmail

Apr 20, 2009, 7:35 AM

Post #5 of 9 (1948 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

Wouldn't it be a better idea to apply Andrew Richards' qmail-verify
> patch to do SMTP rejections rather than bounces? Back in 1998 it was
> reasonable for qmail to accept and bounce, but on today's net where
> 95% of mail is spam and most of that is to nonexistent addresses, it's
> not.


I tend to agree with this, but due to the number of customer complaints
we're
getting, not sending bounce messages isn't an option. Giving rejections
during
an SMTP session provides a means for a spammer to determine legitimate
addresses.


>
> If you're sending vast numbers of bounces, you need to stop sending
> them, not figure out a faster way to send them.


I think a greater problem is differentiating which bounces are as a result
of spam
and which ones aren't. With this patch, bounces can be sent to a dedicated
box to
address this problem without having to worry about the effects that resource

intensive code will have on the main queue.

Paul


josh at honorablemenschen

Apr 20, 2009, 8:14 AM

Post #6 of 9 (1950 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

> Wouldn't it be a better idea to apply Andrew Richards' qmail-verify
>> patch to do SMTP rejections rather than bounces? Back in 1998 it was
>> reasonable for qmail to accept and bounce, but on today's net where
>> 95% of mail is spam and most of that is to nonexistent addresses, it's
>> not.
>
>
> I tend to agree with this, but due to the number of customer complaints
> we're
> getting, not sending bounce messages isn't an option. Giving rejections
> during
> an SMTP session provides a means for a spammer to determine legitimate
> addresses.
>
>
Paul,

can you please give me all of your SMTP server IP addresses so that I can
add them to my deny list? While I understand that rejecting undeliverable
mail may help spammers weed out the email addresses that are invalid, it
also drastically cuts down on the total volume of spam processed, and I
really don't feel like getting spam bouncebacks from someone who isn't
bothering to reject undeliverables at the SMTP layer.

>>
>> If you're sending vast numbers of bounces, you need to stop sending
>> them, not figure out a faster way to send them.
>
>
> I think a greater problem is differentiating which bounces are as a result
> of spam
> and which ones aren't. With this patch, bounces can be sent to a dedicated
> box to
> address this problem without having to worry about the effects that
> resource
>
> intensive code will have on the main queue.
>
If you're concerned about the resources used to process spam on your main
server, why are you accepting them at all? For every message that you
accept that is undeliverable, you still have to process it into the queue
(and do any pre-processing, like virus scanning - you do virus scan at the
SMTP layer, right?), attempt to deliver it, and then send it on to your
bounce processing system via another SMTP delivery once the bounce message
is generated. Wouldn't it just be easier to reject the message altogether
and be done with it?

Yes, spam is a problem. But the solution isn't accepting it and then
trying to figure out what to do with it, and sending it back (most likely
to an innocent party who got joe-jobbed) - it's not accepting it in the
first place. That way, when spam does come in for legitimate addresses,
you have the resources to deal with it (via spamassassin, dspam, or other
means) without impacting the rest of your server as badly.

Josh

Joshua Megerman
SJGames MIB #5273 - OGRE AI Testing Division
You can't win; You can't break even; You can't even quit the game.
- Layman's translation of the Laws of Thermodynamics
josh [at] honorablemenschen


p.a.terry.lists at gmail

Apr 20, 2009, 8:52 AM

Post #7 of 9 (1953 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

> While I understand that rejecting undeliverable
> mail may help spammers weed out the email addresses that are invalid, it
> also drastically cuts down on the total volume of spam processed
>

Fair enough, I'd buy that and will implement it. But this does assume that
you have
a complete list to begin with. There are the cases of customer servers using
ours a
'smart host' and of spammers signing up for accounts and using it for as
long as they
can. For these cases the patch will still prove useful.

Paul


kyle-qmail at memoryhole

Apr 20, 2009, 9:33 AM

Post #8 of 9 (1932 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday, April 20 at 03:35 PM, quoth Paul Terry:
> Giving rejections during an SMTP session provides a means for a
> spammer to determine legitimate addresses.

This is a common complaint, but is generally pointless. Consider what
you're defending against: a vast zombie army of SMTP senders. They
*will* figure out who your valid recipients are, whether you help them
or not. For example, they can create a dummy domain (e.g.
alksdflkaslk.com), and make you send your bounces there. If you send
bounces at all, spammers have the collective ability to force you to
reveal your legitimate addresses. And, since they're using stolen
bandwidth, they don't care how much they use to force that information
out of you... but I suspect you'd be somewhat annoyed if they decided
to do that and waste your bandwidth. (Remember: vast zombie army - you
can't blacklist 'em *all*.)

Rejecting messages during the SMTP connection saves you bandwidth (you
don't have to receive the whole message, and/or you don't have to send
a bounce) and saves you disk space (smaller queue). Plus, it saves
your reputation, as many blacklists (unfortunately) include joe-job
victims. Yes, if you use SMTP-time rejections, spammers CAN use a
dictionary attack to find out what your legitimate addresses are. But
they can do that even without SMTP-time rejections. The only
difference is that SMTP-time rejections save the spammers time and (a
little) bandwidth... which they were probably stealing from someone
*anyway*.

> I think a greater problem is differentiating which bounces are as a
> result of spam and which ones aren't. With this patch, bounces can
> be sent to a dedicated box to address this problem without having to
> worry about the effects that resource intensive code will have on
> the main queue.

Assuming that you CAN determine which ones are the result of spam,
that's reasonable. It seems like a lot of resources to devote to
handling bounces, especially since there's a few simple ways of
reducing your workload, but if it seems reasonable to you... <shrug>
cool. It's certainly a popular thing to use a separate IP address to
send out bounces, to avoid getting your primary server blacklisted by
trigger-happy blacklists.

A few things come to mind as alternatives.

First, your patch seems similar to the BOUNCEQUEUE patch
(http://marc.info/?l=qmail&m=108605073822238&w=2). If I point
BOUNCEQUEUE at a script that uses qmail-remote to forward the message,
I think I've essentially replicated your patch, right?

I think this script should do it...

#!/bin/bash
PATH=/var/qmail/bin:$PATH
BOUNCEHOST=bounces.mydomain.com
# read sender
read -u 1 -d $'\0' sender
# read recipients
i=0
while read -u 1 -d $'\0' recipient ; do
[ -z "$recipient" ] && break
recipients[$i]="$recipient"
i=$(($i+1))
done
# send to bouncehost
/var/qmail/bin/qmail-remote \
$BOUNCEHOST "$sender" "${recipients[@]}" | \
while read -d $'\0' result ; do
case "$result" in
K*) # success
exit 0;;
Z*) # temporary failure
exit 71;;
D*) # permanent failure
exit 31;;
esac
done
exit 71 # could not determine result

Obviously, your script has its own log messages, so that's different,
but the BOUNCEQUEUE patch is a LOT simpler.

Heck, come to think of it, you don't even really need a *patch* to do
this with netqmail. All you need is to point QMAILQUEUE to a script
like the one above, that's been modified to send it off with
qmail-remote if the message sender is null, and use qmail-queue
otherwise. Though at that point, the scripting gets a little more
complicated because of the file-descriptor requirements.

Also... I don't know what anti-spam analysis system you're using, but
if you use SpamAssassin, you can set up spamd on a dedicated host (or
even multiple dedicated hosts) and contact it via spamc over the local
network without needing a separate qmail instance. I have no idea what
the overhead of that would be, and you'd lose the ability to send
bounces from a different IP (which has its own benefits), but it's a
way to offload the resource-intensive code for choosing whether or not
to actually send each bounce.

As far as your particular patch goes... you may want to have a look at
qmail-rspawn.c to compare. For example, if qmail-remote returned
non-zero for some reason---you define exitcode, but don't do anything
with it (why?). That's probably not critical, since qmail-remote is
supposed to ALWAYS return 0, but there's no point in defining exitcode
if you're not going to use it. You're also mishandling the
qmail-remote output; the output CAN be longer than 128 bytes, and if
it is, you may never see the final message report, in which case ret
will be 0, and you'll return an "unknown error from qmail-remote",
even if the message was delivered correctly. Assuming qmail-remote
returns a long output because of some detail of the message and the
bounce server, this will cause a large number of duplicate bounces,
where you never stop re-sending the bounce until it finally expires
from the original queue. Obviously, that's bad. FYI, since you
established the qrs structure with a buffer of 128 chars, the maximum
you can get from a single substdio_get() call is 128 chars, so there's
no point in having buf be larger than 128. And since you don't
*initialize* buf, that means that if the qmail-remote output is more
than 128 chars, not only could you miss the final message report, but
you could potentially walk off into the garbage end of buf. Once
there, if you find a \0K 2-byte sequence, you could incorrectly think
that the message was accepted, even if it was in fact rejected, thus
causing the bounce message to be lost. That's probably bad too.
Granted, in the common case, qmail-remote generates less than 128
characters of output, but...

Anyway, I know I kinda spent this email picking apart your code, but
don't take it badly. You asked for comments! :)

~Kyle
- --
However many holy words you read, However many you speak, what good
will they do you If you do not act on upon them?
-- Buddha
-----BEGIN PGP SIGNATURE-----
Comment: Thank you for using encryption!

iEYEARECAAYFAknso94ACgkQBkIOoMqOI16VVQCfSYWydXGwOpq0BcLza2jLxqS+
sAwAoL97yDTKWVfPj5xGUEQjSfw4gmgb
=mFiD
-----END PGP SIGNATURE-----


p.a.terry.lists at gmail

Apr 20, 2009, 10:40 AM

Post #9 of 9 (1932 views)
Permalink
Re: Patch to redirect bounce messages [In reply to]

> Anyway, I know I kinda spent this email picking apart your code, but
> don't take it badly. You asked for comments! :)
>

Picking apart the code is very much appreciated :)
Much better to find problems here than in production!

Paul

Qmail users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.