Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: exim: users

reactive filtering to log files with exim possible?

 

 

exim users RSS feed   Index | Next | Previous | View Threaded


jeffrey.starin at gmail

Mar 6, 2012, 8:03 AM

Post #1 of 18 (2103 views)
Permalink
reactive filtering to log files with exim possible?

Can Exim be configured to react to log files?

There is talk out there in the internet about other MTAs like Port25
which has the ability to react in real time to log files such that if a
log entry contains a Yahoo 421 message, either switch IPs or otherwise
shut down the queue.

This sounds like a great idea and I'm wondering if Exim has the muscle
to do this?

thanks.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


warren at decoy

Mar 6, 2012, 9:26 AM

Post #2 of 18 (2062 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Tue, Mar 6, 2012 at 6:03 PM, Jeffrey Starin <jeffrey.starin [at] gmail>wrote:

> Can Exim be configured to react to log files?
>
> There is talk out there in the internet about other MTAs like Port25 which
> has the ability to react in real time to log files such that if a log entry
> contains a Yahoo 421 message, either switch IPs or otherwise shut down the
> queue.
>
> This sounds like a great idea and I'm wondering if Exim has the muscle to
> do this?
>


With a bit of external help you could get Exim to do the same thing. Have a
log file monitor that 'watches' (tails) the log file for a specific pattern
and when it encounters a pattern, have it execute Exim with an alternate
configuration file for the relevant message(s).

--
.warren
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


jeffrey.starin at gmail

Mar 6, 2012, 2:15 PM

Post #3 of 18 (2065 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 3/6/2012 12:26 PM, Warren Baker wrote:
> On Tue, Mar 6, 2012 at 6:03 PM, Jeffrey Starin
> <jeffrey.starin [at] gmail <mailto:jeffrey.starin [at] gmail>> wrote:
>
> Can Exim be configured to react to log files?
>
> There is talk out there in the internet about other MTAs like
> Port25 which has the ability to react in real time to log files
> such that if a log entry contains a Yahoo 421 message, either
> switch IPs or otherwise shut down the queue.
>
> This sounds like a great idea and I'm wondering if Exim has the
> muscle to do this?
>
>
>
> With a bit of external help you could get Exim to do the same thing.
> Have a log file monitor that 'watches' (tails) the log file for a
> specific pattern and when it encounters a pattern, have it execute
> Exim with an alternate configuration file for the relevant message(s).
>
> --
> .warren
thank you Warren. Do you think simply "kill -HUP `cat
/var/spool/exim/exim-daemon.pid`" then restarting exim with reference to
a separate config file would suffice?

What happens to the messages already in the queues and what kind of
hiccup could we expect?

Thank you
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


warren at decoy

Mar 7, 2012, 12:09 AM

Post #4 of 18 (2061 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Wed, Mar 7, 2012 at 12:15 AM, Jeffrey Starin <jeffrey.starin [at] gmail>wrote:

> On 3/6/2012 12:26 PM, Warren Baker wrote:
>
>>
>> With a bit of external help you could get Exim to do the same thing. Have
>> a log file monitor that 'watches' (tails) the log file for a specific
>> pattern and when it encounters a pattern, have it execute Exim with an
>> alternate configuration file for the relevant message(s).
>>
>> thank you Warren. Do you think simply "kill -HUP `cat
> /var/spool/exim/exim-daemon.**pid`" then restarting exim with reference
> to a separate config file would suffice?
>
> What happens to the messages already in the queues and what kind of hiccup
> could we expect?
>
> Thank you



You can run a separate instance of Exim with an alternate config file which
could be configured to only handle the delayed Yahoo messages. This way,
the separate instance wouldn't touch the other mails in the queue and you
don't need to HUP anything.

There are a number of ways you could do this, for example have the external
helper freeze the message and then have the other instance unthaw and then
attempt to deliver it or based upon some file existing (the is created by
the external helper) have the secondary exim instance deliver the message.
The primary exim instance wouldn't touch the messages destined for yahoo,
until the file was removed.

There are a number of ways to do this but just about anything possible.


--
.warren
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


exim at u61

Mar 7, 2012, 5:31 AM

Post #5 of 18 (2077 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Wed, 2012-03-07 at 10:09 +0200, Warren Baker wrote:

> You can run a separate instance of Exim with an alternate config file which
> could be configured to only handle the delayed Yahoo messages.

How does one run a second parallel version of Exim ?

Copy and rename the binary ?

Copy and rename /etc/exim ?

All ideas appreciated.


--
Paul.
England,
EU.



--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


jeffrey.starin at gmail

Mar 7, 2012, 5:39 AM

Post #6 of 18 (2062 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 3/7/2012 3:09 AM, Warren Baker wrote:
>
> On Wed, Mar 7, 2012 at 12:15 AM, Jeffrey Starin
> <jeffrey.starin [at] gmail <mailto:jeffrey.starin [at] gmail>> wrote:
>
> On 3/6/2012 12:26 PM, Warren Baker wrote:
>
>
> With a bit of external help you could get Exim to do the same
> thing. Have a log file monitor that 'watches' (tails) the log
> file for a specific pattern and when it encounters a pattern,
> have it execute Exim with an alternate configuration file for
> the relevant message(s).
>
> thank you Warren. Do you think simply "kill -HUP `cat
> /var/spool/exim/exim-daemon.pid`" then restarting exim with
> reference to a separate config file would suffice?
>
> What happens to the messages already in the queues and what kind
> of hiccup could we expect?
>
> Thank you
>
>
>
> You can run a separate instance of Exim with an alternate config file
> which could be configured to only handle the delayed Yahoo messages.
> This way, the separate instance wouldn't touch the other mails in the
> queue and you don't need to HUP anything.
>
> There are a number of ways you could do this, for example have the
> external helper freeze the message and then have the other instance
> unthaw and then attempt to deliver it or based upon some file existing
> (the is created by the external helper) have the secondary exim
> instance deliver the message. The primary exim instance wouldn't touch
> the messages destined for yahoo, until the file was removed.
>
> There are a number of ways to do this but just about anything possible.
>
>
> --
> .warren
A little insight as to what is an external helper? Thanks.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


warren at decoy

Mar 7, 2012, 5:41 AM

Post #7 of 18 (2062 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Wed, Mar 7, 2012 at 3:31 PM, Always Learning <exim [at] u61> wrote:

>
>
> How does one run a second parallel version of Exim ?
>
> Copy and rename the binary ?
>
> Copy and rename /etc/exim ?
>
> All ideas appreciated.



The -C option

You can have the binary point to another config file which has all the
relevant options to enable you to run an alternative setup.


--
.warren
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


warren at decoy

Mar 7, 2012, 7:27 AM

Post #8 of 18 (2060 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Wed, Mar 7, 2012 at 3:39 PM, Jeffrey Starin <jeffrey.starin [at] gmail>wrote:

> A little insight as to what is an external helper? Thanks.
>
>
That is a little beyond the scope of this list but something that tails the
log file (a perl script as an example) and reacts when needed.


thx

--
.warren
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


jeffrey.starin at gmail

Mar 7, 2012, 7:48 AM

Post #9 of 18 (2061 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 3/7/2012 10:27 AM, Warren Baker wrote:
>
> On Wed, Mar 7, 2012 at 3:39 PM, Jeffrey Starin
> <jeffrey.starin [at] gmail <mailto:jeffrey.starin [at] gmail>> wrote:
>
> A little insight as to what is an external helper? Thanks.
>
>
> That is a little beyond the scope of this list but something that
> tails the log file (a perl script as an example) and reacts when needed.
>
>
> thx
>
> --
> .warren
I guess I'm asking, which presumably would be in the scope of the list,
is what exim command(s) would reload a different configuration file
without hiccuping.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


graeme at graemef

Mar 7, 2012, 9:05 AM

Post #10 of 18 (2058 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Tue, 2012-03-06 at 11:03 -0500, Jeffrey Starin wrote:
> Can Exim be configured to react to log files?

Directly? Not that I am aware of.

> There is talk out there in the internet about other MTAs like Port25
> which has the ability to react in real time to log files such that if a
> log entry contains a Yahoo 421 message, either switch IPs or otherwise
> shut down the queue.
>
> This sounds like a great idea and I'm wondering if Exim has the muscle
> to do this?

You can do this with retry rules like the following example:

yahoo.com rcpt_421 F,24h,6h

That is - for all messages for recipients in the yahoo.com domain which
receive a 421 response to a RCPT command, park them in the queue for 24
hours and only try every 6 hours.

I highly recommend familiarising yourself with the retry configuration:

http://www.exim.org/exim-html-current/doc/html/spec_html/ch32.html

It will very likely do exactly what you need without any external
"assistance".

Graeme


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


jeffrey.starin at gmail

Mar 7, 2012, 9:18 AM

Post #11 of 18 (2059 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

thank you! Perfecto!

On 3/7/2012 12:05 PM, Graeme Fowler wrote:
> On Tue, 2012-03-06 at 11:03 -0500, Jeffrey Starin wrote:
>> Can Exim be configured to react to log files?
> Directly? Not that I am aware of.
>
>> There is talk out there in the internet about other MTAs like Port25
>> which has the ability to react in real time to log files such that if a
>> log entry contains a Yahoo 421 message, either switch IPs or otherwise
>> shut down the queue.
>>
>> This sounds like a great idea and I'm wondering if Exim has the muscle
>> to do this?
> You can do this with retry rules like the following example:
>
> yahoo.com rcpt_421 F,24h,6h
>
> That is - for all messages for recipients in the yahoo.com domain which
> receive a 421 response to a RCPT command, park them in the queue for 24
> hours and only try every 6 hours.
>
> I highly recommend familiarising yourself with the retry configuration:
>
> http://www.exim.org/exim-html-current/doc/html/spec_html/ch32.html
>
> It will very likely do exactly what you need without any external
> "assistance".
>
> Graeme
>
>


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


jeffrey.starin at gmail

Mar 7, 2012, 10:01 AM

Post #12 of 18 (2060 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

As the documentation does not specify what section of the exim.conf to
put these parameters in, where does one do so?

Thank you

On 3/7/2012 12:05 PM, Graeme Fowler wrote:
> On Tue, 2012-03-06 at 11:03 -0500, Jeffrey Starin wrote:
>> Can Exim be configured to react to log files?
> Directly? Not that I am aware of.
>
>> There is talk out there in the internet about other MTAs like Port25
>> which has the ability to react in real time to log files such that if a
>> log entry contains a Yahoo 421 message, either switch IPs or otherwise
>> shut down the queue.
>>
>> This sounds like a great idea and I'm wondering if Exim has the muscle
>> to do this?
> You can do this with retry rules like the following example:
>
> yahoo.com rcpt_421 F,24h,6h
>
> That is - for all messages for recipients in the yahoo.com domain which
> receive a 421 response to a RCPT command, park them in the queue for 24
> hours and only try every 6 hours.
>
> I highly recommend familiarising yourself with the retry configuration:
>
> http://www.exim.org/exim-html-current/doc/html/spec_html/ch32.html
>
> It will very likely do exactly what you need without any external
> "assistance".
>
> Graeme
>
>


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


jeffrey.starin at gmail

Mar 7, 2012, 10:08 AM

Post #13 of 18 (2060 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 3/7/2012 12:05 PM, Graeme Fowler wrote:
> On Tue, 2012-03-06 at 11:03 -0500, Jeffrey Starin wrote:
>> Can Exim be configured to react to log files?
> Directly? Not that I am aware of.
>
>> There is talk out there in the internet about other MTAs like Port25
>> which has the ability to react in real time to log files such that if a
>> log entry contains a Yahoo 421 message, either switch IPs or otherwise
>> shut down the queue.
>>
>> This sounds like a great idea and I'm wondering if Exim has the muscle
>> to do this?
> You can do this with retry rules like the following example:
>
> yahoo.com rcpt_421 F,24h,6h
>
> That is - for all messages for recipients in the yahoo.com domain which
> receive a 421 response to a RCPT command, park them in the queue for 24
> hours and only try every 6 hours.
>
> I highly recommend familiarising yourself with the retry configuration:
>
> http://www.exim.org/exim-html-current/doc/html/spec_html/ch32.html
>
> It will very likely do exactly what you need without any external
> "assistance".
>
> Graeme
>
>
My apologies: this information is contained in the "retry" section of
exim.con

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


warren at decoy

Mar 7, 2012, 10:12 AM

Post #14 of 18 (2063 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Wed, Mar 7, 2012 at 8:08 PM, Jeffrey Starin <jeffrey.starin [at] gmail>wrote:

> My apologies: this information is contained in the "retry" section of
> exim.con
>

Correct. I would suggest reading
http://www.exim.org/exim-html-current/doc/html/spec_html/ch32.html for the
retry rules format and the various options that are available.




--
.warren
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


warren at decoy

Mar 7, 2012, 10:16 AM

Post #15 of 18 (2065 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On Wed, Mar 7, 2012 at 5:48 PM, Jeffrey Starin <jeffrey.starin [at] gmail>wrote:

> I guess I'm asking, which presumably would be in the scope of the list, is
> what exim command(s) would reload a different configuration file without
> hiccuping.
>
>

You can create a separate configuration file with alternate settings and
have exim use that file by making use of the -C option (exim -C configfile).
Have a look at
http://www.exim.org/exim-html-current/doc/html/spec_html/ch05.html for
command line options.


--
.warren
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


iane at sussex

Mar 12, 2012, 5:33 AM

Post #16 of 18 (1987 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 6 Mar 2012, at 16:03, Jeffrey Starin wrote:
> Can Exim be configured to react to log files?
>
> There is talk out there in the internet about other MTAs like Port25 which has the ability to react in real time to log files such that if a log entry contains a Yahoo 421 message, either switch IPs or otherwise shut down the queue.
>
> This sounds like a great idea and I'm wondering if Exim has the muscle to do this?
>
> thanks.


I guess that what would be nice here would be to let Exim examine the reply to each SMTP command, and then to act on it post-delivery. Something like a post-delivery ACL.


--
Ian Eiloart
Postmaster, University of Sussex
+44 (0) 1273 87-3148



--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


iane at sussex

Mar 12, 2012, 5:41 AM

Post #17 of 18 (2004 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 7 Mar 2012, at 13:31, Always Learning wrote:
> On Wed, 2012-03-07 at 10:09 +0200, Warren Baker wrote:
>
>> You can run a separate instance of Exim with an alternate config file which
>> could be configured to only handle the delayed Yahoo messages.
>
> How does one run a second parallel version of Exim ?
>
> Copy and rename the binary ?

No, the binary doesn't need to be different. You just launch Exim with different command line options. For example, you might do this to ask Exim to listen for connections on a port. Or you might ask Exim to attempt to deliver mail from the queue. Or you might ask Exim to check the validity of a configuration file.

Among those options, you can specify a different configuration file. For example, you might have one configuration listening on port 25 on your advertised MX IP address, and another listening on port 587 for message submissions. Or you might be pointing to a new configuration file that you want to test before deploying.

> Copy and rename /etc/exim ?

If that's your configuration file, there's no point having two identical copies. But, we use separate configurations for local submissions and for inbound email.

> All ideas appreciated.



--
Ian Eiloart
Postmaster, University of Sussex
+44 (0) 1273 87-3148



--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


iane at sussex

Mar 12, 2012, 6:03 AM

Post #18 of 18 (1991 views)
Permalink
Re: reactive filtering to log files with exim possible? [In reply to]

On 7 Mar 2012, at 15:48, Jeffrey Starin wrote:
> On 3/7/2012 10:27 AM, Warren Baker wrote:
>>
>> On Wed, Mar 7, 2012 at 3:39 PM, Jeffrey Starin <jeffrey.starin [at] gmail <mailto:jeffrey.starin [at] gmail>> wrote:
>>
>> A little insight as to what is an external helper? Thanks.
>>
>>
>> That is a little beyond the scope of this list but something that tails the log file (a perl script as an example) and reacts when needed.
>>
>>
>> thx
>>
>> --
>> .warren
> I guess I'm asking, which presumably would be in the scope of the list, is what exim command(s) would reload a different configuration file without hiccuping.


I think this is all in the scope of the list, which is intended to assist Exim users with any problems relating to persuading Exim to behave as required. The solution probably does require more than a clever configuration file, though.

In this case, the question is how can we persuade Exim to behave intelligently with respect to previous SMTP replies from Yahoo. Specifically, certain replies from Yahoo might require us to (a) back off with delivery attempts, or (b) to use a different IP address for delivery.

Response (a) is what Yahoo are after. It might be possible to do that better with a specific line in the RETRY section of the configuration file. Section 32 of the docs might give you some ideas. For example, can configure a retry strategy based on recipient domain and return code number (sec 32.5).

Response (b) is circumventing Yahoo's anti-spam protection, and it's somewhat harder to achieve. The easy bit is setting up the smtp transport. You need to set the "interface" option, which is expanded at delivery time, so it can be looked up from a file (or ldap server, or whatever), if the recipient domain is yahoo.*, for example. The tricky bit is ensuring that the file has the right content. For that, you probably need to write a script that watches your log file and flips the content of the file when necessary.

Or, you could have an ACL that uses $run to trigger the script when it sees a Yahoo recipient, but that could be very inefficient if, well if you're sending enough mail to Yahoo that you're seeing this problem!

--
Ian Eiloart
Postmaster, University of Sussex
+44 (0) 1273 87-3148



--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

exim users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.