Mailing List Archive: exim: users

Greylisting Within Exim Using Memcached

Index | Next | Previous | View Threaded

d.hill at yournetplus

Aug 27, 2009, 9:03 PM

Post #1 of 6 (742 views)
Permalink

Greylisting Within Exim Using Memcached

I have greylisting set up in Exim without the need for anything external.
It doesn't have any allowances for bypassing yet as that is trivial at
this point. It is the last thing I have configured in acl_smtp_rcpt before
the explicit accept. I understand the implementation only allows the
possibility for using one Memcached server. I wouldn't mind if anyone
would like to offer any constructive criticism and/or ways to clean up the
implementation. The configuration can be found here:

http://mail.yournetplus.com/d.hill/exim-greylist-memcached.conf

The two examples I used as a basis in getting it to work were taken from:

http://wiki.exim.org/GreylistMemcachedPerl

which shows a Perl implementation that uses the Perl module
Cache::Memcached, and:

https://secure.grepular.com/blog/index.php/2009/05/13/accessing-memcached-from-exim/

which shows the basics on how to use Memcached within Exim using the
${readsocket{}} string expansion.

-d

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

exim-users-20081202 at djce

Aug 27, 2009, 11:42 PM

Post #2 of 6 (706 views)
Permalink

Re: Greylisting Within Exim Using Memcached [In reply to]

On Fri, Aug 28, 2009 at 04:03:58AM +0000, Duane Hill wrote:
> I have greylisting set up in Exim without the need for anything external.
> It doesn't have any allowances for bypassing yet as that is trivial at
> this point. It is the last thing I have configured in acl_smtp_rcpt before
> the explicit accept. I understand the implementation only allows the
> possibility for using one Memcached server. I wouldn't mind if anyone
> would like to offer any constructive criticism and/or ways to clean up the
> implementation. The configuration can be found here:
>
> http://mail.yournetplus.com/d.hill/exim-greylist-memcached.conf

First impressions...

I'd be wary of using memcached for greylisting. Since there's no
guarantee that anything you put into the cache will come out again, so any
memcached-using app should degrade gracefully if that happens. Arguably,
yours doesn't; if memcached keeps failing to return what was put in, then
you'll defer forever, which you don't want.

That said:

- only works for IPv4, of course
- all that mucking about with octets can probably be made a lot simpler by
using ${mask:
- since you're injecting keys straight into a memcached command string you
should be extra careful about the format of the keys. Currently I think
your keys can contain spaces (i.e. if the sender and/or recipient contain
spaces), which I think would be a Bad Thing.
- is ${if match{$acl_c_memcache_value}{\N^$\N}}
equivalent to ${if eq{$acl_c_memcache_value}{}} ?

Regards,

--
Dave Evans
http://djce.org.uk/
http://djce.org.uk/pgpkey

Attachments:

signature.asc (0.18 KB)

exim-users at lists

Aug 28, 2009, 1:48 AM

Post #3 of 6 (698 views)
Permalink

Re: Greylisting Within Exim Using Memcached [In reply to]

On 28/08/2009 07:42, Dave Evans wrote:

>> I have greylisting set up in Exim without the need for anything external.
>> It doesn't have any allowances for bypassing yet as that is trivial at
>> this point. It is the last thing I have configured in acl_smtp_rcpt before
>> the explicit accept. I understand the implementation only allows the
>> possibility for using one Memcached server. I wouldn't mind if anyone
>> would like to offer any constructive criticism and/or ways to clean up the
>> implementation. The configuration can be found here:
>>
>> http://mail.yournetplus.com/d.hill/exim-greylist-memcached.conf

I'm glad somebody found my ${readsocket} for memcached document useful.

> First impressions...
>
> I'd be wary of using memcached for greylisting. Since there's no
> guarantee that anything you put into the cache will come out again, so any
> memcached-using app should degrade gracefully if that happens. Arguably,
> yours doesn't; if memcached keeps failing to return what was put in, then
> you'll defer forever, which you don't want.

Why would it fail to return what was put in? One thing that I noticed
was that it doesn't check to make sure that the SET command succeeded.
In the document that I wrote it does this:

condition = ${if eq{MEMCACHED_SET}{STORED}}

Yet in the greylist config it does this:

set acl_c_memcache_result = MEMCACHED_SET

But then doesn't check the value of $acl_c_memcache_result anywhere.

> That said:
>
> - only works for IPv4, of course
> - all that mucking about with octets can probably be made a lot simpler by
> using ${mask:

I agree. It would also be more flexible. Rather than specifying the
number of octets you could specify the mask, so instead of 3 octets
you'd specify a mask of 24.

root [at] have:~# exim4 -be '${mask:192.168.10.15/24}'
192.168.10.0/24
root [at] have:~#

set acl_c_memcache_key =
${mask:$sender_host_address/GREYLIST_IPMASK}:$acl_c_from:$acl_c_rcpt

> - since you're injecting keys straight into a memcached command string you
> should be extra careful about the format of the keys. Currently I think
> your keys can contain spaces (i.e. if the sender and/or recipient contain
> spaces), which I think would be a Bad Thing.

Probably safest to hash the value of the key before using it then, ie:

set acl_c_memcache_key =
${md5:${mask:$sender_host_address/GREYLIST_IPMASK}:$acl_c_from:$acl_c_rcpt}

> - is ${if match{$acl_c_memcache_value}{\N^$\N}}
> equivalent to ${if eq{$acl_c_memcache_value}{}} ?

Yeah.

--
Mike Cardwell - IT Consultant and LAMP developer
Cardwell IT Ltd. (UK Reg'd Company #06920226) http://cardwellit.com/

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

exim-users-20081202 at djce

Aug 28, 2009, 2:03 AM

Post #4 of 6 (696 views)
Permalink

Re: Greylisting Within Exim Using Memcached [In reply to]

On Fri, Aug 28, 2009 at 09:48:54AM +0100, Mike Cardwell wrote:
> > I'd be wary of using memcached for greylisting. Since there's no
> > guarantee that anything you put into the cache will come out again, so any
> > memcached-using app should degrade gracefully if that happens. Arguably,
> > yours doesn't; if memcached keeps failing to return what was put in, then
> > you'll defer forever, which you don't want.
>
> Why would it fail to return what was put in?

Cache got full. memcached got restarted. Wind direction changed. That sort
of thing.

--
Dave Evans
http://djce.org.uk/
http://djce.org.uk/pgpkey

Attachments:

signature.asc (0.18 KB)

d.hill at yournetplus

Aug 28, 2009, 3:56 AM

Post #5 of 6 (687 views)
Permalink

Re: Greylisting Within Exim Using Memcached [In reply to]

On Fri, 28 Aug 2009, Dave Evans wrote:

> On Fri, Aug 28, 2009 at 04:03:58AM +0000, Duane Hill wrote:
>> I have greylisting set up in Exim without the need for anything external.
>> It doesn't have any allowances for bypassing yet as that is trivial at
>> this point. It is the last thing I have configured in acl_smtp_rcpt before
>> the explicit accept. I understand the implementation only allows the
>> possibility for using one Memcached server. I wouldn't mind if anyone
>> would like to offer any constructive criticism and/or ways to clean up the
>> implementation. The configuration can be found here:
>>
>> http://mail.yournetplus.com/d.hill/exim-greylist-memcached.conf
>
> First impressions...
>
> I'd be wary of using memcached for greylisting. Since there's no
> guarantee that anything you put into the cache will come out again, so any
> memcached-using app should degrade gracefully if that happens. Arguably,
> yours doesn't; if memcached keeps failing to return what was put in, then
> you'll defer forever, which you don't want.

I understand this.

> That said:
>
> - only works for IPv4, of course
> - all that mucking about with octets can probably be made a lot simpler by
> using ${mask:

Change has been made. That cleaned things up quite a bit.

> - since you're injecting keys straight into a memcached command string you
> should be extra careful about the format of the keys. Currently I think
> your keys can contain spaces (i.e. if the sender and/or recipient contain
> spaces), which I think would be a Bad Thing.

Why would the sender and/or recipient contain spaces? Shouldn't have Exim
dealt with this by now?

> - is ${if match{$acl_c_memcache_value}{\N^$\N}}
> equivalent to ${if eq{$acl_c_memcache_value}{}} ?

I guess it is. Change has been made.

Thanks for the overall input.

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

d.hill at yournetplus

Aug 28, 2009, 4:06 AM

Post #6 of 6 (685 views)
Permalink

Re: Greylisting Within Exim Using Memcached [In reply to]

On Fri, 28 Aug 2009, Mike Cardwell wrote:

> On 28/08/2009 07:42, Dave Evans wrote:
>
>>> I have greylisting set up in Exim without the need for anything external.
>>> It doesn't have any allowances for bypassing yet as that is trivial at
>>> this point. It is the last thing I have configured in acl_smtp_rcpt before
>>> the explicit accept. I understand the implementation only allows the
>>> possibility for using one Memcached server. I wouldn't mind if anyone
>>> would like to offer any constructive criticism and/or ways to clean up the
>>> implementation. The configuration can be found here:
>>>
>>> http://mail.yournetplus.com/d.hill/exim-greylist-memcached.conf
>
> I'm glad somebody found my ${readsocket} for memcached document useful.
>
>> First impressions...
>>
>> I'd be wary of using memcached for greylisting. Since there's no
>> guarantee that anything you put into the cache will come out again, so any
>> memcached-using app should degrade gracefully if that happens. Arguably,
>> yours doesn't; if memcached keeps failing to return what was put in, then
>> you'll defer forever, which you don't want.
>
> Why would it fail to return what was put in? One thing that I noticed
> was that it doesn't check to make sure that the SET command succeeded.
> In the document that I wrote it does this:
>
> condition = ${if eq{MEMCACHED_SET}{STORED}}

I have changed the set acl variable back to a condition. I was going to do
that and it slipped my mind. Now, if there is an error in storing. a
sender host will not get arbitrarily greylisted.

> Yet in the greylist config it does this:
>
> set acl_c_memcache_result = MEMCACHED_SET
>
> But then doesn't check the value of $acl_c_memcache_result anywhere.
>
>> That said:
>>
>> - only works for IPv4, of course
>> - all that mucking about with octets can probably be made a lot simpler by
>> using ${mask:
>
> I agree. It would also be more flexible. Rather than specifying the
> number of octets you could specify the mask, so instead of 3 octets
> you'd specify a mask of 24.
>
> root [at] have:~# exim4 -be '${mask:192.168.10.15/24}'
> 192.168.10.0/24
> root [at] have:~#
>
> set acl_c_memcache_key =
> ${mask:$sender_host_address/GREYLIST_IPMASK}:$acl_c_from:$acl_c_rcpt
>
>> - since you're injecting keys straight into a memcached command string you
>> should be extra careful about the format of the keys. Currently I think
>> your keys can contain spaces (i.e. if the sender and/or recipient contain
>> spaces), which I think would be a Bad Thing.
>
> Probably safest to hash the value of the key before using it then, ie:
>
>
> set acl_c_memcache_key =
> ${md5:${mask:$sender_host_address/GREYLIST_IPMASK}:$acl_c_from:$acl_c_rcpt}

Ok. It now uses an MD5 hash.

>> - is ${if match{$acl_c_memcache_value}{\N^$\N}}
>> equivalent to ${if eq{$acl_c_memcache_value}{}} ?
>
> Yeah.

Thanks for all the input.

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads