Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: exim: users

greylisting

 

 

First page Previous page 1 2 Next page Last page  View All exim users RSS feed   Index | Next | Previous | View Threaded


cjackson at localsurface

Jun 15, 2005, 2:08 PM

Post #26 of 48 (2458 views)
Permalink
Re: greylisting [In reply to]

David Woodhouse wrote:
> On Wed, 2005-06-15 at 18:43 +0200, Steffen Heil wrote:
>
>>I don't want to take this into account, since I do greylisting at RCPT
>>time.
>
>
> But that presumably means you're doing greylisting unconditionally. It
> makes a lot more sense only to do it for mail which is actually
> considered suspicious for some reason.
>
> Otherwise you just end up delaying a lot of good mail for no reason.
>

But my impression is that one of the prime reasons for greylisting is to
avoid the high cost of virus and spamscanning. According to my logs,
greylisting stops a huge percentage of spam before the recipient check.
If mail is spam-scanned then greylisted, is it spam-scanned again when
it is resent?
Craig


nipsy at bitgnome

Jun 15, 2005, 2:16 PM

Post #27 of 48 (2454 views)
Permalink
Re: greylisting [In reply to]

On 15 Jun 2005, David Woodhouse wrote:
> On Wed, 2005-06-15 at 18:43 +0200, Steffen Heil wrote:
> > I don't want to take this into account, since I do greylisting at RCPT
> > time.
>
> But that presumably means you're doing greylisting unconditionally. It
> makes a lot more sense only to do it for mail which is actually
> considered suspicious for some reason.
>
> Otherwise you just end up delaying a lot of good mail for no reason.

Isn't that really kind of the whole point? If the mail
is legitimate (ideally), the remote side will resend in a bit and
the tuplet will be recognized as valid at that point for some set
period of time (few weeks to months normally).

I know at the university where I work we use greylisting
universally on our external mail relays. We have the occasional
issue with remote SMTP hosts which are simply broken and we
either get them to fix their junk servers or make an exception if
all else fails. This seems to work pretty well.

Outside of the initial delay for the first message, any
regular e-mail traffic will flow normally as it should. You have
to keep an eye on known mailing list hosts of course whenever
they change the envelope sender for each message sent from a list
and just add the exception as needed.

--
Mark Nipper e-contacts:
4475 Carter Creek Parkway nipsy [at] bitgnome
Apartment 724 http://nipsy.bitgnome.net/
Bryan, Texas, 77802-4481 AIM/Yahoo: texasnipsy ICQ: 66971617
(979)575-3193 MSN: nipsy [at] tamu

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
"Give us this day our daily Faith, but deliver us, dear God, from
Belief."

-- Aldous Huxley, _Island_, 1962
----end random quote of the moment----


kjetil at kjernsmo

Jun 15, 2005, 2:41 PM

Post #28 of 48 (2456 views)
Permalink
Re: greylisting [In reply to]

On onsdag 15 juni 2005, 23:16, Mark Nipper wrote:
> On 15 Jun 2005, David Woodhouse wrote:
> > On Wed, 2005-06-15 at 18:43 +0200, Steffen Heil wrote:
> > > I don't want to take this into account, since I do greylisting at
> > > RCPT time.
> >
> > But that presumably means you're doing greylisting unconditionally.
> > It makes a lot more sense only to do it for mail which is actually
> > considered suspicious for some reason.
> >
> > Otherwise you just end up delaying a lot of good mail for no
> > reason.
>
>         Isn't that really kind of the whole point?  If the mail
> is legitimate (ideally), the remote side will resend in a bit and
> the tuplet will be recognized as valid at that point for some set
> period of time (few weeks to months normally).

Yup, but I personally has had ambitions to do a bit of whitelisting in
RCPT TO, based on FOAF social networks for example, or hook it into a
CRM system, if is a commercial setting. I think it would help a bit...

Cheers,

Kjetil
--
Kjetil Kjernsmo
Programmer/Astrophysicist/Skeptic/Ski-orienteer/Orienteer/Mountaineer
kjetil [at] kjernsmo webmaster [at] skepsis editor [at] learn-orienteering
Homepage: http://www.kjetil.kjernsmo.net/ OpenPGP KeyID: 6A6A0BBC


dwmw2 at infradead

Jun 15, 2005, 3:09 PM

Post #29 of 48 (2458 views)
Permalink
Re: greylisting [In reply to]

On Wed, 2005-06-15 at 16:08 -0500, Craig Jackson wrote:
> But my impression is that one of the prime reasons for greylisting is to
> avoid the high cost of virus and spamscanning. According to my logs,
> greylisting stops a huge percentage of spam before the recipient check.
> If mail is spam-scanned then greylisted, is it spam-scanned again when
> it is resent?

In my case, yes -- it's not used as a tactic for load reduction. If
that's what you want then I suppose it does make some sense to do it
before DATA, yes.

--
dwmw2


baloo at ursine

Jun 15, 2005, 6:13 PM

Post #30 of 48 (2458 views)
Permalink
Re: greylisting [In reply to]

On Wednesday June 15 2005 2:16 pm, Mark Nipper wrote:
> On 15 Jun 2005, David Woodhouse wrote:
> > On Wed, 2005-06-15 at 18:43 +0200, Steffen Heil wrote:
> > > I don't want to take this into account, since I do greylisting
> > > at RCPT time.
> >
> > But that presumably means you're doing greylisting
> > unconditionally. It makes a lot more sense only to do it for mail
> > which is actually considered suspicious for some reason.
> >
> > Otherwise you just end up delaying a lot of good mail for no
> > reason.
>
> Isn't that really kind of the whole point? If the mail
> is legitimate (ideally), the remote side will resend in a bit and
> the tuplet will be recognized as valid at that point for some set
> period of time (few weeks to months normally).

That's more or less Debian's idea of it. greylistd (which provides
greylisting for exim4 in Debian) has a default greylist interval of
60 minutes and saves tuplets for 2 weeks. It works out really well
after a day or two.

> Outside of the initial delay for the first message, any
> regular e-mail traffic will flow normally as it should.

It does at ursine.ca, so there's at least a small case-in-point.

> You have to keep an eye on known mailing list hosts of course
> whenever they change the envelope sender for each message sent from
> a list and just add the exception as needed.

I haven't had a problem with mailing list hosts, they get whitelisted
by greylistd just fine on their own.

--
Paul Johnson
Email and Instant Messenger (Jabber): baloo [at] ursine
http://ursine.ca/~baloo/


nipsy at bitgnome

Jun 15, 2005, 10:21 PM

Post #31 of 48 (2457 views)
Permalink
Re: greylisting [In reply to]

On 15 Jun 2005, Paul Johnson wrote:
> On Wednesday June 15 2005 2:16 pm, Mark Nipper wrote:
> > Isn't that really kind of the whole point? If the mail
> > is legitimate (ideally), the remote side will resend in a bit and
> > the tuplet will be recognized as valid at that point for some set
> > period of time (few weeks to months normally).
>
> That's more or less Debian's idea of it. greylistd (which provides
> greylisting for exim4 in Debian) has a default greylist interval of
> 60 minutes and saves tuplets for 2 weeks. It works out really well
> after a day or two.

Oddly enough, the exact setup I run myself! :) I did
have a few people complain though about the initial delay so I
went ahead and scaled it down to 10 minutes instead of the
default 1 hour. I'm sure a few more things get through at this
setting simply because people haven't had time to report junk to
the various RBL's I use also.

>
> > Outside of the initial delay for the first message, any
> > regular e-mail traffic will flow normally as it should.
>
> It does at ursine.ca, so there's at least a small case-in-point.

Same here. No complaints after the first couple of weeks
of operation. People probably don't even realize it is happening
anymore at this point other than the reduced volume of junk.

> > You have to keep an eye on known mailing list hosts of course
> > whenever they change the envelope sender for each message sent from
> > a list and just add the exception as needed.
>
> I haven't had a problem with mailing list hosts, they get whitelisted
> by greylistd just fine on their own.

I've had to add three different list hosts so far:
---
66.35.250.225 # lists-outbound.sourceforge.net
212.16.7.65 # thebsh.namesys.com
63.251.223.186 # lists.developer.com

simply because all of these operate by generating a unique
envelope sender for each outbound message to subscribers. It's
kind of annoying really that lists should do this. I'm also on
the LKML which does NOT do this and is remarkably spam free
given its age and public availability. But I assume the unique
envelope senders help out with bounces and such. Still makes it
annoying with regards to greylisting though.

--
Mark Nipper e-contacts:
4475 Carter Creek Parkway nipsy [at] bitgnome
Apartment 724 http://nipsy.bitgnome.net/
Bryan, Texas, 77802-4481 AIM/Yahoo: texasnipsy ICQ: 66971617
(979)575-3193 MSN: nipsy [at] tamu

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
"Never underestimate the bandwidth of a station wagon filled with
magtape, or a 747 filled with CD-ROMs."
-- from the Jargon File's definition of sneakernet
----end random quote of the moment----


kevin.peuhkurinen at meridiancu

Jun 16, 2005, 6:49 AM

Post #32 of 48 (2459 views)
Permalink
Re: Greylisting [In reply to]

Sorry for having no In-Reply-To header. Responding to digest.

I've been using greylisting for a few months now, using the
implementation listed here:
http://slett.net/spam-filtering-for-mx/exim-greylisting.html#exim-greylist-mysql

I greylist emails that fall into any of these categories:

1. The sending host gave an IP address in it's greeting rather than a
fully qualified domain name.
2. The sending host gave an FQDN in its greeting that pretended to be
from one of my domains
3. A reverse DNS lookup on the sending host's IP address failed
4. The sending host's IP address is in a range allocated to APNIC or LACNIC

My time settings are 5MIN / 12HOUR, / 36DAY.

The results have been pretty spectacular. Less than 5% of the email
greylisted is retried. The remainder represents more than half of the
spam I recieve, which means that SA has to work 50% less than it
otherwise would.

I'm contemplating adding a fifth condition for greylisting which would
be if the sending host is listed in dul.dnsbl.sorbs.net.


pookey at pookey

Jun 6, 2009, 9:47 AM

Post #33 of 48 (2456 views)
Permalink
Re: greylisting [In reply to]

2009/6/6 Terry <terry [at] bluelight>:
> Thanks for the reply do you use grey listing ? I would be very reluctant to
> drop it

I use it selectively... if a host looks naughty, then I do. no rDNS,
silly HELO's, host in XBL.. that kinda thing. I keep an ACL variable
for a score, then only greylist based on if that score is greater than
X.

I think greylisting should be avoided where possible, as it will lead
to legitimate mail being bounced, lost, or delayed - and no one wants
any of that :)


--
Blog: http://pookey.co.uk/blog
Follow me on twitter: http://twitter.com/ipchristian

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


wbh at conducive

Jun 11, 2009, 9:53 AM

Post #34 of 48 (2445 views)
Permalink
Re: greylisting [In reply to]

Ian P. Christian wrote:
> 2009/6/8 David Woodhouse <dwmw2 [at] infradead>:
>> I'd suggest reading http://wiki.exim.org/SimpleGreylisting -- the prose
>> sets out some things that you may want to think about regardless of
>> which greylisting implementation you use, and then there's an example
>> Exim configuration which shouldn't suffer most of the stupid problems
>> that postgrey does.
>
> There's actually a flaw in this implementation here.
>
> # Generate a hashed 'identity' for the mail, as described above.
> warn set acl_m_greyident =
> ${hash{20}{62}{$sender_address$recipients$h_message-id:}}
>
> Because it's common at the moment to get a mail to someone sent from
> their own address without a message ID, hash clashes occour.
>
> I'm currently not sure of the best way to d eal with this - perahps
> adding the Subject line into the hash...
>
> Perhaps I should just block mail sent from someone, to themselves,
> with a null message ID.
>

Got me confused....

- If the message is 'outbound' it has not (and will not) see *your* greylisting,

- but 'should' get a message_ID generated and added by the MTA if there was none
(to self or otherwise). Simple option, 'owed' to our fellow MTA admins anyway,
and 'kinder' to Em Mess Dose hag-ridden et al than blocking.

.. or at least less hassle than arguing with 'em about 'upgrading' or 'standards'.

;-)

Bill



--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


pookey at pookey

Jun 11, 2009, 10:58 AM

Post #35 of 48 (2443 views)
Permalink
Re: greylisting [In reply to]

2009/6/11 W B Hacker <wbh [at] conducive>:
> Got me confused....

turns out I'm probably confused. I'm getting duplicate insertion
errors from something.. I thought that explained it, but actually, it
later turned out it didn't.

There's something wrong with the implementation, or my implementation
of that implementation ;)

I'll past back when (if) I figure out what it is, assuming it's not
just me being stupid!

--
Blog: http://pookey.co.uk/blog
Follow me on twitter: http://twitter.com/ipchristian

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


wbh at conducive

Jun 11, 2009, 11:18 AM

Post #36 of 48 (2443 views)
Permalink
Re: greylisting [In reply to]

Ian P. Christian wrote:
> 2009/6/11 W B Hacker <wbh [at] conducive>:
>> Got me confused....
>
> turns out I'm probably confused. I'm getting duplicate insertion
> errors from something.. I thought that explained it, but actually, it
> later turned out it didn't.
>
> There's something wrong with the implementation, or my implementation
> of that implementation ;)
>
> I'll past back when (if) I figure out what it is, assuming it's not
> just me being stupid!
>

LOL!

Another of Exim's great strengths .... allowing us to ... ah

'express our creativity'.

BTDTGTTSWBH

Bill

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dwmw2 at infradead

Jun 27, 2009, 2:18 AM

Post #37 of 48 (2329 views)
Permalink
Re: greylisting [In reply to]

On Fri, 2009-06-12 at 10:38 -0300, Eduardo M KALINOWSKI wrote:
> One thing that struck me is: once a greylisted message is seen again
> and accepted (because the delay is over), couldn't its entry be
> removed from the greylist table?
...
> What do you guys think? Is it worth it? Or it is better to leave
> old entries (retried or not) to be bulk deleted from cron?

Please don't drop me from Cc when replying to me. It's quite impolite.

I did think about removing entries when we accept mail -- but since we
still need to run the cron job _anyway_ to clean up after the messages
which _weren't_ retried, I didn't really see the point.

It's just more complexity without any real benefit -- the size of the
greylist db isn't really much of an issue in practice, is it?

--
David Woodhouse Open Source Technology Centre
David.Woodhouse [at] intel Intel Corporation


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dwmw2 at infradead

Jun 27, 2009, 2:32 AM

Post #38 of 48 (2331 views)
Permalink
Re: greylisting [In reply to]

On Thu, 2009-06-11 at 13:41 +0100, Ian P. Christian wrote:
> 2009/6/8 David Woodhouse <dwmw2 [at] infradead>:
> > I'd suggest reading http://wiki.exim.org/SimpleGreylisting -- the prose
> > sets out some things that you may want to think about regardless of
> > which greylisting implementation you use, and then there's an example
> > Exim configuration which shouldn't suffer most of the stupid problems
> > that postgrey does.
>
> There's actually a flaw in this implementation here.

Er, thanks for dropping me from Cc when you criticize my work...! :)

> # Generate a hashed 'identity' for the mail, as described above.
> warn set acl_m_greyident =
> ${hash{20}{62}{$sender_address$recipients$h_message-id:}}
>
> Because it's common at the moment to get a mail to someone sent from
> their own address without a message ID, hash clashes occour.

Yeah, at the time I first implemented this I was just rejecting all mail
without a Message-Id, so it wasn't much of an issue.

> I'm currently not sure of the best way to deal with this - perahps
> adding the Subject line into the hash...

That seems like it would be a reasonable thing to do. Is it enough,
though? A lot of spam messages have the same subject line too.

What else could we include -- bearing in mind that we have to be sure
that it _won't_ get changed by the sending MTA between retry attempts. I
suppose we could use the full From:, To: and Cc: headers -- and maybe
also the Date: header?

> Perhaps I should just block mail sent from someone, to themselves,
> with a null message ID.

You could use PRVS and just reject _all_ mail which is faked to appear
as if it's from your own addresses, surely?

--
David Woodhouse Open Source Technology Centre
David.Woodhouse [at] intel Intel Corporation


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dms at samersoff

Jun 27, 2009, 6:28 AM

Post #39 of 48 (2326 views)
Permalink
Re: greylisting [In reply to]

David,

I think SQL for graylist is totally overkill.

Check this implementation:
http://www.beastsoft.net/cgi-bin/hg/hgwebdir.cgi/greyd/

-Dmitry

David Woodhouse wrote:
> On Thu, 2009-06-11 at 13:41 +0100, Ian P. Christian wrote:
>> 2009/6/8 David Woodhouse <dwmw2 [at] infradead>:
>>> I'd suggest reading http://wiki.exim.org/SimpleGreylisting -- the prose
>>> sets out some things that you may want to think about regardless of
>>> which greylisting implementation you use, and then there's an example
>>> Exim configuration which shouldn't suffer most of the stupid problems
>>> that postgrey does.
>> There's actually a flaw in this implementation here.
>
> Er, thanks for dropping me from Cc when you criticize my work...! :)
>
>> # Generate a hashed 'identity' for the mail, as described above.
>> warn set acl_m_greyident =
>> ${hash{20}{62}{$sender_address$recipients$h_message-id:}}
>>
>> Because it's common at the moment to get a mail to someone sent from
>> their own address without a message ID, hash clashes occour.
>
> Yeah, at the time I first implemented this I was just rejecting all mail
> without a Message-Id, so it wasn't much of an issue.
>
>> I'm currently not sure of the best way to deal with this - perahps
>> adding the Subject line into the hash...
>
> That seems like it would be a reasonable thing to do. Is it enough,
> though? A lot of spam messages have the same subject line too.
>
> What else could we include -- bearing in mind that we have to be sure
> that it _won't_ get changed by the sending MTA between retry attempts. I
> suppose we could use the full From:, To: and Cc: headers -- and maybe
> also the Date: header?
>
>> Perhaps I should just block mail sent from someone, to themselves,
>> with a null message ID.
>
> You could use PRVS and just reject _all_ mail which is faked to appear
> as if it's from your own addresses, surely?
>


--
Dmitry Samersoff
dms [at] samersoff, http://devnull.samersoff.net
* There will come soft rains ...


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dwmw2 at infradead

Jun 27, 2009, 7:59 AM

Post #40 of 48 (2327 views)
Permalink
Re: greylisting [In reply to]

On Sat, 2009-06-27 at 17:28 +0400, Dmitry Samersoff wrote:
> I think SQL for graylist is totally overkill.

It's only sqlite -- it doesn't require a separate database server; it's
purely within Exim. You've got to have _some_ kind of database, and this
is more efficient than just doing it with text files (as my original
implementation did.

> Check this implementation:
> http://www.beastsoft.net/cgi-bin/hg/hgwebdir.cgi/greyd/

Ew, Mercurial and C++... not the best first impression.

A separate dæmon written in C++ with a 'thread pool' implementation and
weird OS 'abstraction' layers to handle signals... that's not overkill?

You also don't seem to be passing it anything other than $sender_address
and $sender_host_address -- and you're even assuming the latter is
Legacy IP, afaict.

--
dwmw2


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dms at samersoff

Jun 27, 2009, 8:49 AM

Post #41 of 48 (2323 views)
Permalink
Re: greylisting [In reply to]

David,

David Woodhouse wrote:
> On Sat, 2009-06-27 at 17:28 +0400, Dmitry Samersoff wrote:
>> I think SQL for graylist is totally overkill.
>
> It's only sqlite -- it doesn't require a separate database server; it's
> purely within Exim.

It doesn't make it faster and doesn't excuse SQL parser, transactions
and bunch of other staff not needed in this case.

> You've got to have _some_ kind of database, and this
> is more efficient than just doing it with text files (as my original
> implementation did.

sqlite is nice product (good step back to 1994) but people tend to
consider it as a cure-all-diseases magic pile. We need a record manager
here but not a database - i.e. Berkeley DB, not sqlite.

>> Check this implementation:
>> http://www.beastsoft.net/cgi-bin/hg/hgwebdir.cgi/greyd/
>
> Ew, Mercurial and C++... not the best first impression.
>
> A separate dæmon written in C++ with a 'thread pool' implementation and
> weird OS 'abstraction' layers to handle signals... that's not overkill?

It's really fast and scalable (actually what it was written for - one of
mid size ISP asked me for help). Also it couldn't cause email loss -
i.e. if something goes wrong e-mail just passed in.

> You also don't seem to be passing it anything other than $sender_address
> and $sender_host_address -- and you're even assuming the latter is
> Legacy IP, afaict.

I'm checking sender host address and sender from address, e.g:
209.85.218.168:*@gmail.com

This combination is sufficient enough but not perfect. What else you
suggest to check?


--
Dmitry Samersoff
dms [at] samersoff, http://devnull.samersoff.net
* There will come soft rains ...


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dwmw2 at infradead

Jun 27, 2009, 9:08 AM

Post #42 of 48 (2325 views)
Permalink
Re: greylisting [In reply to]

On Sat, 2009-06-27 at 19:49 +0400, Dmitry Samersoff wrote:
> David,
>
> David Woodhouse wrote:
> > On Sat, 2009-06-27 at 17:28 +0400, Dmitry Samersoff wrote:
> >> I think SQL for graylist is totally overkill.
> >
> > It's only sqlite -- it doesn't require a separate database server; it's
> > purely within Exim.
>
> It doesn't make it faster and doesn't excuse SQL parser, transactions
> and bunch of other staff not needed in this case.
>
> > You've got to have _some_ kind of database, and this
> > is more efficient than just doing it with text files (as my original
> > implementation did.
>
> sqlite is nice product (good step back to 1994) but people tend to
> consider it as a cure-all-diseases magic pile. We need a record manager
> here but not a database - i.e. Berkeley DB, not sqlite.

True -- it would be nice if we could use Berkeley DB that way from Exim,
but it's read-only.

> >> Check this implementation:
> >> http://www.beastsoft.net/cgi-bin/hg/hgwebdir.cgi/greyd/
> >
> > Ew, Mercurial and C++... not the best first impression.
> >
> > A separate dæmon written in C++ with a 'thread pool' implementation and
> > weird OS 'abstraction' layers to handle signals... that's not overkill?
>
> It's really fast and scalable (actually what it was written for - one of
> mid size ISP asked me for help). Also it couldn't cause email loss -
> i.e. if something goes wrong e-mail just passed in.

Sounds like it's being used too much. Ideally, I believe greylisting
should only be invoked for mails which look suspicious in some way, if
they come from a host which hasn't previously been observed to queue and
retry.

> > You also don't seem to be passing it anything other than $sender_address
> > and $sender_host_address -- and you're even assuming the latter is
> > Legacy IP, afaict.
>
> I'm checking sender host address and sender from address, e.g:
> 209.85.218.168:*@gmail.com

How's it going to cope with what I get on your incoming mail:
2001:4830:2446:ff00:214:51ff:fe65:c65c:dms [at] beastsoft

> This combination is sufficient enough but not perfect. What else you
> suggest to check?

There is some discussion of that on the wiki page to which I referred.

--
dwmw2


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


dms at samersoff

Jun 27, 2009, 10:00 AM

Post #43 of 48 (2324 views)
Permalink
Re: greylisting [In reply to]

David,

>>> A separate dæmon written in C++ with a 'thread pool' implementation and
>>> weird OS 'abstraction' layers to handle signals... that's not overkill?
>> It's really fast and scalable (actually what it was written for - one of
>> mid size ISP asked me for help). Also it couldn't cause email loss -
>> i.e. if something goes wrong e-mail just passed in.
>
> Sounds like it's being used too much. Ideally, I believe greylisting
> should only be invoked for mails which look suspicious in some way, if
> they come from a host which hasn't previously been observed to queue and
> retry.

I try to greylist all incoming mail as early as possible to minimize
relay load and (less important) traffic.

>>> You also don't seem to be passing it anything other than $sender_address
>>> and $sender_host_address -- and you're even assuming the latter is
>>> Legacy IP, afaict.
>> I'm checking sender host address and sender from address, e.g:
>> 209.85.218.168:*@gmail.com
>
> How's it going to cope with what I get on your incoming mail:
> 2001:4830:2446:ff00:214:51ff:fe65:c65c:dms [at] beastsoft

What is it? MSG ID? My outgoing e-mail not graylisted - I use
login/pass & SSL.

> There is some discussion of that on the wiki page to which I referred.

Some comments on wiki page:

I.
"One problem is that some "genuine" mail servers might be broken in the
same way as we expect the spam bots to be."

Much more often problem is clusters: lots of SMTP clusters can resend
e-mail from another IP address. Fortunately most of them is not
spammers. I think a) the problem should be mentioned in this chapter b)
it's better to whitelist such clusters rather then exclude sender IP
from hash calculation.

II.

"Firstly, there's the database of "known resenders", which lists the
hosts that are known to retry sending mail."

In the NAT world I'm in doubt that such list have a sense.

--
Dmitry Samersoff
dms [at] samersoff, http://devnull.samersoff.net
* There will come soft rains ...


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


list at srv0

Jun 27, 2009, 10:27 AM

Post #44 of 48 (2324 views)
Permalink
Re: greylisting [In reply to]

Le samedi 27 juin 2009 à 15:28:34, Dmitry Samersoff, vous avez écrit :
> David,
>
> I think SQL for graylist is totally overkill.
[...]

I used 'greylistd' python script ( using Exim socket ), then decided to
reimplement its in purely within Exim ${dlfunc.

In RCPT or DATA :

defer condition = ${if ...}{1}{0}}
set acl_m_gris = ${dlfunc{/usr/lib/exim/exim-gris.so}{gris}}
condition = ${if >{$acl_m_gris}{0}}
message = Greylisting, please try again later
($acl_m_gris secondes)

This ${dlfunc use Exim functions for accessing Exim's hints database. This
module does not contain code for reading DBM files, all the relevant
functions are called as Exim macros (exim-4.69/src/dbstuff.h,
exim-4.69/src/dbfunctions.h). The DBM file has stored in Exim
spool_directory/db/greylisting

For maintaining greylisting database, this function auto remove obsolete data
(macro LAST_UPDATE 60*15).

To dump out the contents of the database:
% exim -be '${dlfunc{/usr/lib/exim/exim-gris.so}{dump}}'
#
# Greylisting Sat Jun 27 18:45:57 2009
#
{
key = "131.111.8/exim-users [at] exim/recipient [at] domain"
flag = W
last = Thu Jun 25 13:29:04 2009
first = Thu Jun 25 13:19:54 2009
count = 1
}
[...]
# Total: 83 (White: 57|Grey: 25)

% gcc -O2 -Wall -Werror -shared -fPIC -g \
-I./exim-4.69/build-Linux-i386/ \
-L/usr/lib \
-DLAST_UPDATE=60*15 \
-DMAX_GREY=60*60*8 \
-DRETRY_MAX=60*60*24*36 \
-DRETRY_MIN=60*5 \
-o /usr/lib/exim/exim-gris.so exim-gris.c \
&& strip /usr/lib/exim/exim-gris.so


-- I'm French. Sorry for my poor English.

--
Serge Demonchaux
Attachments: exim-gris.c (8.46 KB)


dwmw2 at infradead

Jun 27, 2009, 10:38 AM

Post #45 of 48 (2325 views)
Permalink
Re: greylisting [In reply to]

On Sat, 2009-06-27 at 21:00 +0400, Dmitry Samersoff wrote:
> David,
>
> >>> A separate dæmon written in C++ with a 'thread pool' implementation and
> >>> weird OS 'abstraction' layers to handle signals... that's not overkill?
> >> It's really fast and scalable (actually what it was written for - one of
> >> mid size ISP asked me for help). Also it couldn't cause email loss -
> >> i.e. if something goes wrong e-mail just passed in.
> >
> > Sounds like it's being used too much. Ideally, I believe greylisting
> > should only be invoked for mails which look suspicious in some way, if
> > they come from a host which hasn't previously been observed to queue and
> > retry.
>
> I try to greylist all incoming mail as early as possible to minimize
> relay load and (less important) traffic.
>
> >>> You also don't seem to be passing it anything other than $sender_address
> >>> and $sender_host_address -- and you're even assuming the latter is
> >>> Legacy IP, afaict.
> >> I'm checking sender host address and sender from address, e.g:
> >> 209.85.218.168:*@gmail.com
> >
> > How's it going to cope with what I get on your incoming mail:
> > 2001:4830:2446:ff00:214:51ff:fe65:c65c:dms [at] beastsoft
>
> What is it? MSG ID? My outgoing e-mail not graylisted - I use
> login/pass & SSL.

That's just $sender_host_address:$sender_address for the mail that you
sent, on one of its hops on the way to me.

I was asking how your code would cope with it, if I was using your code.

> > There is some discussion of that on the wiki page to which I referred.
>
> Some comments on wiki page:
>
> I.
> "One problem is that some "genuine" mail servers might be broken in the
> same way as we expect the spam bots to be."
>
> Much more often problem is clusters: lots of SMTP clusters can resend
> e-mail from another IP address. Fortunately most of them is not
> spammers. I think a) the problem should be mentioned in this chapter b)
> it's better to whitelist such clusters rather then exclude sender IP
> from hash calculation.

That would be nice if we know where they are. If they're on the dnswl,
they _will_ be skipped by the configuration I suggest.

But that's still not sufficient for the hosts that aren't -- the problem
is that if you include the sender IP in the hash, you'll settle into a
mode where you _always_ defer the mail from the first MTA, and always
accept it from the backup, when it's deferred. You're forcing your mail
to _always_ go through a slower path than it needs to.

> II.
>
> "Firstly, there's the database of "known resenders", which lists the
> hosts that are known to retry sending mail."
>
> In the NAT world I'm in doubt that such list have a sense.

That's why we use $sender_helo_name _and_ $sender_host_address, rather
than just $sender_host_address.

It's not perfect, but the alternative is that you just keep on deferring
mail from the same hosts over and over again, even though you gain no
benefit by doing so. You can of course do that if you wish, but many
people won't want to.

--
dwmw2


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


hs at schlittermann

Jun 27, 2009, 1:50 PM

Post #46 of 48 (2314 views)
Permalink
Re: greylisting [In reply to]

Dmitry Samersoff <dms [at] samersoff> (Sa 27 Jun 2009 19:00:13 CEST):
...
>
> Much more often problem is clusters: lots of SMTP clusters can resend
> e-mail from another IP address. Fortunately most of them is not
> spammers. I think a) the problem should be mentioned in this chapter b)
> it's better to whitelist such clusters rather then exclude sender IP
> from hash calculation.

I'm greylisting on '$sender_address|$local_part@$domain'. It works with
cluster based senders (web.de, I think).

It's perl based as exim "plugin", but limited to a single host.

http://www.schlittermann.de/doc/grey.shtml

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de ---------------------------- internet & unix support -
Heiko Schlittermann HS12-RIPE -----------------------------------------
gnupg encrypted messages are welcome - key ID: 48D0359B ---------------
gnupg fingerprint: 3061 CFBF 2D88 F034 E8D2 7E92 EE4E AC98 48D0 359B -
Attachments: signature.asc (0.19 KB)


chris+exim at qwirx

Jun 27, 2009, 2:07 PM

Post #47 of 48 (2318 views)
Permalink
Re: greylisting [In reply to]

Hi all,

On Sat, 27 Jun 2009, Heiko Schlittermann wrote:

> Dmitry Samersoff <dms [at] samersoff> (Sa 27 Jun 2009 19:00:13 CEST):
>>
>> Much more often problem is clusters: lots of SMTP clusters can resend
>> e-mail from another IP address. Fortunately most of them is not
>> spammers. I think a) the problem should be mentioned in this chapter
>> b) it's better to whitelist such clusters rather then exclude sender IP
>> from hash calculation.

It's not exactly a solution, but Jaco Kroon in South Africa is running an
RBL which advertises hosts that pass greylisting. The big server farms are
all on it, all of their servers, and new ones get added pretty quickly.
We're using it effectively share our greylist database with him, by
contributing our database to his RBL as well.

http://jkroon.blogs.uls.co.za/it/email/spam/getting-a-grip-on-greylisting

> It's perl based as exim "plugin", but limited to a single host.

I do greylisting in pure exim + mysql. You can see how in my spam
presentation at www.exim-new-users.co.uk, although the site seems to be
down at the moment so I can't give you a link. I've also updated my config
a bit since then, especially to integrate with what Jaco is doing.

Cheers, Chris.
--
_____ __ _
\ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


martin at antibodymx

Jun 7, 2010, 4:54 AM

Post #48 of 48 (1239 views)
Permalink
Re: greylisting [In reply to]

On Mon, June 7, 2010 11:57, Chris Laif wrote:
> Greylisting Using Memcached (only ~15 lines exim.conf, no Perl)
> http://www.noerenberg.de/hajo/pub/exim-greylisting-memcached.conf.txt

And people accuse Perl of being unreadable line noise...... :)



--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

First page Previous page 1 2 Next page Last page  View All exim users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.