Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SpamAssassin: users

Blacklists Compared 17 October 2009

 

 

SpamAssassin users RSS feed   Index | Next | Previous | View Threaded


marc at perkel

Oct 23, 2009, 3:33 PM

Post #1 of 18 (2290 views)
Permalink
Blacklists Compared 17 October 2009

http://www.sdsc.edu/~jeff/spam/cbc.html

The races!

Big upset for Baracudda as Spamhaus takes back the #1 position and Spam
Eating Monkey comes in second. (I don't count apews) Hostkarma pulls
ahead of Uceprotect who have been running neck and neck for 5th and 6th
place.


mysqlstudent at gmail

Apr 6, 2010, 7:41 PM

Post #2 of 18 (2042 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Hi,

Last October Marc posted the following URL that compared the various RBLs:

> http://www.sdsc.edu/~jeff/spam/cbc.html

It seems barracuda is still leading, but is that also everyone's
experience? Can anyone provide details on how Jeff computed this
information and is it as cut-and-dried as this makes it seem? IOW,
barracuda, the free service, is "better" than all the rest...

Thanks,
Alex


support at junkemailfilter

Apr 6, 2010, 9:45 PM

Post #3 of 18 (2043 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On 4/6/2010 7:41 PM, Alex wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
> Thanks,
> Alex
>
>

I don't know the details but from what I understand it is a raw count of
who spams him. So the experience of others might vary. There's not a lot
of comparisons out there so this gives me some clue. But it doesn'y say
anything about the quality of the lists as it has apews listed highly.
If I created a list that blacklisted everything I would be first.

--
Marc Perkel - Sales/Support
support [at] junkemailfilter
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400


support at junkemailfilter

Apr 6, 2010, 9:46 PM

Post #4 of 18 (2047 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Here's another good list that rates quality.

http://www.intra2net.com/en/support/antispam/index.php

On 4/6/2010 7:41 PM, Alex wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
> Thanks,
> Alex
>
>

--
Marc Perkel - Sales/Support
support [at] junkemailfilter
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400


jm at jmason

Apr 7, 2010, 3:15 AM

Post #5 of 18 (2038 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

he doesn't take FPs into account. this is a very serious problem with
the methodology.

--j.

On Wed, Apr 7, 2010 at 03:41, Alex <mysqlstudent [at] gmail> wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
> Thanks,
> Alex
>
>


ned at unixmail

Apr 7, 2010, 3:38 AM

Post #6 of 18 (2035 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Alex wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>

As others have noted, FPs are not taken into account so one must
consider that.

Last year when the barracuda config was first posted to this list, I
implemented it on my personal mail server with a very high score so as
to trigger automatic quarantines for all mail hitting the list, and have
since checked all hits by hand. I currently use zen.spamhaus at the smtp
stage to reject spam, so hits against barracuda only comprise of those
that are "missed" by zen. I was particularly interested in FPs.

During the last year I don't think I've seen a single FP hit against
barracuda :surprised: That said, I still haven't found the confidence to
implement it at the smtp stage for outright rejection but the numbers
I'm seeing do tend towards telling me the list is of generally high quality.


corpus.defero at idnet

Apr 7, 2010, 4:01 AM

Post #7 of 18 (2040 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On Wed, 2010-04-07 at 11:38 +0100, Ned Slider wrote:
> Alex wrote:
> > Hi,
> >
> > Last October Marc posted the following URL that compared the various RBLs:
> >
> >> http://www.sdsc.edu/~jeff/spam/cbc.html
> >
> > It seems barracuda is still leading, but is that also everyone's
> > experience? Can anyone provide details on how Jeff computed this
> > information and is it as cut-and-dried as this makes it seem? IOW,
> > barracuda, the free service, is "better" than all the rest...
> >
>
> As others have noted, FPs are not taken into account so one must
> consider that.
>
> Last year when the barracuda config was first posted to this list, I
> implemented it on my personal mail server with a very high score so as
> to trigger automatic quarantines for all mail hitting the list, and have
> since checked all hits by hand. I currently use zen.spamhaus at the smtp
> stage to reject spam, so hits against barracuda only comprise of those
> that are "missed" by zen. I was particularly interested in FPs.
>
> During the last year I don't think I've seen a single FP hit against
> barracuda :surprised: That said, I still haven't found the confidence to
> implement it at the smtp stage for outright rejection but the numbers
> I'm seeing do tend towards telling me the list is of generally high quality.
>
>
In reality I make use of Barracuda first at SMTP time, Spamhaus after
and have done so since 2008. I've never seen a FP from Barracuda in that
time.

I'm no fan of Barracuda - and that is widely documented. However, they
are a legal, professional business that is accountable. Spamhaus, on the
other hand, are not. Whilst their efforts in blocklisting are laudable
and noted, they appear to operate in a somewhat underground manner
without any proper base or contact details - not unlike gypsies.

Until they become fully legitimate and accountable their business
credibility will remain in question. With Barracuda, yes, you know they
are selling GPL code, you know that one of them is a former spammer, you
know about 'emailreg'. They make no secret of it. You don't, however,
know about just who is pulling the strings at Spamhaus. On several
occasions in the past I have received obvious and clear spam from the
likes of IHM in Nottingham, B2B deals, and uncounted attacks from
Emailvision in France all of which pass through Spamhaus and have you
saying 'Why is this?'

I have found (and I fully expect another round) that if you bad mouth or
question Spamhaus you are subjected to abuse, port scans, publication of
personal data in news groups and the like.

As far as the Barracuda list is concerned, I have total confidence in
it, and the company that operates it. Given the number of anti-spam
appliances they have in the field doing a very good job with it I would
say 'have confidence in it'.


spamassassin-users at lists

Apr 7, 2010, 4:27 AM

Post #8 of 18 (2040 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On 07/04/2010 12:01, corpus.defero wrote:

>> During the last year I don't think I've seen a single FP hit against
>> barracuda :surprised: That said, I still haven't found the confidence to
>> implement it at the smtp stage for outright rejection but the numbers
>> I'm seeing do tend towards telling me the list is of generally high quality.
>>
> In reality I make use of Barracuda first at SMTP time, Spamhaus after
> and have done so since 2008. I've never seen a FP from Barracuda in that
> time.

b.barracudacentral.org is amongst my top three lists these days. Along
with zen.spamhaus.org and bl.spamcop.net. There is no noticable
difference in the FP rate between them here and all three hit on a *lot*
of spam.

--
Mike Cardwell - Perl/Java/Web developer, Linux admin, Email admin
Read my tech Blog - https://secure.grepular.com/
Follow me on Twitter - http://twitter.com/mickeyc
Hire me - http://cardwellit.com/ http://uk.linkedin.com/in/mikecardwell


Ralf.Hildebrandt at charite

Apr 7, 2010, 4:34 AM

Post #9 of 18 (2039 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

* Ned Slider <ned [at] unixmail>:

> Last year when the barracuda config was first posted to this list, I
> implemented it on my personal mail server with a very high score so
> as to trigger automatic quarantines for all mail hitting the list,
> and have since checked all hits by hand. I currently use zen.spamhaus
> at the smtp stage to reject spam, so hits against barracuda only
> comprise of those that are "missed" by zen. I was particularly
> interested in FPs.

We use the same scheme

> During the last year I don't think I've seen a single FP hit against
> barracuda

I activated barracuda one week ago and already got 2 FPs. So one's MMV

--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebrandt [at] charite | http://www.charite.de


Mariusz.Kruk at epsilon

Apr 7, 2010, 5:12 AM

Post #10 of 18 (2041 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On Wednesday, 7 of April 2010, Marc Perkel wrote:
> Here's another good list that rates quality.
>
> http://www.intra2net.com/en/support/antispam/index.php

The methodology behind this rating is kinda peculiar.
What good is counting messages hit by lists? If I make a DNSBL which just
marks gmail as ham and everything else as spam, what accuracy would you
expect? Reasonably speaking, it should be inaccurate like hell but if you get
majority of mails from gmail, you'd get accuracy rate of 9x percent which
would be kinda silly. Repeating the same test is not giving any new
information.

--
/\-\/\-\/\-\/\-\/\-\/\-\/\
\ Kruk [at] epsilon /
/ http://epsilon.eu.org/ \
\/-/\/-/\/-/\/-/\/-/\/-/\/


raymond at prolocation

Apr 7, 2010, 6:14 AM

Post #11 of 18 (2041 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Hi!

>>> http://www.sdsc.edu/~jeff/spam/cbc.html

>> It seems barracuda is still leading, but is that also everyone's
>> experience? Can anyone provide details on how Jeff computed this
>> information and is it as cut-and-dried as this makes it seem? IOW,
>> barracuda, the free service, is "better" than all the rest...

> spams him. So the experience of others might vary. There's not a lot of
> comparisons out there so this gives me some clue. But it doesn'y say anything
> about the quality of the lists as it has apews listed highly. If I created a
> list that blacklisted everything I would be first.

Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
Its of no use at all IMHO.

Bye,
Raymond.


mysqlstudent at gmail

Apr 7, 2010, 6:44 AM

Post #12 of 18 (2036 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Hi,

>>>> http://www.sdsc.edu/~jeff/spam/cbc.html
[...]

> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
> Its of no use at all IMHO.

Yes, certainly, and I guess it was a loaded question of me to ask,
because it was almost too obvious that I thought I was missing
something. I don't think it's _completely_ useless though, because the
two leaders are for the most part reputable and don't just block ANY
IP.

Raymond, I remember your name from reading at surbl.org (actually, you
must go to www.surbl.org, heh). Is this the list you recommend?

I somewhat agree with the comment about barracuda being accountable,
but mxtools is apparently running the business side of things at
SURBL, and for large systems you need to pay, so I would assume they
want you to come back to renew.

I'm sure searching could probably eventually find something, but does
anyone know or have links that show the surbl and barracuda motivation
behind their service? Of course it is to block spam, but what does
barracuda expect to gain by making it free, for example, where surbl
charges for theirs? They simply hope people will contribute back to it
to make it better for their appliance customers?

An occasional FP is almost equally as subjective as anything else, and
a difficult metric in which to place any comparison value.

Would a more detailed and thorough comparison be any use? IOW, number
of changes per day, how many total entries, etc, or is everyone okay
with how it is currently?

Thanks,
Alex


raymond at prolocation

Apr 7, 2010, 7:12 AM

Post #13 of 18 (2035 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Hi!

>> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
>> Its of no use at all IMHO.

> Yes, certainly, and I guess it was a loaded question of me to ask,
> because it was almost too obvious that I thought I was missing
> something. I don't think it's _completely_ useless though, because the
> two leaders are for the most part reputable and don't just block ANY
> IP.

> Raymond, I remember your name from reading at surbl.org (actually, you
> must go to www.surbl.org, heh). Is this the list you recommend?

I dont recommend any list. Its up to anyone to pick and mix. Some list
work for the other, some dont. My personal experience with the cuda list
is not good. That doesnt mean it will work for others. That wasnt the
story at all. You asked if this comparison is a benchmark, it is, but
doesnt say anything about FP rating or effectiveness so its just a
number. A rather useless number also IMHO.

> I somewhat agree with the comment about barracuda being accountable,
> but mxtools is apparently running the business side of things at
> SURBL, and for large systems you need to pay, so I would assume they
> want you to come back to renew.

Not interested to talk about that. You asked about comparison, not about
pricing or surbl or mxtools or whatever. Please check my e-mail from thats
used on the list. I post on personal title here. If you wanna post or talk
to mxtools please try another list ;) Its offtopic.

> I'm sure searching could probably eventually find something, but does
> anyone know or have links that show the surbl and barracuda motivation
> behind their service? Of course it is to block spam, but what does
> barracuda expect to gain by making it free, for example, where surbl

Simple, to stop spam? For SURBL, the main reason was to help people out.
And we do that and have done that for allmost 8 years now.

We moved after giving away free service, and years of free time. Yes a
blacklist doesnt compile just like that... That didnt turn out for the
long term so we switched models. All is explained on different lists and
if you like to know, check there. We basicly wanted to secure service
level and existance for over time.

There is a lot of talks about Cuda, ask the people from Spamhaus or
any some of the open source vendor who's code is used on the Cuda devices,
on the SA list are enough people who can help you out with getting some
clue whats going on. I wont poke around in that fire, you get burned
easilly. So this is all you get ;)

> An occasional FP is almost equally as subjective as anything else, and
> a difficult metric in which to place any comparison value.
>
> Would a more detailed and thorough comparison be any use? IOW, number
> of changes per day, how many total entries, etc, or is everyone okay
> with how it is currently?

Independent testing like the VB tests tell me much more.

http://www.virusbtn.com/virusbulletin/archive/2010/03/vb201003-vbspam-comparative

And yes that more or less the commercial products, but it shows also how
lits like SURBL perform. But also ratings of the large vendors. And the
FP levels.

Even the SA corpus, where we have a open ticket about the ratings isnt
currently ok with rating. Score is calculated over time but domains
expire. Its hard... retesting doesnt give a full solution for that either.

See:

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6386

There is much more to do when rating things then just do some simple
counting. The scoring in SA is done pretty well (compliments!) but even
there are gaps and holes like pointed out.

So its certainly not something you can put aside in just a 5 mins talk.

bye,
Raymond.


corpus.defero at idnet

Apr 7, 2010, 7:41 AM

Post #14 of 18 (2038 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On Wed, 2010-04-07 at 15:14 +0200, Raymond Dijkxhoorn wrote:
> Hi!
>
> >>> http://www.sdsc.edu/~jeff/spam/cbc.html
>
> >> It seems barracuda is still leading, but is that also everyone's
> >> experience? Can anyone provide details on how Jeff computed this
> >> information and is it as cut-and-dried as this makes it seem? IOW,
> >> barracuda, the free service, is "better" than all the rest...
>
> > spams him. So the experience of others might vary. There's not a lot of
> > comparisons out there so this gives me some clue. But it doesn'y say anything
> > about the quality of the lists as it has apews listed highly. If I created a
> > list that blacklisted everything I would be first.
>
> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
> Its of no use at all IMHO.
>
> Bye,
> Raymond.
They have - it's called 'UCEPROTECT' ......


maillists at conactive

Apr 7, 2010, 7:44 AM

Post #15 of 18 (2031 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Alex wrote on Tue, 6 Apr 2010 22:41:18 -0400:

> > http://www.sdsc.edu/~jeff/spam/cbc.html

This list is not useable for choosing your preferred RBL. It doesn't take
false positives into account. apews.org for instance is not usable at all
and sorbs is only usable when you exclude their spamtraps list.

Kai

--
Get your web at Conactive Internet Services: http://www.conactive.com


support at junkemailfilter

Apr 7, 2010, 10:56 AM

Post #16 of 18 (2030 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On 4/7/2010 7:41 AM, corpus.defero wrote:
> On Wed, 2010-04-07 at 15:14 +0200, Raymond Dijkxhoorn wrote:
>
>> Hi!
>>
>>
>>>>> http://www.sdsc.edu/~jeff/spam/cbc.html
>>>>>
>>
>>>> It seems barracuda is still leading, but is that also everyone's
>>>> experience? Can anyone provide details on how Jeff computed this
>>>> information and is it as cut-and-dried as this makes it seem? IOW,
>>>> barracuda, the free service, is "better" than all the rest...
>>>>
>>
>>> spams him. So the experience of others might vary. There's not a lot of
>>> comparisons out there so this gives me some clue. But it doesn'y say anything
>>> about the quality of the lists as it has apews listed highly. If I created a
>>> list that blacklisted everything I would be first.
>>>
>> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
>> Its of no use at all IMHO.
>>
>> Bye,
>> Raymond.
>>
> They have - it's called 'UCEPROTECT' ......
>
>
>

Amen on the UCEPROTECT. What a bizarre company.

--
Marc Perkel - Sales/Support
support [at] junkemailfilter
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400


jdfalk-lists at cybernothing

Apr 8, 2010, 9:00 AM

Post #17 of 18 (2004 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

On Apr 7, 2010, at 4:15 AM, Justin Mason wrote:

> he doesn't take FPs into account. this is a very serious problem with
> the methodology.

+1

--
J.D. Falk <jdfalk [at] returnpath>
Return Path Inc


mysqlstudent at gmail

Apr 8, 2010, 10:03 PM

Post #18 of 18 (1988 views)
Permalink
Re: Blacklists Compared 17 October 2009 [In reply to]

Hi,

> Independent testing like the VB tests tell me much more.
>
> http://www.virusbtn.com/virusbulletin/archive/2010/03/vb201003-vbspam-comparative
>
> And yes that more or less the commercial products, but it shows also how
> lits like SURBL perform. But also ratings of the large vendors. And the FP
> levels.

Yes, that is good information. Sure appreciate your feedback.

> There is much more to do when rating things then just do some simple
> counting. The scoring in SA is done pretty well (compliments!) but even
> there are gaps and holes like pointed out.

I'm hoping to stay on topic with the RBLs and as they pertain to SA,
so I would also like to determine how to make an educated decision.
I've done some further research, and have found a few more useful
comparisons to add to the list:

http://spamlinks.net/filter-dnsbl-lists.htm

At the bottom of this page is a section that lists other comparisons.
It would be nice to know how current they are, however. The link also
contains a handful of neat command-line tools and libraries that can
be used to query the RBLs.

I'm especially interested in using multi.uribl.com and
zen.spamhaus.org at SMTP restriction time. However, it seems like zen
is the one that is most popular. Should uribl.com not be used for
this?

At SMTP time, are there others that people should consider using?

This would be really helpful, because there doesn't seem to be any
authoritative info out there about achieving the best configuration,
outside of simple trial-and-error.

Thanks,
Alex

SpamAssassin users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.