Mailing List Archive: SpamAssassin: users

Hostkarma: to be or not to be in SA defaults

Index | Next | Previous | View Threaded

sa-list at alexb

Sep 29, 2009, 10:41 PM

Post #1 of 13 (1108 views)
Permalink

Hostkarma: to be or not to be in SA defaults

been following Warren Togami's aggressive lobbying for adding RBLs to
SA's defaults, and I have some questions:

- is it wise to add yet even more lookups to BLs and slow down SA's
already huge amount of DNS lookups.

- is the BL in question (which ever it may be) prepared for sustaining
the global traffic load of millions of default SA setups.

- does the BL have a track record, wide acceptance, safety and
reliability to become a standard in SA?

- shouldn't SA be conservative and deliver *safe* default setups
allowing the end user/admin/whatever decide how far he/she wants to hog
his setup with by querying yet more BLs.

- With all respect for Mark and his efforts: there is a track of one man
operated BLs being DDOS'd to oblivion, operators disappearing, etc.
Should this be weighted as well?

I believe these points should have more weight than arguing about
trivial naming or BL colours....

comments?

have a good day...

wtogami at redhat

Sep 30, 2009, 3:31 AM

Post #2 of 13 (1061 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

On 09/30/2009 01:41 AM, Yet Another Ninja wrote:
> been following Warren Togami's aggressive lobbying for adding RBLs to
> SA's defaults, and I have some questions:
>
> - is it wise to add yet even more lookups to BLs and slow down SA's
> already huge amount of DNS lookups.
>
> - is the BL in question (which ever it may be) prepared for sustaining
> the global traffic load of millions of default SA setups.
>
> - does the BL have a track record, wide acceptance, safety and
> reliability to become a standard in SA?
>
> - shouldn't SA be conservative and deliver *safe* default setups
> allowing the end user/admin/whatever decide how far he/she wants to hog
> his setup with by querying yet more BLs.
>
> - With all respect for Mark and his efforts: there is a track of one man
> operated BLs being DDOS'd to oblivion, operators disappearing, etc.
> Should this be weighted as well?
>
> I believe these points should have more weight than arguing about
> trivial naming or BL colours....
>

These are good questions. I am only proposing at this point putting
this DNSBL into the sandbox so it can be tested against the corpa and we
can get some real statistics.

Warren Togami
wtogami [at] redhat

kremels at kreme

Sep 30, 2009, 7:16 AM

Post #3 of 13 (1055 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

On 29-Sep-2009, at 23:41, Yet Another Ninja wrote:

> been following Warren Togami's aggressive lobbying for adding RBLs
> to SA's defaults, and I have some questions:
>
> - is it wise to add yet even more lookups to BLs and slow down SA's
> already huge amount of DNS lookups.

Slow down? DNS lookups are one of the fastest things you can do.

--
"There will always be women in rubber flirting with me."

marc at perkel

Sep 30, 2009, 9:32 AM

Post #4 of 13 (1048 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

Yet Another Ninja wrote:
> been following Warren Togami's aggressive lobbying for adding RBLs to
> SA's defaults, and I have some questions:
>
> - is it wise to add yet even more lookups to BLs and slow down SA's
> already huge amount of DNS lookups.
>
> - is the BL in question (which ever it may be) prepared for sustaining
> the global traffic load of millions of default SA setups.
>
> - does the BL have a track record, wide acceptance, safety and
> reliability to become a standard in SA?
>
> - shouldn't SA be conservative and deliver *safe* default setups
> allowing the end user/admin/whatever decide how far he/she wants to
> hog his setup with by querying yet more BLs.
>
> - With all respect for Mark and his efforts: there is a track of one
> man operated BLs being DDOS'd to oblivion, operators disappearing, etc.
> Should this be weighted as well?
>
> I believe these points should have more weight than arguing about
> trivial naming or BL colours....
>
> comments?
>
> have a good day...
>
>
>

I have a lot of mighty servers set up ad have servers at 4 locations. I
have 50mb bought and using about 30 of it now. I am not sure what it
takes to support a default SA inclusion. Does anyone know if what I
described sounds like it is enough?

marc at perkel

Sep 30, 2009, 9:33 AM

Post #5 of 13 (1064 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

LuKreme wrote:
> On 29-Sep-2009, at 23:41, Yet Another Ninja wrote:
>
>> been following Warren Togami's aggressive lobbying for adding RBLs to
>> SA's defaults, and I have some questions:
>>
>> - is it wise to add yet even more lookups to BLs and slow down SA's
>> already huge amount of DNS lookups.
>
> Slow down? DNS lookups are one of the fastest things you can do.
>
>

I agree. Slow is MySQL basian processing.

wtogami at redhat

Sep 30, 2009, 11:08 AM

Post #6 of 13 (1055 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

On 09/30/2009 12:32 PM, Marc Perkel wrote:
> I have a lot of mighty servers set up ad have servers at 4 locations. I
> have 50mb bought and using about 30 of it now. I am not sure what it
> takes to support a default SA inclusion. Does anyone know if what I
> described sounds like it is enough?
>

You personally run all mirrors for DNS lookups? I believe all the other
major DNSBL's have many mirrors not all hosted in the same place.

Warren

raymond at prolocation

Sep 30, 2009, 1:25 PM

Post #7 of 13 (1053 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

Hi!

>> I have a lot of mighty servers set up ad have servers at 4 locations. I
>> have 50mb bought and using about 30 of it now. I am not sure what it
>> takes to support a default SA inclusion. Does anyone know if what I
>> described sounds like it is enough?

> You personally run all mirrors for DNS lookups? I believe all the other
> major DNSBL's have many mirrors not all hosted in the same place.

With SURBL we use much more then 50 mbit. And dont even mention the stuff
you get for free with the regular DDoS once you get on the radar. If you
say, hey i have 20 mbit this should do the trick. Please stop. Seriously.

It certainly does not sound as enough to me. But then again, i dont use
public mirrors so it wont affect me. But i doubt its a wise move to
include this inside SA just like that.

We advised SA in the past about inclusions like this and that time there
was conscences about the whole idea. Imagine a floaky BL, this will impact
the complete mailprocessing flow of many many ISPs and organisations.

Its not like pusing out a new version of RedHat (where we also provide
mirrors there) this is much more about impacting people's internal
mailflows. I would really appreciate that this would be handled with
great care.

Mark, how many people are there working on your BL, more then 1? Not
saying this is bad, just pointing out the risk adding stuff inside SA. Its
not a playground its legacy production stuff for many people.

The DoS/DDoS is really a risk, many of the BL operators have been bitten a
lot of times. For SURBL the worst DDoS we have faced got us a little over 40
Gbit/s. If you feel your company can live without network for some days,
sure, go ahead.... :-) If not, think twice. The DDoS we had lasted for
about 4 days. And we regularly get DDoS attacks, shorter and longer, on
the websites....

To be hounest, and this Mark, is not against you, but the current
situation. How the servers are setup, the single company stuff with the
RBL servers. I rather say, lets include the Barracuda BL, i am not a fan
of that specific list, but the infra is backed up by a large company doing
gigabits of traffic. Not a 'we have 20 mbit left over, lets do it' Any
university user inside .nl has more then that available.

I sincerly hope people realize its a serious thing, and take this mail to
improove things and setups. And please dont include lists that are not up
to the task yet).

thanks for you time.

Raymond Dijkxhoorn.

sa-list at alexb

Sep 30, 2009, 2:01 PM

Post #8 of 13 (1048 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

On 9/30/2009 10:25 PM, Raymond Dijkxhoorn wrote:
> Hi!
>
>>> I have a lot of mighty servers set up ad have servers at 4 locations. I
>>> have 50mb bought and using about 30 of it now. I am not sure what it
>>> takes to support a default SA inclusion. Does anyone know if what I
>>> described sounds like it is enough?
>
>> You personally run all mirrors for DNS lookups? I believe all the
>> other major DNSBL's have many mirrors not all hosted in the same place.
>
> With SURBL we use much more then 50 mbit. And dont even mention the
> stuff you get for free with the regular DDoS once you get on the radar.
> If you say, hey i have 20 mbit this should do the trick. Please stop.
> Seriously.
>
> It certainly does not sound as enough to me. But then again, i dont use
> public mirrors so it wont affect me. But i doubt its a wise move to
> include this inside SA just like that.
>
> We advised SA in the past about inclusions like this and that time there
> was conscences about the whole idea. Imagine a floaky BL, this will
> impact the complete mailprocessing flow of many many ISPs and
> organisations.
>
> Its not like pusing out a new version of RedHat (where we also provide
> mirrors there) this is much more about impacting people's internal
> mailflows. I would really appreciate that this would be handled with
> great care.
>
> Mark, how many people are there working on your BL, more then 1? Not
> saying this is bad, just pointing out the risk adding stuff inside SA.
> Its not a playground its legacy production stuff for many people.
>
> The DoS/DDoS is really a risk, many of the BL operators have been bitten
> a lot of times. For SURBL the worst DDoS we have faced got us a little
> over 40 Gbit/s. If you feel your company can live without network for
> some days, sure, go ahead.... :-) If not, think twice. The DDoS we had
> lasted for about 4 days. And we regularly get DDoS attacks, shorter and
> longer, on the websites....
>
> To be hounest, and this Mark, is not against you, but the current
> situation. How the servers are setup, the single company stuff with the
> RBL servers. I rather say, lets include the Barracuda BL, i am not a fan
> of that specific list, but the infra is backed up by a large company
> doing gigabits of traffic. Not a 'we have 20 mbit left over, lets do it'
> Any university user inside .nl has more then that available.
>
> I sincerly hope people realize its a serious thing, and take this mail
> to improove things and setups. And please dont include lists that are
> not up to the task yet).

This may be of interest..
http://www.uribl.com/mirrors.shtml

wtogami at redhat

Sep 30, 2009, 2:25 PM

Post #9 of 13 (1047 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

Nobody has yet proposed HOSTKARMA to become enabled by default. I am
only interested at the moment in testing how good it is in masschecks.

I would like to similarly add other DNSBL's that I haven't tried before
like spameatingmonkey or intercept to the masschecks. If you look
around online there isn't any good measure of DNSBL quality out there.
Our weekly_mass_check and sandbox is a very easy to way to measure this
stuff.

http://stats.dnsbl.com/
1067 hams is really the sample size he uses to measure false positives?
We can do a LOT better than this.

Any other free DNSBL's people are interested in testing?

Warren Togami
wtogami [at] redhat

rickm at ummm-beer

Sep 30, 2009, 2:40 PM

Post #10 of 13 (1057 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

Yet Another Ninja wrote:
> On 9/30/2009 10:25 PM, Raymond Dijkxhoorn wrote:
>> I sincerly hope people realize its a serious thing, and take this mail
>> to improove things and setups. And please dont include lists that are
>> not up to the task yet).
>
> This may be of interest..
> http://www.uribl.com/mirrors.shtml

I can see myself :)

Regards,

Rick

sm at resistor

Sep 30, 2009, 10:32 PM

Post #11 of 13 (1042 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

Hi Marc,
At 09:32 30-09-2009, Marc Perkel wrote:
>I have a lot of mighty servers set up ad have servers at 4
>locations. I have 50mb bought and using about 30 of it now. I am not
>sure what it takes to support a default SA inclusion. Does anyone
>know if what I described sounds like it is enough?

They can still be a soft target. Most of the DNSBLs were unprepared
to deal with denial of service attacks. Some of them have closed
down after an attack. That can be a problem for users as most people
have a "configure and forget" setup or it's a default vendor setup.

The bandwidth may be enough for current usage. The more mirrors you
have, the better. If your DNSBL is effective, you might be able to
get help with that. The "problems" with your setup is not worse than
other resources that are commonly used by users from this mailing list.

Someone pointed out that it's not a good idea to do more DNS lookups
as it affects the performance of SpamAssassin. It does not matter
whether your DNSBL is included in the default configuration as people
will use it if they believe that it is effective in stopping
spam. If you are concerned about marketing, then it may matter to you. :-)

Regards,
-sm

marc at perkel

Oct 1, 2009, 10:35 AM

Post #12 of 13 (1028 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

SM wrote:
> Hi Marc,
> At 09:32 30-09-2009, Marc Perkel wrote:
>> I have a lot of mighty servers set up ad have servers at 4 locations.
>> I have 50mb bought and using about 30 of it now. I am not sure what
>> it takes to support a default SA inclusion. Does anyone know if what
>> I described sounds like it is enough?
>
> They can still be a soft target. Most of the DNSBLs were unprepared
> to deal with denial of service attacks. Some of them have closed down
> after an attack. That can be a problem for users as most people have
> a "configure and forget" setup or it's a default vendor setup.
>
> The bandwidth may be enough for current usage. The more mirrors you
> have, the better. If your DNSBL is effective, you might be able to
> get help with that. The "problems" with your setup is not worse than
> other resources that are commonly used by users from this mailing list.
>
> Someone pointed out that it's not a good idea to do more DNS lookups
> as it affects the performance of SpamAssassin. It does not matter
> whether your DNSBL is included in the default configuration as people
> will use it if they believe that it is effective in stopping spam. If
> you are concerned about marketing, then it may matter to you. :-)
>
> Regards,
> -sm
>

I guess that if HOSTKARMA were included in the default build then I will
need more mirrors to handle the load.

michael.monnerie at is

Nov 11, 2009, 7:14 AM

Post #13 of 13 (774 views)
Permalink

Re: Hostkarma: to be or not to be in SA defaults [In reply to]

On Donnerstag 01 Oktober 2009 Marc Perkel wrote:
> I guess that if HOSTKARMA were included in the default build then I
> will need more mirrors to handle the load.

If that is wanted, I could talk to ISPs for hosting such DNS here in
Austria. After all, we are all getting advantages from this, and as long
as it's a free service, I can offer free mirrors.

mfg zmi
--
// Michael Monnerie, Ing.BSc ----- http://it-management.at
// Tel: 0660 / 415 65 31 .network.your.ideas.
// PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads