Mailing List Archive: SpamAssassin: users

DUL/DUL+ redesign to improve DNS cache hit ratio [Was: TTL and DNSBLs]

Index | Next | Previous | View Threaded

andrzej.filip at gmail

Jul 5, 2011, 2:19 PM

Post #1 of 6 (330 views)
Permalink

DUL/DUL+ redesign to improve DNS cache hit ratio [Was: TTL and DNSBLs]

"David F. Skoll" <dfs [at] roaringpenguin> wrote:
> On Mon, 04 Jul 2011 13:52:00 +0200
> Axb <axb.lists [at] gmail> wrote:
>
>> BLs generally adjust their negative TTL to get a practical balance
>> between query load and positive hits.
>> Gaming these settings can become a costly process.
>
> My experiments on real mail servers show that DNS caching is quite
> ineffective for DNSBLs (at least for typical ones like Spamhaus that
> use a short TTL on the order of 15-30 minutes.)
>
> Results of my experiments are in these slides (PDF):
> http://ipv6summit.ca/index.php/v6/2011/paper/view/8/4
>
> Executive summary: On a very quiet mail server, assuming a 15-minute
> TTL, there was only a 50% cache hit rate on DNSBL lookups. On a
> fairly busy mail server, the cache hit rate fell to 22%.
>
> The problem, of course, is that most mail servers are hit by
> connections from all over the place... spammers have a lot of IP
> addresses to choose from, so you don't get much repetition within the
> TTL of a typical DNSBL. If you really need high-performance DNSBL
> lookups, you need to arrange for a zone transfer and run a local
> authoritative name server for the DNSBL.

Would you recommend redesigning (mainly) DUL/DUL+ DNSBL lists to improve
DNS cache hit ratio?

--
[pl>en: Andrew] Andrzej Adam Filip : anfi [at] onet
The Second Law of Thermodynamics:
If you think things are in a mess now, just wait!
-- Jim Warner

andrzej.filip at gmail

Jul 5, 2011, 2:26 PM

Post #2 of 6 (312 views)
Permalink

DUL/DUL+ redesign to improve DNS cache hit ratio [Was: TTL and DNSBLs] [In reply to]

dfs at roaringpenguin

Jul 5, 2011, 3:55 PM

Post #3 of 6 (305 views)
Permalink

Re: DUL/DUL+ redesign to improve DNS cache hit ratio [Was: TTL and DNSBLs] [In reply to]

On Tue, 05 Jul 2011 23:26:16 +0200
Andrzej Adam Filip <andrzej.filip [at] gmail> wrote:

> Would you recommend redesigning (mainly) DUL/DUL+ DNSBL lists to
> improve DNS cache hit ratio?

No, not really. The poor cache hit ratio doesn't seem to be a problem
in practice (most people were surprised by the results). If you have
a high-enough lookup volume that it does become a problem, you just arrange
to obtain (or buy) the data and run a local authoritative name server.

Regards,

David.

andrzej.filip at gmail

Jul 5, 2011, 11:15 PM

Post #4 of 6 (298 views)
Permalink

Re: DUL/DUL+ redesign to improve DNS cache hit ratio [SA v. MTA] [In reply to]

"David F. Skoll" <dfs [at] roaringpenguin> wrote:
> On Tue, 05 Jul 2011 23:26:16 +0200
> Andrzej Adam Filip <andrzej.filip [at] gmail> wrote:
>
>> Would you recommend redesigning (mainly) DUL/DUL+ DNSBL lists to
>> improve DNS cache hit ratio?
>
> No, not really. The poor cache hit ratio doesn't seem to be a problem
> in practice (most people were surprised by the results). If you have
> a high-enough lookup volume that it does become a problem, you just arrange
> to obtain (or buy) the data and run a local authoritative name server.

You are most likely right in case of SA asking all configured DNSBL to
generate spam score - improvement of some/minority DNS cache hit ration
would not be impressive in improving overall preference.
*BUT*
It may improve performance e.g. in case of hundredths mail servers in a
data/co-location center using shared forwarder and rejecting on first
DNSBL hit. Somehow I doubt buying data for such "reseller" configuration
is legally encouraged by paid DNSBL operators.

--
[pl>en: Andrew] Andrzej Adam Filip : anfi [at] onet
Welcome to Lake Wobegon, where all the men are strong, the women are pretty,
and the children are above-average.
-- Garrison Keillor

dfs at roaringpenguin

Jul 6, 2011, 5:00 AM

Post #5 of 6 (292 views)
Permalink

Re: DUL/DUL+ redesign to improve DNS cache hit ratio [SA v. MTA] [In reply to]

On Wed, 06 Jul 2011 08:15:47 +0200
Andrzej Adam Filip <andrzej.filip [at] gmail> wrote:

> It may improve performance e.g. in case of hundredths mail servers in
> a data/co-location center using shared forwarder and rejecting on
> first DNSBL hit. Somehow I doubt buying data for such "reseller"
> configuration is legally encouraged by paid DNSBL operators.

This is true. But it's also not in paid DNSBL operators interest to
improve the hit ratio. If the cache hit ratio is improved too much,
the DNSBL operators would be unable to detect heavy users and ask
(threaten) them for money. :) In the limiting case, if the cache
becomes *too* effective, the organization hosting the cache *is*
effectively providing the whole data set to its users.

Regards,

David.

andrzej.filip at gmail

Jul 6, 2011, 11:13 AM

Post #6 of 6 (291 views)
Permalink

Re: DUL/DUL+ redesign to improve DNS cache hit ratio [SA v. MTA] [In reply to]

"David F. Skoll" <dfs [at] roaringpenguin> wrote:
> On Wed, 06 Jul 2011 08:15:47 +0200
> Andrzej Adam Filip <andrzej.filip [at] gmail> wrote:
>
>> It may improve performance e.g. in case of hundredths mail servers in
>> a data/co-location center using shared forwarder and rejecting on
>> first DNSBL hit. Somehow I doubt buying data for such "reseller"
>> configuration is legally encouraged by paid DNSBL operators.
>
> This is true. But it's also not in paid DNSBL operators interest to
> improve the hit ratio. If the cache hit ratio is improved too much,
> the DNSBL operators would be unable to detect heavy users and ask
> (threaten) them for money. :) In the limiting case, if the cache
> becomes *too* effective, the organization hosting the cache *is*
> effectively providing the whole data set to its users.

To put it short:
a) Only DNSBL listing "net ranges" (e.g. DUL/DUL+, network "reputation")
can be quite easily redesigned to improve DNS hit ratio (IMHO)
b) Free of charge DNSBL would benefit the most
c) In case of DUL list quality is not (IMHO) defined by big */16 entries
(e.g. home ADSL ranges) that will generate most DNS cache hits

--
[pl>en: Andrew] Andrzej Adam Filip : anfi [at] onet
I do not believe that this generation of Americans is willing to resign
itself to going to bed each night by the light of a Communist moon...
-- Lyndon B. Johnson

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads