Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SpamAssassin: users

These are getting through SA...

 

 

First page Previous page 1 2 Next page Last page  View All SpamAssassin users RSS feed   Index | Next | Previous | View Threaded


luis.otegui at gmail

Jun 8, 2007, 2:04 PM

Post #1 of 40 (1334 views)
Permalink
These are getting through SA...

Hi, could somebody run this mail trough SA and give me the scores?
They aren't scoring very much here...

Return-Path: <budmedienkulturhaussow [at] medienkulturhaus>
X-Original-To: user [at] domain
Delivered-To: user [at] use@domain.com
Received: from localhost (localhost [127.0.0.1])
by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE;
Fri, 8 Jun 2007 17:25:09 -0300 (ART)
X-Virus-Scanned: by amavisd-new-2.4.4 (20061120) (Debian) at biol.unlp.edu.ar
X-Spam-Score: 3.964
X-Spam-Level: ***
X-Spam-Status: No, score=3.964 tagged_above=-100 required=5
tests=[BAYES_99=3.5, HTML_30_40=0.463, HTML_MESSAGE=0.001]
Received: from nahuel.biol.unlp.edu.ar ([127.0.0.1])
by localhost (nahuel.biol.unlp.edu.ar [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id C9PabKr0nGQL; Fri, 8 Jun 2007 17:25:08 -0300 (ART)
Received: from formiak.senix.pl (formiak.senix.pl [83.12.246.226])
by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 2A4A87E47;
Fri, 8 Jun 2007 17:25:03 -0300 (ART)
Received: from [83.12.246.226] by mail.medienkulturhaus.at; Fri, 8 Jun
2007 20:25:53 -0100
Date: Fri, 8 Jun 2007 20:25:53 -0100
From: "Deana Adams" <budmedienkulturhaussow [at] medienkulturhaus>
X-Mailer: The Bat! (v2.00.3) Business
Reply-To: budmedienkulturhaussow [at] medienkulturhaus
X-Priority: 3 (Normal)
Message-ID: <969404800.15691885521876 [at] medienkulturhaus>
To: internet [at] biol
Subject: Can you imagine that you are healthy?
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----------4B80CCCC51A29EFD"


------------4B80CCCC51A29EFD
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 7bit

LegalRXMedications drug store introduce all medications that you have
a need in in order to renew your health at low price.
We work across the globe with clients from Europe, America and Asia.
At the time you got no need to search for chemist's shop at your local area.
We surely bring pills of the best quality to the distant parts of the world.
Visit our site to obtain drugs you instantly need direct to your location.
http://teethcat.hk/
We're ratified by VISA & VeriSign then we provide effective and
dependable purchase.

------------4B80CCCC51A29EFD
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML><HEAD><TITLE></TITLE>
</HEAD>
<BODY>

<b><font color="#00CC33"><em>LegalRXMedications</em></font> drug store
introduce all medications that you have a need in in order to renew
your health at low price. <br>
We work across the globe with clients from Europe, America and Asia. <br>
At the time you got no need to search for chemist's shop at your local area.<br>
We surely bring pills of the best quality to the distant parts of the world.
<br>
<br>
<a href="http://teethcat.hk/"><em>Visit our site to obtain drugs you
instantly need direct to your location.</em></a></b>
<br>
<font color="#D9EDFF">http://teethcat.hk/</font>
<br><b>We're ratified by <font color="#FF0000"><em>VISA</em></font> &
<font color="#FF0000"><em>VeriSign</em></font> then we provide
effective and dependable purchase.
</b>

</BODY></HTML>
------------4B80CCCC51A29EFD--


--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------


raymond at prolocation

Jun 8, 2007, 2:15 PM

Post #2 of 40 (1272 views)
Permalink
Re: These are getting through SA... [In reply to]

Hi!

> They aren't scoring very much here...
>
> Return-Path: <budmedienkulturhaussow [at] medienkulturhaus>
> X-Original-To: user [at] domain
> Delivered-To: user [at] use@domain.com
> Received: from localhost (localhost [127.0.0.1])
> by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE;
> Fri, 8 Jun 2007 17:25:09 -0300 (ART)

X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50,
CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00,
RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00)

Allmost 15, not bad i think.

Bye,
Raymond.


mail at arni

Jun 8, 2007, 2:15 PM

Post #3 of 40 (1272 views)
Permalink
Re: These are getting through SA... [In reply to]

Luis Hernán Otegui schrieb:
> Hi, could somebody run this mail trough SA and give me the scores?
> They aren't scoring very much here...
>
Hi, your mailing probably broke half of the email so these scores are
only an estimate - if you want me to try again attach the mail as a raw
text (or .eml as many clients call it)

X-Spam-Report:
* 0.0 MISSING_MID Missing Message-Id: header
* 0.0 MISSING_DATE Missing Date: header
* 2.5 MISSING_HB_SEP Missing blank line between message header
and body
* 0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy
says domain
* signs some mails
* 1.3 MISSING_HEADERS Missing To: header
* 0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
* [score: 0.5000]
* 1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8
confidence level
* above 50%
* [cf: 100]
* 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
* 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level
above 50%
* [cf: 100]
* 2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
* [URIs: teethcat.hk]
* 1.8 MISSING_SUBJECT Missing Subject: header

arni


luis.otegui at gmail

Jun 8, 2007, 2:28 PM

Post #4 of 40 (1273 views)
Permalink
Re: These are getting through SA... [In reply to]

Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
source here, most of the other relevant info is in the header of the
mail I sent before to test). Where did you get them?

Thanks,


Luis

2007/6/8, Raymond Dijkxhoorn <raymond [at] prolocation>:
> Hi!
>
> > They aren't scoring very much here...
> >
> > Return-Path: <budmedienkulturhaussow [at] medienkulturhaus>
> > X-Original-To: user [at] domain
> > Delivered-To: user [at] use@domain.com
> > Received: from localhost (localhost [127.0.0.1])
> > by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE;
> > Fri, 8 Jun 2007 17:25:09 -0300 (ART)
>
> X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
> score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50,
> CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00,
> RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
> RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00)
>
> Allmost 15, not bad i think.
>
> Bye,
> Raymond.
>


--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------


spamassassin at dostech

Jun 8, 2007, 2:31 PM

Post #5 of 40 (1270 views)
Permalink
Re: These are getting through SA... [In reply to]

Luis Hernán Otegui wrote:
> Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> source here, most of the other relevant info is in the header of the
> mail I sent before to test). Where did you get them?

Run sa-update to get URIBL_BLACK and URIBL_GREY.

Daryl


raymond at prolocation

Jun 8, 2007, 2:32 PM

Post #6 of 40 (1272 views)
Permalink
Re: These are getting through SA... [In reply to]

Hi!

> Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> source here, most of the other relevant info is in the header of the
> mail I sent before to test). Where did you get them?

>> X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
>> score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50,
>> CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00,
>> RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
>> RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00)

Most likely since we get SURBL updates and so earlier then most of you
do. Could be timings issues.

Bye,
Raymond.


luis.otegui at gmail

Jun 8, 2007, 2:38 PM

Post #7 of 40 (1271 views)
Permalink
Re: These are getting through SA... [In reply to]

Well, right now I'm running these commands to get updates:

sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com

sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org

sa-update doesn't download URIBL_BLACK and URIBL_GREY

What am I doing wrong?


Luis

2007/6/8, Daryl C. W. O'Shea <spamassassin [at] dostech>:
> Luis Hernán Otegui wrote:
> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> > source here, most of the other relevant info is in the header of the
> > mail I sent before to test). Where did you get them?
>
> Run sa-update to get URIBL_BLACK and URIBL_GREY.
>
> Daryl
>


--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------


id at w98

Jun 8, 2007, 2:39 PM

Post #8 of 40 (1272 views)
Permalink
Re: These are getting through SA... [In reply to]

> X-Spam-Status: No, score=3.964 tagged_above=-100 required=5
> tests=[BAYES_99=3.5, HTML_30_40=0.463, HTML_MESSAGE=0.001]

To me, it looks like enough tokens were seen to flag it as BAYES_99, but
that the host and IP it came from didn't trigger any RBL hits, which
left your point score well under your 5.0 tolerance.

I can't speak for others, but I personally set my 'required' level to
3.5, so anything SpamAssassin flags as BAYES_99 automatically gets
flagged as spam. And since I use {SPAM _SCORE(0)_} as a subject rewrite,
I can catch any messages that score over another threshold (for me, it's
10.0) and filter them to /dev/null if they score that high -- the only
things that score that high are messages with bayes scores of 80-100
plus RBL hits.

Just my $0.02.


spamassassin at dostech

Jun 8, 2007, 2:41 PM

Post #9 of 40 (1274 views)
Permalink
Re: These are getting through SA... [In reply to]

If you've got the current update from updates.spamassassin.org you've
got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out
that they didn't hit for Raymond either, so you won't see them in this case.

Daryl


Luis Hernán Otegui wrote:
> Well, right now I'm running these commands to get updates:
>
> sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
>
> sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
>
> sa-update doesn't download URIBL_BLACK and URIBL_GREY
>
> What am I doing wrong?
>
>
> Luis
>
> 2007/6/8, Daryl C. W. O'Shea <spamassassin [at] dostech>:
>> Luis Hernán Otegui wrote:
>> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
>> > source here, most of the other relevant info is in the header of the
>> > mail I sent before to test). Where did you get them?
>>
>> Run sa-update to get URIBL_BLACK and URIBL_GREY.
>>
>> Daryl
>>
>
>


luis.otegui at gmail

Jun 8, 2007, 2:46 PM

Post #10 of 40 (1274 views)
Permalink
Re: These are getting through SA... [In reply to]

OK, i?ve been googlin' around, and it seems like an issue between
Amavis (or MailScanner, for waht I've found) and some unsupported
versions of Net::DNS, because when I run the message through SA by
itself, this comes out:

Content analysis details: (9.7 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP
3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
[score: 0.9999]
0.0 MISSING_MID Missing Message-Id: header
0.0 MISSING_DATE Missing Date: header
1.3 MISSING_HEADERS Missing To: header
2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: teethcat.hk]
1.8 MISSING_SUBJECT Missing Subject: header
2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
0.5 FM_NO_TO FM_NO_TO


So I'm blaming it on Amavis... (Net::DNS 0.59 here)...

I'll post this issue to the Amavis list.


Luis
2007/6/8, Daryl C. W. O'Shea <spamassassin [at] dostech>:
> If you've got the current update from updates.spamassassin.org you've
> got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out
> that they didn't hit for Raymond either, so you won't see them in this case.
>
> Daryl
>
>
> Luis Hernán Otegui wrote:
> > Well, right now I'm running these commands to get updates:
> >
> > sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
> >
> > sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
> >
> > sa-update doesn't download URIBL_BLACK and URIBL_GREY
> >
> > What am I doing wrong?
> >
> >
> > Luis
> >
> > 2007/6/8, Daryl C. W. O'Shea <spamassassin [at] dostech>:
> >> Luis Hernán Otegui wrote:
> >> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> >> > source here, most of the other relevant info is in the header of the
> >> > mail I sent before to test). Where did you get them?
> >>
> >> Run sa-update to get URIBL_BLACK and URIBL_GREY.
> >>
> >> Daryl
> >>
> >
> >
>
>


--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------


bill at inetmsg

Jun 8, 2007, 3:56 PM

Post #11 of 40 (1272 views)
Permalink
Re: These are getting through SA... [In reply to]

Daryl C. W. O'Shea wrote the following on 6/8/2007 2:41 PM -0800:
> If you've got the current update from updates.spamassassin.org you've
> got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns
> out that they didn't hit for Raymond either, so you won't see them in
> this case.
>
> Daryl
>
>
> Luis Hernán Otegui wrote:
>> Well, right now I'm running these commands to get updates:
>>
>> sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
>>
>> sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
>>
>> sa-update doesn't download URIBL_BLACK and URIBL_GREY
>>
>> What am I doing wrong?
>>
>>
>> Luis
>>
>> 2007/6/8, Daryl C. W. O'Shea <spamassassin [at] dostech>:
>>> Luis Hernán Otegui wrote:
>>> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
>>> > source here, most of the other relevant info is in the header of the
>>> > mail I sent before to test). Where did you get them?
>>>
>>> Run sa-update to get URIBL_BLACK and URIBL_GREY.
>>>
>>> Daryl
For what it's worth, and maybe it's different than the issues you are
seeing, but I have been finding with SA 3.2.0, that some messages are
not firing on URIBL tests. However, if the headers of the message are
removed, the URIBL tests run fine. I posted a bug report to bugzilla
(Bug 5506). I am running SA on Fedora 7 with perl 5.8.8 and was doing
my test strictly with SA:

spamassassin -t < test.txt

I did this because I wanted to take amavisd-new out of the picture.
Also, someone mentioned Net::DNS, my version is 0.59.

Bill


guenther at rudersport

Jun 8, 2007, 4:56 PM

Post #12 of 40 (1270 views)
Permalink
Re: These are getting through SA... [In reply to]

On Fri, 2007-06-08 at 18:46 -0300, Luis Hernán Otegui wrote:
> OK, i?ve been googlin' around, and it seems like an issue between
> Amavis (or MailScanner, for waht I've found) and some unsupported
> versions of Net::DNS, because when I run the message through SA by
> itself, this comes out:

Whatever you manually fed SA was even more borked than the inline
copy-n-paste of a message in your OP. Looking briefly at your original
paste, I do see these:

> Date: Fri, 8 Jun 2007 20:25:53 -0100
> From: "Deana Adams" <budmedienkulturhaussow [at] medienkulturhaus>
> Message-ID: <969404800.15691885521876 [at] medienkulturhaus>
> To: internet [at] biol
> Subject: Can you imagine that you are healthy?

However, your manual run hit hard on...

> 0.0 MISSING_MID Missing Message-Id: header
> 0.0 MISSING_DATE Missing Date: header
> 1.3 MISSING_HEADERS Missing To: header
> 1.8 MISSING_SUBJECT Missing Subject: header
> 2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
> 0.5 FM_NO_TO FM_NO_TO

The "-1.8 ALL_TRUSTED" seems to support the assumption that you fed a
body only. Could be due to the exact details how you did it, though.
Also, this run didn't identify a HTML part at all...

The only difference that accounts for the spamminess in the second run
is the URIBL_BLACK hit. Maybe an oops, maybe a misconfiguration, maybe
due to not running in real time, but long after.

> So I'm blaming it on Amavis... (Net::DNS 0.59 here)...

I don't see much evidence for this, yet. ;)

guenther


--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}


luis.otegui at gmail

Jun 8, 2007, 6:24 PM

Post #13 of 40 (1267 views)
Permalink
Re: These are getting through SA... [In reply to]

What I copied and pasted into my message was the original spammy
message (the source of it) as IMP showed it. The posterior ALL_TRUSTED
occured because it has already been scanned and tagged by my servers.
But the main difference between the live run and the ones I did with
SA by itself (both as root and as user amavis) is the URIDNSBL hit.

Well, the blaming on Net::DNS wasn't an easy way out. I ran Amavis in
debug mode and spotted out some warnings about the use of (.) in
concatenation string in Util.pm (not literally, i'll post the correct
output on monday, when I get back to work). From this debug, I see
Amavis loading up the URIDNSBL plugin at startup, but lately it simply
doesn't fire up on any spammy link (I googled for them, since the DDoS
attack blocked the website).
Anyway, seems like my perl installation came out buggy (upgraded from
source to 5.8.8 before upgrading SA from 3.1.8 to 3.2.0), and it is
messing things up. Lately some errors with Net::SMTP came out when
reporting to SpamCop, so I guess I'll have to start it all over again
from scratch, but this time making sure all compiles ok.

Thanks,


Luis

2007/6/8, guenther <guenther [at] rudersport>:
> On Fri, 2007-06-08 at 18:46 -0300, Luis Hernán Otegui wrote:
> > OK, i?ve been googlin' around, and it seems like an issue between
> > Amavis (or MailScanner, for waht I've found) and some unsupported
> > versions of Net::DNS, because when I run the message through SA by
> > itself, this comes out:
>
> Whatever you manually fed SA was even more borked than the inline
> copy-n-paste of a message in your OP. Looking briefly at your original
> paste, I do see these:
>
> > Date: Fri, 8 Jun 2007 20:25:53 -0100
> > From: "Deana Adams" <budmedienkulturhaussow [at] medienkulturhaus>
> > Message-ID: <969404800.15691885521876 [at] medienkulturhaus>
> > To: internet [at] biol
> > Subject: Can you imagine that you are healthy?
>
> However, your manual run hit hard on...
>
> > 0.0 MISSING_MID Missing Message-Id: header
> > 0.0 MISSING_DATE Missing Date: header
> > 1.3 MISSING_HEADERS Missing To: header
> > 1.8 MISSING_SUBJECT Missing Subject: header
> > 2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
> > 0.5 FM_NO_TO FM_NO_TO
>
> The "-1.8 ALL_TRUSTED" seems to support the assumption that you fed a
> body only. Could be due to the exact details how you did it, though.
> Also, this run didn't identify a HTML part at all...
>
> The only difference that accounts for the spamminess in the second run
> is the URIBL_BLACK hit. Maybe an oops, maybe a misconfiguration, maybe
> due to not running in real time, but long after.
>
> > So I'm blaming it on Amavis... (Net::DNS 0.59 here)...
>
> I don't see much evidence for this, yet. ;)
>
> guenther
>
>
> --
> char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
> (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
>
>


--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------


Mark.Martinec at ijs

Jun 12, 2007, 3:53 AM

Post #14 of 40 (1240 views)
Permalink
Re: These are getting through SA... [In reply to]

Luis,

> I don't have any URIBL rules firing up (SA 3.2.0 from source here,
> most of the other relevant info is in the header of the mail I sent
> before to test). Where did you get them?
>[...]
> But the main difference between the live run and the ones I did with
> SA by itself (both as root and as user amavis) is the URIDNSBL hit.
>[...]
> From this debug, I see Amavis loading up the URIDNSBL plugin at startup,
> but lately it simply doesn't fire up on any spammy link (I googled
> for them, since the DDoS attack blocked the website).

I came across the same issue yesterday, with the same type
of a spam message, which would mostly get hits from URIBL tests,
but lots of other RBL checks come back emptyhanded.

On the first appearance it seems that SA under amavisd-new didn't
fire on DNSBL tests, but spamassassin from a command line did.

Investigating the problem more thoroughly turned out that even
a command line SA check behaved intermittently, sometimes
returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes
none of these URIBL tests - they were timing out.

What is your setting for rbl_timeout ?

Mine was fairly low, 5 seconds, and I find the dynamic timeout
(for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf)
does not work as advertised:

In addition, whenever the effective timeout is lowered due to addi-
tional query results returning, the remaining queries are always
given at least one more second before timing out

Namely with 22 RBL results coming back, the last one
(which was the crucial URIBL test) had a timeout of 0
and was ignored even though dns result did arrive.

Moreover, there is a bug in Mail::SpamAssassin::Dns, where
a late-spawned URIBL queries (which only start after Razor,
DCC and Pyzor are run) are being timed against start time
of the first wave of plain RBL dns queries, which are fired-off
seconds earlier, so there is a good chance that URIBL queries
time out in 0 seconds and their resultes are never collected.
The problem is made worse when for example Razor itself also
times out (thus extending time between the two rounds of
dns queries being sent).

Luis, check your DNS if it is responponding quickly,
try extending rbl_timeout to maybe 10 seconds, see if
there are many timeouts in RBL, URIBL, Razor or DCC queries.

Mark


luis.otegui at gmail

Jun 12, 2007, 7:49 AM

Post #15 of 40 (1244 views)
Permalink
Re: These are getting through SA... [In reply to]

Well, I dint't have rbl_timeout set, but after your mail, I did. The
DNSs I have set in resolv.conf are mine, they both cache and work as
internal and external resolvers. But the UNLP NOC got screwed in the
last days, so setting the timeout a little higher wont't hurt. Thanks
for the suggestion.
However, I upgraded to Amavis 2.5.1 yesterday (and rebuilt the AWL and
the Bayes SQL databases, because they got corrupted) and everythig
got back to normal. Updated several modules as Amavis required, and
everything got back to the usual behavior. URIBL rules got fired (on
several mails), and Razor and Pyzor got me results again.
Additionally, SA stopped complaining about some minor issues when
running sa-compile.

Thanks again,


Luix
2007/6/12, Mark Martinec <Mark.Martinec [at] ijs>:
> Luis,
>
> > I don't have any URIBL rules firing up (SA 3.2.0 from source here,
> > most of the other relevant info is in the header of the mail I sent
> > before to test). Where did you get them?
> >[...]
> > But the main difference between the live run and the ones I did with
> > SA by itself (both as root and as user amavis) is the URIDNSBL hit.
> >[...]
> > From this debug, I see Amavis loading up the URIDNSBL plugin at startup,
> > but lately it simply doesn't fire up on any spammy link (I googled
> > for them, since the DDoS attack blocked the website).
>
> I came across the same issue yesterday, with the same type
> of a spam message, which would mostly get hits from URIBL tests,
> but lots of other RBL checks come back emptyhanded.
>
> On the first appearance it seems that SA under amavisd-new didn't
> fire on DNSBL tests, but spamassassin from a command line did.
>
> Investigating the problem more thoroughly turned out that even
> a command line SA check behaved intermittently, sometimes
> returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes
> none of these URIBL tests - they were timing out.
>
> What is your setting for rbl_timeout ?
>
> Mine was fairly low, 5 seconds, and I find the dynamic timeout
> (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf)
> does not work as advertised:
>
> In addition, whenever the effective timeout is lowered due to addi-
> tional query results returning, the remaining queries are always
> given at least one more second before timing out
>
> Namely with 22 RBL results coming back, the last one
> (which was the crucial URIBL test) had a timeout of 0
> and was ignored even though dns result did arrive.
>
> Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> a late-spawned URIBL queries (which only start after Razor,
> DCC and Pyzor are run) are being timed against start time
> of the first wave of plain RBL dns queries, which are fired-off
> seconds earlier, so there is a good chance that URIBL queries
> time out in 0 seconds and their resultes are never collected.
> The problem is made worse when for example Razor itself also
> times out (thus extending time between the two rounds of
> dns queries being sent).
>
> Luis, check your DNS if it is responponding quickly,
> try extending rbl_timeout to maybe 10 seconds, see if
> there are many timeouts in RBL, URIBL, Razor or DCC queries.
>
> Mark
>


--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------


Mark.Martinec+sa at ijs

Jun 12, 2007, 9:19 AM

Post #16 of 40 (1242 views)
Permalink
Re: These are getting through SA... [In reply to]

Luis,

> > Namely with 22 RBL results coming back, the last one
> > (which was the crucial URIBL test) had a timeout of 0
> > and was ignored even though dns result did arrive.
> >
> > Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> > a late-spawned URIBL queries (which only start after Razor,
> > DCC and Pyzor are run) are being timed against start time
> > of the first wave of plain RBL dns queries, which are fired-off
> > seconds earlier, so there is a good chance that URIBL queries
> > time out in 0 seconds and their resultes are never collected.

I submitted a problem report, with a proposed patch:

http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511

Things are much more predictable now.

Mark


bill at inetmsg

Jun 12, 2007, 2:46 PM

Post #17 of 40 (1240 views)
Permalink
Re: These are getting through SA... [In reply to]

Mark Martinec wrote the following on 6/12/2007 3:53 AM -0800:
> Luis,
>
>
>> I don't have any URIBL rules firing up (SA 3.2.0 from source here,
>> most of the other relevant info is in the header of the mail I sent
>> before to test). Where did you get them?
>> [...]
>> But the main difference between the live run and the ones I did with
>> SA by itself (both as root and as user amavis) is the URIDNSBL hit.
>> [...]
>> From this debug, I see Amavis loading up the URIDNSBL plugin at startup,
>> but lately it simply doesn't fire up on any spammy link (I googled
>> for them, since the DDoS attack blocked the website).
>>
>
> I came across the same issue yesterday, with the same type
> of a spam message, which would mostly get hits from URIBL tests,
> but lots of other RBL checks come back emptyhanded.
>
> On the first appearance it seems that SA under amavisd-new didn't
> fire on DNSBL tests, but spamassassin from a command line did.
>
> Investigating the problem more thoroughly turned out that even
> a command line SA check behaved intermittently, sometimes
> returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes
> none of these URIBL tests - they were timing out.
>
> What is your setting for rbl_timeout ?
>
> Mine was fairly low, 5 seconds, and I find the dynamic timeout
> (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf)
> does not work as advertised:
>
> In addition, whenever the effective timeout is lowered due to addi-
> tional query results returning, the remaining queries are always
> given at least one more second before timing out
>
> Namely with 22 RBL results coming back, the last one
> (which was the crucial URIBL test) had a timeout of 0
> and was ignored even though dns result did arrive.
>
> Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> a late-spawned URIBL queries (which only start after Razor,
> DCC and Pyzor are run) are being timed against start time
> of the first wave of plain RBL dns queries, which are fired-off
> seconds earlier, so there is a good chance that URIBL queries
> time out in 0 seconds and their resultes are never collected.
> The problem is made worse when for example Razor itself also
> times out (thus extending time between the two rounds of
> dns queries being sent).
>
> Luis, check your DNS if it is responponding quickly,
> try extending rbl_timeout to maybe 10 seconds, see if
> there are many timeouts in RBL, URIBL, Razor or DCC queries.
>
> Mark
>
Mark, just curious if you are running Botnet? I found that some
messages cause the Botnet RDNS test to timeout after hanging for about
30 seconds, and then network test randomly fail (primarily URIBL
tests). I found that if I disable Botnet, then all network tests will
run fine on the very same messages.

Bill


prandal at herefordshire

Jun 12, 2007, 2:59 PM

Post #18 of 40 (1248 views)
Permalink
RE: These are getting through SA... [In reply to]

Well caught, Mark!

I'd come to similar conclusions even without digging into the code when
I saw DNS-related strangeness when I was testing SA 3.2.0 a few weeks
back.

I'll second your request that SA process all results it has collected on
timeout, instead of discarding them.

Cheers,

Phil

-----Original Message-----
From: Mark Martinec [mailto:Mark.Martinec+sa [at] ijs]
Sent: 12 June 2007 17:20
To: users [at] spamassassin
Subject: Re: These are getting through SA...

Luis,

> > Namely with 22 RBL results coming back, the last one
> > (which was the crucial URIBL test) had a timeout of 0
> > and was ignored even though dns result did arrive.
> >
> > Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> > a late-spawned URIBL queries (which only start after Razor,
> > DCC and Pyzor are run) are being timed against start time
> > of the first wave of plain RBL dns queries, which are fired-off
> > seconds earlier, so there is a good chance that URIBL queries
> > time out in 0 seconds and their resultes are never collected.

I submitted a problem report, with a proposed patch:

http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511

Things are much more predictable now.

Mark


Mark.Martinec+sa at ijs

Jun 12, 2007, 3:05 PM

Post #19 of 40 (1243 views)
Permalink
Re: These are getting through SA... [In reply to]

Bill,

> Mark, just curious if you are running Botnet? I found that some
> messages cause the Botnet RDNS test to timeout after hanging for about
> 30 seconds, and then network test randomly fail (primarily URIBL
> tests). I found that if I disable Botnet, then all network tests will
> run fine on the very same messages.

Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc.,
before the first and the second round of DNS launches). Please try the patch
attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
(applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too.

Mark


prandal at herefordshire

Jun 12, 2007, 3:07 PM

Post #20 of 40 (1237 views)
Permalink
RE: These are getting through SA... [In reply to]

Bill,

I was getting this sort of symptom without using Botnet.

It's almost as if something's deadlocking somewhere in SA (until the
timeout kicks in).

Phil

-----Original Message-----
From: Bill Landry [mailto:bill [at] inetmsg]
Sent: 12 June 2007 22:47
To: users [at] spamassassin
Subject: Re: These are getting through SA...

Mark, just curious if you are running Botnet? I found that some
messages cause the Botnet RDNS test to timeout after hanging for about
30 seconds, and then network test randomly fail (primarily URIBL
tests). I found that if I disable Botnet, then all network tests will
run fine on the very same messages.

Bill


bill at inetmsg

Jun 12, 2007, 3:30 PM

Post #21 of 40 (1242 views)
Permalink
Re: These are getting through SA... [In reply to]

Mark Martinec wrote the following on 6/12/2007 3:05 PM -0800:
> Bill,
>
>
>> Mark, just curious if you are running Botnet? I found that some
>> messages cause the Botnet RDNS test to timeout after hanging for about
>> 30 seconds, and then network test randomly fail (primarily URIBL
>> tests). I found that if I disable Botnet, then all network tests will
>> run fine on the very same messages.
>>
>
> Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc.,
> before the first and the second round of DNS launches). Please try the patch
> attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
> (applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too.
>
> Mark
>
Mark, I patched Dns.pm but this didn't resolve the issue for me. You
can test with the sample messages I posted to bugzilla:

http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506

The only way I can get the URIBL tests to report hits it to run the
messages through SA without the headers (samples without headers also
posted to the bugzilla).

Bill


Mark.Martinec+sa at ijs

Jun 12, 2007, 4:29 PM

Post #22 of 40 (1248 views)
Permalink
Re: These are getting through SA... [In reply to]

Bill,

> Mark, I patched Dns.pm but this didn't resolve the issue for me.
> You can test with the sample messages I posted to bugzilla:
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506

Yes, it is the same problem as I describe in
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
but to fix it requires my 'feature request' to be implemented:

Now for the last part - a feature request. I think that no attempt
has been made to collect already received DNS responses when
timeout is reached. Given an asynchronous nature of DNS lookups
in this module, I think it would be worthwhile to collect whatever
is still in the IP receive queue after a timeout.



The problem with BOTNET is that it tries to do a reverse DNS
lookup on 66.17.235.109, which has broken DNS servers and
none are reachable, so it hangs in sub get_rdns query($ip,'PTR','IN')
for 24 seconds.

After BOTNET finally times out, the Dns.pm harvest_dnsbl_queries
tries to collect its RBL results, but abandons all attempts right away
because it sees that 24 seconds has passed by and just declares
them timed out, despite the fact that DNS results are waiting
in TCP/IP received queue.

Mark


prandal at herefordshire

Jun 13, 2007, 12:56 AM

Post #23 of 40 (1240 views)
Permalink
RE: These are getting through SA... [In reply to]

What happens if Botnet is patched to use Mail::SpamAssassin::DnsResolver
instead of Net::DNS::Resolver?

I'm musuing about Net::DNS::Resolver's default timeouts and retries...

Phil (probably barking up the wrong tree)
--
Phil Randal
Network Engineer
Herefordshire Council
Hereford, UK

> -----Original Message-----
> From: Bill Landry [mailto:bill [at] inetmsg]
> Sent: 12 June 2007 23:30
> To: users [at] spamassassin
> Subject: Re: These are getting through SA...
>
> Mark Martinec wrote the following on 6/12/2007 3:05 PM -0800:
> > Bill,
> >
> >
> >> Mark, just curious if you are running Botnet? I found that some
> >> messages cause the Botnet RDNS test to timeout after
> hanging for about
> >> 30 seconds, and then network test randomly fail (primarily URIBL
> >> tests). I found that if I disable Botnet, then all
> network tests will
> >> run fine on the very same messages.
> >>
> >
> > Thanks, looks like the same cause (Botnet runs with Razor,
> dcc, etc.,
> > before the first and the second round of DNS launches).
> Please try the patch
> > attached to
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
> > (applies to SA 2.3.1 or 2.3.0), it is likely to fix these
> symptoms too.
> >
> > Mark
> >
> Mark, I patched Dns.pm but this didn't resolve the issue for me. You
> can test with the sample messages I posted to bugzilla:
>
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506
>
> The only way I can get the URIBL tests to report hits it to run the
> messages through SA without the headers (samples without headers also
> posted to the bugzilla).
>
> Bill
>


Mark.Martinec+sa at ijs

Jun 13, 2007, 2:08 AM

Post #24 of 40 (1240 views)
Permalink
Re: These are getting through SA... [In reply to]

Phil,

> What happens if Botnet is patched to use Mail::SpamAssassin::DnsResolver
> instead of Net::DNS::Resolver?
> I'm musuing about Net::DNS::Resolver's default timeouts and retries...
> Phil (probably barking up the wrong tree)

It would do good if Botnet would impose a time limit on its DNS queries.
It would also sidestep the Dns.pm problem, but not fix it.

If the time spent by Razor+dcc+Botnet+(not sure what else)
is longer than rbl_timeout, then replies to RBL queries are
thrown away by mistake.

Mark


Mark.Martinec+sa at ijs

Jun 15, 2007, 3:36 AM

Post #25 of 40 (1235 views)
Permalink
Re: These are getting through SA... [In reply to]

Phil, Bill,

> Mark, I patched Dns.pm but this didn't resolve the issue for me.
> You can test with the sample messages I posted to bugzilla:
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506

> I was getting this sort of symptom without using Botnet.
> It's almost as if something's deadlocking somewhere in SA
> (until the timeout kicks in).

>> If the time spent by Razor+dcc+Botnet+(not sure what else)
>> is longer than rbl_timeout, then replies to RBL queries are
>> thrown away by mistake.

There is now an additional patch at:
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
which should fix this.

Mark

First page Previous page 1 2 Next page Last page  View All SpamAssassin users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.