Mailing List Archive: SpamAssassin: users

Rule to match X-Spam-Flag

Index | Next | Previous | View Threaded

sandro at e-den

Jun 9, 2011, 2:09 AM

Post #1 of 15 (343 views)
Permalink

Rule to match X-Spam-Flag

Hi,

I find a lot of spam that has already passed other spam-filters with
spamassassin better tuned than mine an already have a X-Spam-Flag to YES.

I tried to add a rule to match that case:

header CUSTOM_X_SPAM_FLAG X-Spam-Flag =~ /\bYES\b/i
score CUSTOM_X_SPAM_FLAG 5

But spamassassin -t < /tmp/spam does not show any hit ot that rule. Moreover
using flag -D I don't see it being called. I set it in /etc/spamassassin/local.cf

Is it any possible to match on that rule?

TIA
sandro
*:-)

--
Sandro Dentella *:-)
http://www.reteisi.org Soluzioni libere per le scuole
http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy

Mark.Martinec+sa at ijs

Jun 9, 2011, 2:46 AM

Post #2 of 15 (329 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

Sandro,

> I find a lot of spam that has already passed other spam-filters with
> spamassassin better tuned than mine an already have a X-Spam-Flag to YES.
>
> I tried to add a rule to match that case:
>
> header CUSTOM_X_SPAM_FLAG X-Spam-Flag =~ /\bYES\b/i
> score CUSTOM_X_SPAM_FLAG 5
>
> But spamassassin -t < /tmp/spam does not show any hit ot that rule.
> Moreover using flag -D I don't see it being called. I set it in
> /etc/spamassassin/local.cf
>
> Is it any possible to match on that rule?

It is an unfortunate consequence of a M::S::PerMsgStatus::check()
removing any 'x-spam-*' header fileds _before_ performing any checks.
It would probably make more sense to do so after checks but before
collecting a report or a rewritten message. I'm just not sure what
other code or rules depend on this, so fixing your case might
break something else (or may not, needs investigating).
You may open a problem report.

As a workaround, you may add some header rewrite rule to your MTA
which could rewrite a X-Spam-Flag to something else, like X-X-Spam-Flag.

Mark

axb.lists at gmail

Jun 9, 2011, 2:50 AM

Post #3 of 15 (330 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On 2011-06-09 11:46, Mark Martinec wrote:
> Sandro,
>
>> I find a lot of spam that has already passed other spam-filters with
>> spamassassin better tuned than mine an already have a X-Spam-Flag to YES.
>>
>> I tried to add a rule to match that case:
>>
>> header CUSTOM_X_SPAM_FLAG X-Spam-Flag =~ /\bYES\b/i
>> score CUSTOM_X_SPAM_FLAG 5
>>
>> But spamassassin -t< /tmp/spam does not show any hit ot that rule.
>> Moreover using flag -D I don't see it being called. I set it in
>> /etc/spamassassin/local.cf
>>
>> Is it any possible to match on that rule?
>
> It is an unfortunate consequence of a M::S::PerMsgStatus::check()
> removing any 'x-spam-*' header fileds _before_ performing any checks.
> It would probably make more sense to do so after checks but before
> collecting a report or a rewritten message. I'm just not sure what
> other code or rules depend on this, so fixing your case might
> break something else (or may not, needs investigating).
> You may open a problem report.
>
> As a workaround, you may add some header rewrite rule to your MTA
> which could rewrite a X-Spam-Flag to something else, like X-X-Spam-Flag.

or if you want to be rather radical, reject at MTA level with a header
check.

Motto: "Dear Sender: if you pre-tag your mail as spam, keeep it"

me at junc

Jun 9, 2011, 7:43 AM

Post #4 of 15 (329 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Thu, 9 Jun 2011 11:46:41 +0200, Mark Martinec wrote:

> As a workaround, you may add some header rewrite rule to your MTA
> which could rewrite a X-Spam-Flag to something else, like
> X-X-Spam-Flag.

will not give invalid dkim ?

seems aol.com does not use amavisd-new, if thay did this header was not
in there mails, i consider make it a clamav signature and run virus
scanning in proxy to reject it, or let postfix reject it

Mark.Martinec+sa at ijs

Jun 9, 2011, 8:06 AM

Post #5 of 15 (328 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

Benny,

> > As a workaround, you may add some header rewrite rule to your MTA
> > which could rewrite a X-Spam-Flag to something else, like
> > X-X-Spam-Flag.
>
> will not give invalid dkim ?

No, unless the X-Spam-Flag were signed, which is unlikely.

Mark

joseph.sniderman at thoroquel

Jun 9, 2011, 8:37 AM

Post #6 of 15 (328 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On 06/09/2011 11:06 AM, Mark Martinec wrote:
> Benny,
>
>>> As a workaround, you may add some header rewrite rule to your MTA
>>> which could rewrite a X-Spam-Flag to something else, like
>>> X-X-Spam-Flag.
>>
>> will not give invalid dkim ?
>
> No, unless the X-Spam-Flag were signed, which is unlikely.

Even so, one could add (instead of rewriting) an X-X-Spam-Flag or
X-Original-Spam-Flag or whatever, while leaving the X-Spam-Flag intact
and in place. That way, even if for some reason the X-Spam-Flag were
signed, DKIM would be unaffected. Or one could perform DKIM
verification first [1], then re-write the header, then pass the mail to
spamassassin.

[1] using opendkim or dkim-filter or whatever. not sure if spamassassin
will use that result or perform its own verification, but either way if
the goal is to tag, so what if spamassassin also sees a DKIM failure. if
humans want to know that it passed for whatever reason, the
authentication-results header would still be there.

--
Joe Sniderman <joseph.sniderman [at] thoroquel>

spamassassin at dostech

Jun 9, 2011, 7:00 PM

Post #7 of 15 (315 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On 09/06/2011 5:09 AM, Alessandro Dentella wrote:
>
>
> Hi,
>
>
> I find a lot of spam that has already passed other spam-filters with
> spamassassin better tuned than mine an already have a X-Spam-Flag to YES.
>
> I tried to add a rule to match that case:
>
> header CUSTOM_X_SPAM_FLAG X-Spam-Flag =~ /\bYES\b/i
> score CUSTOM_X_SPAM_FLAG 5
>
> But spamassassin -t< /tmp/spam does not show any hit ot that rule. Moreover
> using flag -D I don't see it being called. I set it in /etc/spamassassin/local.cf
>
> Is it any possible to match on that rule?

Have you tried matching against the ALL:raw header? I don't think it
will work but I can't remember for sure.

header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i

A custom plugin may also be able to use get_pristine_header() to match
against what you want to match against. Again, though, I can't remember
for sure when/where the X-Spam headers get stripped out.

Daryl

me at junc

Jun 9, 2011, 7:08 PM

Post #8 of 15 (317 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Thu, 09 Jun 2011 22:00:09 -0400, Daryl C. W. O'Shea wrote:

> header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i

aol have left out the space before YES

will test it and report back, thanks for this tip if it works :-)

me at junc

Jun 9, 2011, 7:26 PM

Post #9 of 15 (317 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Fri, 10 Jun 2011 04:08:08 +0200, Benny Pedersen wrote:
> On Thu, 09 Jun 2011 22:00:09 -0400, Daryl C. W. O'Shea wrote:
>
>> header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i
>
> aol have left out the space before YES
>
> will test it and report back, thanks for this tip if it works :-)

# header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i
# aol forget the space before YES
# does the other versions exists ?
header X_SPAM_FLAG_YES ALL:raw =~ /\bX-Spam-Flag:YES\b/i
describe X_SPAM_FLAG_YES Header: says its spam
score X_SPAM_FLAG_YES 1.5

tested and works

spamassassin at dostech

Jun 9, 2011, 7:38 PM

Post #10 of 15 (316 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On 09/06/2011 10:26 PM, Benny Pedersen wrote:
> On Fri, 10 Jun 2011 04:08:08 +0200, Benny Pedersen wrote:
>> On Thu, 09 Jun 2011 22:00:09 -0400, Daryl C. W. O'Shea wrote:
>>
>>> header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i
>>
>> aol have left out the space before YES
>>
>> will test it and report back, thanks for this tip if it works :-)
>
> # header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i
> # aol forget the space before YES
> # does the other versions exists ?
> header X_SPAM_FLAG_YES ALL:raw =~ /\bX-Spam-Flag:YES\b/i
> describe X_SPAM_FLAG_YES Header: says its spam
> score X_SPAM_FLAG_YES 1.5
>
> tested and works

Good. This may work then, too:

header CUSTOM_X_SPAM_FLAG X-Spam-Flag:raw =~ /\bYES\b/i

Daryl

me at junc

Jun 10, 2011, 12:01 AM

Post #11 of 15 (315 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Thu, 09 Jun 2011 22:38:25 -0400, Daryl C. W. O'Shea wrote:

> header CUSTOM_X_SPAM_FLAG X-Spam-Flag:raw =~ /\bYES\b/i

nope, is headers case sensitive ?

spamassassin have case like the above, but aol changed it all
uppercase, and my rule works from spamassassin, but fails in amavis
since this header is removed case insensitive, what a life :-)

sandro at e-den

Jun 10, 2011, 2:29 AM

Post #12 of 15 (319 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Thu, Jun 09, 2011 at 10:38:25PM -0400, Daryl C. W. O'Shea wrote:
> On 09/06/2011 10:26 PM, Benny Pedersen wrote:
> >On Fri, 10 Jun 2011 04:08:08 +0200, Benny Pedersen wrote:
> >>On Thu, 09 Jun 2011 22:00:09 -0400, Daryl C. W. O'Shea wrote:
> >>
> >>>header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i
> >>
> >>aol have left out the space before YES
> >>
> >>will test it and report back, thanks for this tip if it works :-)
> >
> ># header CUSTOM_X_SPAM_FLAG ALL:raw =~ /\bX-Spam-Flag: YES\b/i
> ># aol forget the space before YES
> ># does the other versions exists ?
> >header X_SPAM_FLAG_YES ALL:raw =~ /\bX-Spam-Flag:YES\b/i
> >describe X_SPAM_FLAG_YES Header: says its spam
> >score X_SPAM_FLAG_YES 1.5
> >
> >tested and works
>
> Good. This may work then, too:
>
> header CUSTOM_X_SPAM_FLAG X-Spam-Flag:raw =~ /\bYES\b/i

I can't test this now but yesterday I implemented with postfix the suggested
version with X-Original-Spam-Flag and it just worked:

cat /etc/postfix/header_checks

/^X-Spam-Flag: YES/ PREPEND X-Original-Spam-Flag: YES
/^Received:/ HOLD

fw-omma:~# grep CUSTOM_X /etc/spamassassin/local.cf

header CUSTOM_X_SPAM_FLAG X-Original-Spam-Flag =~ /\bYES\b/i
score CUSTOM_X_SPAM_FLAG 2

thanks to all of you for the great support.

sandro
*:-)

--
Sandro Dentella *:-)
http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy

Mark.Martinec+sa at ijs

Jun 10, 2011, 3:29 AM

Post #13 of 15 (315 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

Benny,

> On Thu, 09 Jun 2011 22:38:25 -0400, Daryl C. W. O'Shea wrote:
> > header CUSTOM_X_SPAM_FLAG X-Spam-Flag:raw =~ /\bYES\b/i

Can't work, the M::S::PerMsgStatus::check_timed calls:

$self->{msg}->delete_header('X-Spam-.*');

before invoking any checks. Moving that call further down
makes it work.

> nope, is headers case sensitive ?

Header field names are case-insensitive as per RFC 5322,
and are treated as such by SpamAssassin and by amavis.

> spamassassin have case like the above, but aol changed it all
> uppercase, and my rule works from spamassassin, but fails in amavis
> since this header is removed case insensitive, what a life :-)

Strange. A message passed from amavis to SpamAssassin is
pristine, no header fields are removed or changed, just some
header fields are prepended (like a Return-Path and some
X-Amavis-* informational additions).

The X-Spam-* header fields are deleted by amavisd on-the-fly
as a last step when a mesasge is fed back to a MTA. So SA would
not see these header fields only if SA were invoked separately
and after amavisd, which is not normally the case.

Mark

me at junc

Jun 10, 2011, 1:52 PM

Post #14 of 15 (309 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Fri, 10 Jun 2011 11:29:23 +0200, Alessandro Dentella wrote:

> thanks to all of you for the great support.

super duper will test this in postfix, just a shame one need to turn
postfix into a content filter just to make it work :(

sandro at e-den

Jun 14, 2011, 2:57 AM

Post #15 of 15 (295 views)
Permalink

Re: Rule to match X-Spam-Flag [In reply to]

On Fri, Jun 10, 2011 at 10:52:25PM +0200, Benny Pedersen wrote:
> On Fri, 10 Jun 2011 11:29:23 +0200, Alessandro Dentella wrote:
>
> >thanks to all of you for the great support.
>
> super duper will test this in postfix, just a shame one need to turn
> postfix into a content filter just to make it work :(

that's really already the way I setup postfix to work with MailScanner.
A rule set it on hold and Mailwatch picks it from the queue and process it.

This is just one more row in the header filter...

sandro
*:-)

--
Sandro Dentella *:-)
http://www.reteisi.org Soluzioni libere per le scuole
http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads