Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: SpamAssassin: users

Bayes now changed to autolearn=unavailable.

 

 

SpamAssassin users RSS feed   Index | Next | Previous | View Threaded


simon at klunky

Feb 27, 2012, 2:48 AM

Post #1 of 7 (737 views)
Permalink
Bayes now changed to autolearn=unavailable.

Good morning everyone,

Recently I enabled shortcircuit for ham on a server because Bayes
seems reasonably well trained. It works well.

I noticed that emails that did not hit BAYES_00 (so no shortcircuit)
were not autolearnt by SA. Even though these were well below the
autolearn threshold of -1.

In the example below, the score was -7.8. Below this, the bayes
settings in the local.cf are listed. A score of beneath -1 should have
been autolearnt.

No, score=-7.8 required=5.0 tests=DCC_CHECK,RCVD_IN_DNSWL_HI,
RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_SAFE,SPF_PASS,T_RP_MATCHES_RCVD,URI_HEX
shortcircuit=no autolearn=unavailable version=3.3.1

# grep -i bayes local.cf|grep -v ^#
use_bayes 1
bayes_auto_learn 1
bayes_sql_override_username spamd
bayes_expiry_max_db_size 500000
bayes_auto_expire 1
bayes_auto_learn_threshold_nonspam -1
bayes_auto_learn_threshold_spam 13.0
bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn DBI:mysql:spamassassin:localhost
bayes_sql_username xxxxxxxxx
bayes_sql_password xxxxxxxxx
bayes_ignore_to users [at] spamassassin
shortcircuit BAYES_00 ham

Ham that shortcircuits has autolearn=disabled, which makes sense to me
as it is already in Bayes.

Is there something wrong with mysql on this server that may have caused
the problem, or is a SA configuration problem?


Regards, S.

--
PGP is optional: 4BA78604
simon @ klunky . org
simon @ klunky . co.uk
I won't accept your confidentiality
agreement, and your Emails are kept.
~ֿ~


michael.scheidell at secnap

Feb 27, 2012, 3:41 AM

Post #2 of 7 (660 views)
Permalink
Re: Bayes now changed to autolearn=unavailable. [In reply to]

On 2/27/12 5:48 AM, Simon Loewenthal wrote:
> seems reasonably well trained. It works well.
>
> I noticed that emails that did not hit BAYES_00 (so no shortcircuit)
> were not autolearnt by SA. Even though these were well below the
> autolearn threshold of -1.
>
> In the example below, the score was -7.8. Below this, the bayes
> settings in the local.cf are listed. A score of beneath -1 should have
> been autolearnt.
> DCC_CHECK,RCVD_IN_DNSWL_HI,
> RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_SAFE,SPF_PASS,T_RP_MATCHES_RCVD,URI_HEX
>
for each rule triggered, does it have an autolearn flag? you need enough
rules that together score below -1 and do NOT have noautolearn flags.




--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator

______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________


rwmaillists at googlemail

Feb 27, 2012, 4:55 AM

Post #3 of 7 (658 views)
Permalink
Re: Bayes now changed to autolearn=unavailable. [In reply to]

On Mon, 27 Feb 2012 11:48:50 +0100
Simon Loewenthal wrote:


> Recently I enabled shortcircuit for ham on a server because Bayes
> seems reasonably well trained. It works well.
>
> I noticed that emails that did not hit BAYES_00 (so no shortcircuit)
> were not autolearnt by SA. Even though these were well below the
> autolearn threshold of -1.
>
> In the example below, the score was -7.8. Below this, the bayes
> settings in the local.cf are listed. A score of beneath -1 should
> have been autolearnt.
>
> No, score=-7.8 required=5.0 tests=DCC_CHECK,RCVD_IN_DNSWL_HI,
> RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_SAFE,SPF_PASS,T_RP_MATCHES_RCVD,URI_HEX
> shortcircuit=no autolearn=unavailable version=3.3.1
> ...
> Ham that shortcircuits has autolearn=disabled, which makes sense to me
> as it is already in Bayes.

Correctly identified isn't the same as "in Bayes", it's presumably
because its not considered safe to autolearn on a incomplete set
of rules.

autolearn=unavailable means that there was some problem with
accessing Bayes - note that there is no BAYES_* hit at all in the test
list, it's not just that it lacks BAYES_00.

I think this might be due to an auto-expiry. Try setting
"bayes_auto_expire 0" and running "sa-learn --force-expire" from
cron in the middle of the night. This is considered best-practice in
any case.


simon at klunky

Feb 27, 2012, 6:24 AM

Post #4 of 7 (658 views)
Permalink
Re: Bayes now changed to autolearn=unavailable. [In reply to]

On 27/02/12 13:55, RW wrote:
> On Mon, 27 Feb 2012 11:48:50 +0100
> Simon Loewenthal wrote:
>
>
>> Recently I enabled shortcircuit for ham on a server because Bayes
>> seems reasonably well trained. It works well.
>>
>> I noticed that emails that did not hit BAYES_00 (so no shortcircuit)
>> were not autolearnt by SA. Even though these were well below the
>> autolearn threshold of -1.
>>
>> In the example below, the score was -7.8. Below this, the bayes
>> settings in the local.cf are listed. A score of beneath -1 should
>> have been autolearnt.
>>
>> No, score=-7.8 required=5.0 tests=DCC_CHECK,RCVD_IN_DNSWL_HI,
>> RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_SAFE,SPF_PASS,T_RP_MATCHES_RCVD,URI_HEX
>> shortcircuit=no autolearn=unavailable version=3.3.1
>> ...
>> Ham that shortcircuits has autolearn=disabled, which makes sense to me
>> as it is already in Bayes.
> Correctly identified isn't the same as "in Bayes", it's presumably
> because its not considered safe to autolearn on a incomplete set
> of rules.
>
> autolearn=unavailable means that there was some problem with
> accessing Bayes - note that there is no BAYES_* hit at all in the test
> list, it's not just that it lacks BAYES_00.
>
> I think this might be due to an auto-expiry. Try setting
> "bayes_auto_expire 0" and running "sa-learn --force-expire" from
> cron in the middle of the night. This is considered best-practice in
> any case.
BAYES_00 and BAYES_99 hit on other ham/spam without any problem. Only
then the result will mention autolearn=disabled instead of
autolearn=unavailable. I don't follow why it would be unavailable for
non-shortcirtcuited emails, but disabled for shortcircuited emails.
It has to have accessed the Bayes dB in order to score with BAYES_00/50/99.

Best time me to run sa-learn --force-expire is this evening, as only US
time zones are using the server. The main users are in Oceania and the EU.

Cheers,
S

--
PGP is optional: 4BA78604
simon @ klunky . org
simon @ klunky . co.uk
I won't accept your confidentiality
agreement, and your Emails are kept.
~ֿ~


simon at klunky

Feb 27, 2012, 6:29 AM

Post #5 of 7 (654 views)
Permalink
Re: Bayes now changed to autolearn=unavailable. [In reply to]

On 27/02/12 15:24, Simon Loewenthal wrote:
> On 27/02/12 13:55, RW wrote:
>> On Mon, 27 Feb 2012 11:48:50 +0100
>> Simon Loewenthal wrote:
>>
>>
>>> Recently I enabled shortcircuit for ham on a server because Bayes
>>> seems reasonably well trained. It works well.
>>>
>>> I noticed that emails that did not hit BAYES_00 (so no shortcircuit)
>>> were not autolearnt by SA. Even though these were well below the
>>> autolearn threshold of -1.
>>>
>>> In the example below, the score was -7.8. Below this, the bayes
>>> settings in the local.cf are listed. A score of beneath -1 should
>>> have been autolearnt.
>>>
>>> No, score=-7.8 required=5.0 tests=DCC_CHECK,RCVD_IN_DNSWL_HI,
>>> RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_SAFE,SPF_PASS,T_RP_MATCHES_RCVD,URI_HEX
>>> shortcircuit=no autolearn=unavailable version=3.3.1
>>> ...
>>> Ham that shortcircuits has autolearn=disabled, which makes sense to me
>>> as it is already in Bayes.
>> Correctly identified isn't the same as "in Bayes", it's presumably
>> because its not considered safe to autolearn on a incomplete set
>> of rules.
>>
>> autolearn=unavailable means that there was some problem with
>> accessing Bayes - note that there is no BAYES_* hit at all in the test
>> list, it's not just that it lacks BAYES_00.
>>
>> I think this might be due to an auto-expiry. Try setting
>> "bayes_auto_expire 0" and running "sa-learn --force-expire" from
>> cron in the middle of the night. This is considered best-practice in
>> any case.
> BAYES_00 and BAYES_99 hit on other ham/spam without any problem. Only
> then the result will mention autolearn=disabled instead of
> autolearn=unavailable. I don't follow why it would be unavailable for
> non-shortcirtcuited emails, but disabled for shortcircuited emails.
> It has to have accessed the Bayes dB in order to score with
> BAYES_00/50/99.
>
> Best time me to run sa-learn --force-expire is this evening, as only
> US time zones are using the server. The main users are in Oceania and
> the EU.
>
> Cheers,
> S
>
> --
Hi RW,

Being impatient, I just ran the autoexpire. New Email still has the
same disabled or unavailable message, so no change.

# /etc/init.d/spamassassin restart
Restarting SpamAssassin Mail Filter Daemon: spamd.
# sa-learn --force-expire
# sa-learn --force-expire --username=spamd (did it with the
username=spamd specified as I forgot above)
#
# sa-learn --dump magic
0.000 0 3 0 non-token data: bayes db version
0.000 0 1555 0 non-token data: nspam
0.000 0 14622 0 non-token data: nham
0.000 0 527466 0 non-token data: ntokens
0.000 0 1308090923 0 non-token data: oldest atime
0.000 0 1330350454 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal
sync atime
0.000 0 1330352706 0 non-token data: last expiry atime
0.000 0 22118400 0 non-token data: last expire
atime delta
0.000 0 1041 0 non-token data: last expire
reduction count


simon at klunky

Feb 27, 2012, 7:17 AM

Post #6 of 7 (654 views)
Permalink
Re: Bayes now changed to autolearn=unavailable. [In reply to]

On 27/02/12 15:29, Simon Loewenthal wrote:
> On 27/02/12 15:24, Simon Loewenthal wrote:
>> On 27/02/12 13:55, RW wrote:
>>> On Mon, 27 Feb 2012 11:48:50 +0100
>>> Simon Loewenthal wrote:
>>>
>>>
>>>> Recently I enabled shortcircuit for ham on a server because Bayes
>>>> seems reasonably well trained. It works well.
>>>>
>>>> I noticed that emails that did not hit BAYES_00 (so no shortcircuit)
>>>> were not autolearnt by SA. Even though these were well below the
>>>> autolearn threshold of -1.
>>>>
>>>> In the example below, the score was -7.8. Below this, the bayes
>>>> settings in the local.cf are listed. A score of beneath -1 should
>>>> have been autolearnt.
>>>>
>>>> No, score=-7.8 required=5.0 tests=DCC_CHECK,RCVD_IN_DNSWL_HI,
>>>> RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_SAFE,SPF_PASS,T_RP_MATCHES_RCVD,URI_HEX
>>>> shortcircuit=no autolearn=unavailable version=3.3.1
>>>> ...
>>>> Ham that shortcircuits has autolearn=disabled, which makes sense to me
>>>> as it is already in Bayes.
>>> Correctly identified isn't the same as "in Bayes", it's presumably
>>> because its not considered safe to autolearn on a incomplete set
>>> of rules.
>>>
>>> autolearn=unavailable means that there was some problem with
>>> accessing Bayes - note that there is no BAYES_* hit at all in the test
>>> list, it's not just that it lacks BAYES_00.
>>>
>>> I think this might be due to an auto-expiry. Try setting
>>> "bayes_auto_expire 0" and running "sa-learn --force-expire" from
>>> cron in the middle of the night. This is considered best-practice in
>>> any case.
>> BAYES_00 and BAYES_99 hit on other ham/spam without any problem. Only
>> then the result will mention autolearn=disabled instead of
>> autolearn=unavailable. I don't follow why it would be unavailable for
>> non-shortcirtcuited emails, but disabled for shortcircuited emails.
>> It has to have accessed the Bayes dB in order to score with
>> BAYES_00/50/99.
>>
>> Best time me to run sa-learn --force-expire is this evening, as only
>> US time zones are using the server. The main users are in Oceania and
>> the EU.
>>
>> Cheers,
>> S
>>
>> --
> Hi RW,
>
> Being impatient, I just ran the autoexpire. New Email still has the
> same disabled or unavailable message, so no change.
>
> # /etc/init.d/spamassassin restart
> Restarting SpamAssassin Mail Filter Daemon: spamd.
> # sa-learn --force-expire
> # sa-learn --force-expire --username=spamd (did it with the
> username=spamd specified as I forgot above)
> #
> # sa-learn --dump magic
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 1555 0 non-token data: nspam
> 0.000 0 14622 0 non-token data: nham
> 0.000 0 527466 0 non-token data: ntokens
> 0.000 0 1308090923 0 non-token data: oldest atime
> 0.000 0 1330350454 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal
> sync atime
> 0.000 0 1330352706 0 non-token data: last expiry atime
> 0.000 0 22118400 0 non-token data: last expire
> atime delta
> 0.000 0 1041 0 non-token data: last expire
> reduction count
>
I just noticed that something came through and was learnt :)

No, score=-6.9 required=5.0 tests=BAYES_00,HTML_MESSAGE,
RCVD_IN_DNSWL_HI,SPF_HELO_PASS,SPF_PASS,T_FRT_COCK,T_KHOP_FOREIGN_CLICK,
T_REMOTE_IMAGE,T_RP_MATCHES_RCVD shortcircuit=no autolearn=ham version=3.3.1

Fantastic.

--
PGP is optional: 4BA78604
simon @ klunky . org
simon @ klunky . co.uk
I won't accept your confidentiality
agreement, and your Emails are kept.
~ֿ~


rwmaillists at googlemail

Feb 27, 2012, 9:04 AM

Post #7 of 7 (666 views)
Permalink
Re: Bayes now changed to autolearn=unavailable. [In reply to]

On Mon, 27 Feb 2012 16:17:49 +0100
Simon Loewenthal wrote:


> >> BAYES_00 and BAYES_99 hit on other ham/spam without any problem.
> >> Only then the result will mention autolearn=disabled instead of
> >> autolearn=unavailable. I don't follow why it would be unavailable
> >> for non-shortcirtcuited emails, but disabled for shortcircuited
> >> emails. It has to have accessed the Bayes dB in order to score with
> >> BAYES_00/50/99.

If bayes is unavailible shortcircuiting on BAYES_00 isn't possible

Shortcircuited mail is never autolearned, so it shows as
autolearn=disabled


> I just noticed that something came through and was learnt :)
>
> No, score=-6.9 required=5.0 tests=BAYES_00,HTML_MESSAGE,
> RCVD_IN_DNSWL_HI,SPF_HELO_PASS,SPF_PASS,T_FRT_COCK,T_KHOP_FOREIGN_CLICK,
> T_REMOTE_IMAGE,T_RP_MATCHES_RCVD shortcircuit=no autolearn=ham
> version=3.3.1
>
> Fantastic.

Not really, you have shortcircuit=no

SpamAssassin users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.