Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: exim: users

X-Spam-Report for Clean messages

 

 

exim users RSS feed   Index | Next | Previous | View Threaded


gb-mail at sbgnet

May 2, 2008, 11:15 AM

Post #1 of 5 (131 views)
Permalink
X-Spam-Report for Clean messages

I have a need for more detail in my log. We are using exim as an
inbound relay and pre-filtering before delivering to our mail system.

The reject.log file is very useful but would be more helpful if it had
CLEAN messages logged also. Now obviously CLEAN messages should not be
part of reject.log so the best solution would be a log just for
X-Spam-Report (SPAM and CLEAN)

Searching for these terms is pretty rough since they are in so many conf
files uploaded....

--snip--
warn add_header = X-Spam-Score: $spam_score ($spam_bar)\n\
X-Spam-Report: $spam_report
deny condition = ${if >{$spam_score_int}{90} {1}}
message = Your message was RejecteD as Spam
accept
--snip--

I know log size will increase, but I do not expect the logs to grow more
than %33 considering yesterday we
rejected 167,000 Unknown User
rejected 133,000 spam
and
delivered 91,000 messages




--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


wbh at conducive

May 2, 2008, 4:43 PM

Post #2 of 5 (117 views)
Permalink
Re: X-Spam-Report for Clean messages [In reply to]

Gordon wrote:
> I have a need for more detail in my log. We are using exim as an
> inbound relay and pre-filtering before delivering to our mail system.
>
> The reject.log file is very useful but would be more helpful if it had
> CLEAN messages logged also. Now obviously CLEAN messages should not be
> part of reject.log so the best solution would be a log just for
> X-Spam-Report (SPAM and CLEAN)
>
> Searching for these terms is pretty rough since they are in so many conf
> files uploaded....
>
> --snip--
> warn add_header = X-Spam-Score: $spam_score ($spam_bar)\n\
> X-Spam-Report: $spam_report
> deny condition = ${if >{$spam_score_int}{90} {1}}
> message = Your message was RejecteD as Spam
> accept
> --snip--
>
> I know log size will increase, but I do not expect the logs to grow more
> than %33 considering yesterday we
> rejected 167,000 Unknown User
> rejected 133,000 spam
> and
> delivered 91,000 messages
>
>
>
>

Not a problem. You have a number of available tools, some seldom used:

1) log_selector =

lets you activate or supress various 'built-in' details - listed in docs.

2) log_message =

activates when an acl triggers

3) logwrite =

activates when an acl is traversed - triggered or not

4) adding a specifier as to *which* log, as in the use of :panic: below

logwrite = :panic:,VRL,$sender_host_address,$tod_epoch

Lets you put things into the log *you* choose, even if contrarian to
inbuilt log assignment.

N.B.

In a 'healthy' Exim environment, the 'panic' log is the least used,
hence not a bad place to do custom writes for cron'ed 'harvesting'.

Other options include writes to other-than the bespoke logfiles or to
SQL DB's, as in:

set acl_c19 = ${lookup pgsql{INSERT into ... <whatever>


SQL brings its own set of overhead & 'challenges', responsibility to
admin - but can be read-from or written-to from *anywhere* in Exim -
main section thru acl's thru router/transport sets.

HTH,

Bill


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


gb-mail at sbgnet

May 5, 2008, 7:36 AM

Post #3 of 5 (103 views)
Permalink
Re: X-Spam-Report for Clean messages [In reply to]

On 5/2/2008 7:43 PM, W B Hacker wrote:
> Gordon wrote:
>> I have a need for more detail in my log. We are using exim as an
>> inbound relay and pre-filtering before delivering to our mail system.
>>
>> The reject.log file is very useful but would be more helpful if it had
>> CLEAN messages logged also. Now obviously CLEAN messages should not be
>> part of reject.log so the best solution would be a log just for
>> X-Spam-Report (SPAM and CLEAN)
>>
>> Searching for these terms is pretty rough since they are in so many conf
>> files uploaded....
>>
>> --snip--
>> warn add_header = X-Spam-Score: $spam_score ($spam_bar)\n\
>> X-Spam-Report: $spam_report
>> deny condition = ${if >{$spam_score_int}{90} {1}}
>> message = Your message was RejecteD as Spam
>> accept
>> --snip--
>>
>> I know log size will increase, but I do not expect the logs to grow more
>> than %33 considering yesterday we
>> rejected 167,000 Unknown User
>> rejected 133,000 spam
>> and
>> delivered 91,000 messages
>>
>>
>>
>>
>
> Not a problem. You have a number of available tools, some seldom used:
>
> 1) log_selector =
>
> lets you activate or supress various 'built-in' details - listed in docs.
>
> 2) log_message =
>
> activates when an acl triggers
>
> 3) logwrite =
>
> activates when an acl is traversed - triggered or not
>
> 4) adding a specifier as to *which* log, as in the use of :panic: below
>
> logwrite = :panic:,VRL,$sender_host_address,$tod_epoch
>
> Lets you put things into the log *you* choose, even if contrarian to
> inbuilt log assignment.
>
> N.B.
>
> In a 'healthy' Exim environment, the 'panic' log is the least used,
> hence not a bad place to do custom writes for cron'ed 'harvesting'.
>
> Other options include writes to other-than the bespoke logfiles or to
> SQL DB's, as in:
>
> set acl_c19 = ${lookup pgsql{INSERT into ... <whatever>
>
>
> SQL brings its own set of overhead & 'challenges', responsibility to
> admin - but can be read-from or written-to from *anywhere* in Exim -
> main section thru acl's thru router/transport sets.
>
> HTH,
>
> Bill
>
>

Thank you!

4) adding a specifier as to *which* log, as in the use of :panic: below

logwrite = :panic:,VRL,$sender_host_address,$tod_epoch

Lets you put things into the log *you* choose, even if contrarian to
inbuilt log assignment.


Works with two caveats, and a question.

Question the VRL in the example, I can not find any detail so I assume
it is simple text...

1)
I got excited about choosing my own log and only succeed in logging to
panic.log When I choose from defined exim logs it works as expected.

...:saheaders: and :/var/log/exim/saheaders: both failed...

Success logging to reject.log, some messages may not in fact be rejects
but... If I put the messages in main.log my logwatch scripts will never
finish. As it stands they take up to 12 hours to run now.

2)
I have not weighed the impact of this yet, but I am leaning towards
leaving it as is...

logwrite =:reject:X-Spam-Score: $spam_score, X-Spam-Report:
$spam_report

is writing the entire entry on a single line. Useful for grep, not so
useful for reading. Example entry below

2008-05-05 10:22:51 1Jt1aa-000170-7V X-Spam-Score: 17.8, X-Spam-Report:
----\n Content preview: Our researchers have made something that will
keep you ahead\n of the game! Perform the most amazing change in your
body! http://likamen.com/\nfor the second time. After the controversial
winningleg kept him out of the\n main squad for nearly a
year.school's website, the lockdown was canceled.\n Our researchers
have made something that will keep you ahead of the game!\n Perform
the most amazing change in your body! [...] \n ----\n Content
analysis details: (17.8 points, 29.0 required)\n ----\n pts
rule name description\n ---- ----------------------
--------------------------------------------------\n 2.0
DATE_IN_PAST_96_XX Date: is 96 hours or more before Received: date\n
0.0 UNPARSEABLE_RELAY Informational: message has unparseable
relay lines\n 0.1 HTML_50_60 BODY: Message is 50% to 60%
HTML\n 0.0 HTML_MESSAGE BODY: HTML included in message\n
0.0 BAYES_50 BODY: Bayesian spam probability is 40 to
60%\n [score: 0.5000]\n 3.8 URIBL_AB_SURBL Contains
an URL listed in the AB SURBL blocklist\n [URIs: likamen.com]\n 4.1
URIBL_JP_SURBL Contains an URL listed in the JP SURBL
blocklist\n [URIs: likamen.com]\n 3.0 URIBL_OB_SURBL
Contains an URL listed in the OB SURBL blocklist\n [URIs:
likamen.com]\n 4.5 URIBL_SC_SURBL Contains an URL listed in
the SC SURBL blocklist\n [URIs: likamen.com]\n 0.3 MIME_






--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


gb-mail at sbgnet

May 5, 2008, 8:02 AM

Post #4 of 5 (103 views)
Permalink
Re: X-Spam-Report for Clean messages [In reply to]

> Not a problem. You have a number of available tools, some seldom used:
>
> 1) log_selector =
>
> lets you activate or supress various 'built-in' details - listed in docs.
>
> 2) log_message =
>
> activates when an acl triggers
>
> 3) logwrite =
>
> activates when an acl is traversed - triggered or not
>
> 4) adding a specifier as to *which* log, as in the use of :panic: below
>
> logwrite = :panic:,VRL,$sender_host_address,$tod_epoch
>
> Lets you put things into the log *you* choose, even if contrarian to
> inbuilt log assignment.
>
> N.B.
>
> In a 'healthy' Exim environment, the 'panic' log is the least used,
> hence not a bad place to do custom writes for cron'ed 'harvesting'.
>
> Other options include writes to other-than the bespoke logfiles or to
> SQL DB's, as in:
>
> set acl_c19 = ${lookup pgsql{INSERT into ... <whatever>
>
>
> SQL brings its own set of overhead & 'challenges', responsibility to
> admin - but can be read-from or written-to from *anywhere* in Exim -
> main section thru acl's thru router/transport sets.
>
> HTH,
>
> Bill
>
>




Sent with even less detail... the list bounced the reply, sorry for
duplicates...




Thank you!

4) adding a specifier as to *which* log, as in the use of :panic: below

logwrite = :panic:,VRL,$sender_host_address,$tod_epoch

Lets you put things into the log *you* choose, even if contrarian to
inbuilt log assignment.


Works with two caveats, and a question.

Question the VRL in the example, I can not find any detail so I assume
it is simple text...

1)
I got excited about choosing my own log and only succeed in logging to
panic.log When I choose from defined exim logs it works as expected.

...:saheaders: and :/var/log/exim/saheaders: both failed...

Success logging to reject.log, some messages may not in fact be rejects
but... If I put the messages in main.log my logwatch scripts will never
finish. As it stands they take up to 12 hours to run now.

2)
I have not weighed the impact of this yet, but I am leaning towards
leaving it as is...

logwrite =:reject:X-Spam-Score: $spam_score, X-Spam-Report:
$spam_report

is writing the entire entry on a single line. Useful for grep, not so
useful for reading.


--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


wbh at conducive

May 5, 2008, 9:01 AM

Post #5 of 5 (102 views)
Permalink
Re: X-Spam-Report for Clean messages [In reply to]

Gordon wrote:

*snip*
>
> Sent with even less detail... the list bounced the reply, sorry for
> duplicates...
>
no sweat..
>
>
>
> Thank you!
>
> 4) adding a specifier as to *which* log, as in the use of :panic: below
>
> logwrite = :panic:,VRL,$sender_host_address,$tod_epoch
>
> Lets you put things into the log *you* choose, even if contrarian to
> inbuilt log assignment.
>
>
> Works with two caveats, and a question.
>
> Question the VRL in the example, I can not find any detail so I assume
> it is simple text...

Yes - unique to one of our setups, where VRL = Viral, LBL = Local
BlackList, RBL = Remote BlackList... etc. Used for fast stats from the
SQL DB.

... but left in to demonstrate that you can easily intermix your choice
of text or codes.

>
> 1)
> I got excited about choosing my own log and only succeed in logging to
> panic.log When I choose from defined exim logs it works as expected.
>
> ...:saheaders: and :/var/log/exim/saheaders: both failed...
>
> Success logging to reject.log, some messages may not in fact be rejects
> but... If I put the messages in main.log my logwatch scripts will never
> finish. As it stands they take up to 12 hours to run now.
>

Last time I looked, only the 'standard' main, panic, and reject logs are
selectable with that option - and then only if you haven't handed off
the whole shebang to syslogd.

> 2)
> I have not weighed the impact of this yet, but I am leaning towards
> leaving it as is...
>
> logwrite =:reject:X-Spam-Score: $spam_score, X-Spam-Report:
> $spam_report
>

Better to write a more terse bespoke message for logging purposes than
to call X-headers. Easier to grep, takes less log space, separates what
Lusers see from what you need internally, 'can be' more CPU efficient.

FWIW - We have SA optioned to not even generate spam reports.

Who ever reads 'em?

> is writing the entire entry on a single line. Useful for grep, not so
> useful for reading.

Reading log entries on a 'single line' benefits greatly from 1440 or
1680 - wide LCD's and terminal sessions with small-but-clear fonts.


Bill

--
## List details at http://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

exim users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.