Mailing List Archive: ClamAV: devel

false positive rate for Phishing.Email

Index | Next | Previous | View Threaded

kgc at corp

Jun 14, 2007, 11:01 AM

Post #1 of 6 (1987 views)
Permalink

false positive rate for Phishing.Email

Our systems have been having a lot of trouble with long startup times so I
was eager to get everything over to 0.91rc1. The long startup problems are
fixed, thanks for that. However, I was a bit leery of the new
anti-phishing system so I setup a test on one of my mx servers to fork off
all mail flagged by it instead of outright rejecting them. Out of 912
messages that were caught by Phishing.Email a full 123 were human verified
false positives. An 87% accuracy rate is pretty awful and so it is clear
that this feature is not ready for production mail systems yet.

Nearly all of the 123 messages are legit or at least would appear to be
legit list traffic.

I could work towards anonymizing the 123 messages so I can provide them as
examples if that would be helpful.

Meanwhile, how can I turn off just Phishing.Email while leaving the other
signature passed phishing filters active? It wasn't clear how to do this
after reading the documentation.

Thanks!

--
Kelsey Cummings - kgc [at] corp sonic.net, inc.
System Architect 2260 Apollo Way
707.522.1000 Santa Rosa, CA 95407
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

kgc at corp

Jun 14, 2007, 10:41 AM

Post #2 of 6 (1863 views)
Permalink

false positive rate for Phishing.Email [In reply to]

edwintorok at gmail

Jun 14, 2007, 11:13 AM

Post #3 of 6 (1861 views)
Permalink

Re: false positive rate for Phishing.Email [In reply to]

On 6/14/07, Kelsey Cummings <kgc [at] corp> wrote:
> Out of 912
> messages that were caught by Phishing.Email a full 123 were human verified
> false positives.

Please post output of:
clamconf|grep Phish

If you have PhishingRestrictedScan = No, its obvious what the problem is.

> An 87% accuracy rate is pretty awful and so it is clear
> that this feature is not ready for production mail systems yet.
>
> Nearly all of the 123 messages are legit or at least would appear to be
> legit list traffic.

You can run with --debug, and look for Phishcheck: messages.

>
> I could work towards anonymizing the 123 messages so I can provide them as
> examples if that would be helpful.

That is quite a lot of work for 123 messages. Can you just anonymize a
few of them,
and attach them to a bugreport on our bugzilla.

>
> Meanwhile, how can I turn off just Phishing.Email while leaving the other
> signature passed phishing filters active?

PhishingScanURLs No

> It wasn't clear how to do this after reading the documentation.
man clamd.conf:
PhishingScanURLs BOOL
Scan URLs found in mails for phishing attempts.
Default: yes

P.S.: please don't post messages twice to the list.

Best regards,
Edwin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

kgc at corp

Jun 14, 2007, 12:02 PM

Post #4 of 6 (1855 views)
Permalink

Re: false positive rate for Phishing.Email [In reply to]

On Thu, Jun 14, 2007 at 09:13:04PM +0300, T?r?k Edvin wrote:
> On 6/14/07, Kelsey Cummings <kgc [at] corp> wrote:
> > Out of 912
> > messages that were caught by Phishing.Email a full 123 were human verified
> > false positives.
>
> Please post output of:
> clamconf|grep Phish

# clamconf | grep Phish
PhishingSignatures = yes
PhishingScanURLs = no
PhishingAlwaysBlockCloak = no
PhishingAlwaysBlockSSLMismatch = no
PhishingRestrictedScan = yes

PhishingScanURLs being = yes for the testing of course.

> > I could work towards anonymizing the 123 messages so I can provide them as
> > examples if that would be helpful.
>
> That is quite a lot of work for 123 messages. Can you just anonymize a
> few of them,
> and attach them to a bugreport on our bugzilla.

Will do.

> > Meanwhile, how can I turn off just Phishing.Email while leaving the other
> > signature passed phishing filters active?
>
> PhishingScanURLs No
>
> > It wasn't clear how to do this after reading the documentation.
> man clamd.conf:
> PhishingScanURLs BOOL
> Scan URLs found in mails for phishing attempts.
> Default: yes

I figured that out, it just wasn't obvious that this was the experimental
feature that was on by default for 0.91rc1. Sorry about the duplicate post
-- I thought the first got stuck due to an email address change on my end
and I updated my subsription information and reposted.

--
Kelsey Cummings - kgc [at] corp sonic.net, inc.
System Architect 2260 Apollo Way
707.522.1000 Santa Rosa, CA 95407
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

clamav-devel at subscriptions

Jun 14, 2007, 12:14 PM

Post #5 of 6 (1858 views)
Permalink

Re: false positive rate for Phishing.Email [In reply to]

On Thursday, June 14, 2007 at 11:13 AM, T�r�k Edvin wrote:
> On 6/14/07, Kelsey Cummings <kgc [at] corp> wrote:
> > Out of 912 messages that were caught by Phishing.Email a
> > full 123 were human verified false positives.
>
> Please post output of:
> clamconf|grep Phish

I reported this problem as bug 534 on 6/4/2007 and provided a sample case. For some reason, this bug and others I reported () haven't been acknowledged and all appear as locked in bugzilla.

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

jef at math

Jun 14, 2007, 12:31 PM

Post #6 of 6 (1872 views)
Permalink

Re: false positive rate for Phishing.Email [In reply to]

On Thu, 14 Jun 2007, [ISO-8859-1] T�r�k Edvin wrote:
> PhishingScanURLs No

I've been having a similar problem with a high number of false positives
with 0.91rc1

So far I've been manually examining them (we have a relatively low volume
site) and submitting them as false positives through clamav.net when it's
been reasonable. (It's a little bit of work, but worth it to help with
the testing of the feature.)

Using clamav from the command line, adding the --no-phishing-scan-urls
argument eliminates the problem.

Jeffrey Moskot
System Administrator
jef [at] math

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads