Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ClamAV: devel

Bloom Hash AV Matcher

 

 

ClamAV devel RSS feed   Index | Next | Previous | View Threaded


sherpya at netfarm

May 29, 2007, 10:02 PM

Post #1 of 3 (1621 views)
Permalink
Bloom Hash AV Matcher

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Some time ago two guys wrote a patch for clamav
to use a different filter that works with the
collaboration of bm matcher,
in brief bloom av gives no false negative
but may have false positive, the file then is passed to
the bm matcher.

The attached patch is rather old, with new changes to the
engine I don't known if it still works,
and how it's easy to adapt.
it also needs to be tweaked to support scan with offset
(right now I've made as a false positive so the scan is passed to bm)

bloom av is faster than bm, the overall scan speed
is improved since the hypothesis is that
non virus files are a lot more than virus files.

I've attached also a profiled scan
look the detail:

[bm + ac]
54.63 166.07 166.07 8012 0.02 0.02 cli_bm_scanbuff
22.41 234.20 68.13 139428866 0.00 0.00 cli_findpos
15.26 280.59 46.39 8012 0.01 0.01 cli_ac_scanbuff

[(bloom | bm) + ac]
27.85 67.22 67.22 139428866 0.00 0.00 cli_findpos
27.31 133.15 65.93 245 0.27 0.27 cli_bm_scanbuff
19.52 180.26 47.11 8012 0.01 0.01 cli_ac_scanbuff

and
2.20 217.76 5.31 8012 0.00 0.00 cli_bloom_filter_scanbuff

so we gain 8012 - 245 bm scans, replaced by 8012 bloom scans that are faster

the logic of the overall scan is:
bloom first, if positive bm,
then ac as normal flow

the patch has not yet included gpl header, but the guy gave me the permission
to distribute it as GPL

Hope this helps

- --
Gianluigi Tiesi <sherpya [at] netfarm>
EDP Project Leader
Netfarm S.r.l. - http://www.netfarm.it/
Free Software: http://oss.netfarm.it/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGXQVr3UE5cRfnO04RAvs6AJ0bp5AofqW5c/ssdW9BdVCd4rwaVQCcCQP9
WrTvFvzBrCKjr3ELiamVvgI=
=cN7P
-----END PGP SIGNATURE-----
Attachments: profiler.txt (4.68 KB)


gim at skrzynka

May 30, 2007, 11:52 PM

Post #2 of 3 (1544 views)
Permalink
Re: Bloom Hash AV Matcher [In reply to]

Gianluigi Tiesi in message '[Clamav-devel] Bloom Hash AV Matcher' wrote:
>
> the logic of the overall scan is:
> bloom first, if positive bm,
> then ac as normal flow
>
> the patch has not yet included gpl header, but the guy gave me the permission
>

so, where's the patch?

how much additional memory does it take?

cheers,
--
main(int a[puts("Michal 'GiM' Spadlinski")]){}
Attachments: signature.asc (0.18 KB)


sherpya at netfarm

May 31, 2007, 1:01 AM

Post #3 of 3 (1550 views)
Permalink
Re: Bloom Hash AV Matcher [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

GiM wrote:
> Gianluigi Tiesi in message '[Clamav-devel] Bloom Hash AV Matcher' wrote:
>> the logic of the overall scan is:
>> bloom first, if positive bm,
>> then ac as normal flow
>>
>> the patch has not yet included gpl header, but the guy gave me the permission
>>
>
> so, where's the patch?
>
> how much additional memory does it take?
>
sorry this list seems to remove non txt attachments

I don't known about memory requirements, I don't think it will
affect too much

I've attached the patch

- --
Gianluigi Tiesi <sherpya [at] netfarm>
EDP Project Leader
Netfarm S.r.l. - http://www.netfarm.it/
Free Software: http://oss.netfarm.it/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGXoDM3UE5cRfnO04RAr3hAKCjfk5eD+xK4XbUG7vIHlk5RGfc8ACeODzM
5fwerVRNXQMgz8GkXuXdgl4=
=CiHB
-----END PGP SIGNATURE-----
Attachments: bloom-clamav.diff (15.8 KB)

ClamAV devel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.