edwintorok at gmail
Jul 16, 2008, 8:53 AM
Post #2 of 2
(123 views)
Permalink
|
|
Re: Patch to improve loading speed of compressed database
[In reply to]
|
|
On 2008-07-15 11:30, Damien Deville wrote:
> Dear Clamav-Devel,
>
> Here is a set of patches of libclamav and clamd for clamav-0.93.3 that
> reduces the loading time of compressed antiviral database in cvd files.
> They can be found at the following url: http://pbmtp.free.fr/clamav/
>
> We noticed by instrumenting the loading of database that the
> uncompressing of compressed cvd files is performed line by line using
> gzgets function and that this function is the hotspot of the loading time.
>
> The proposed patch optimizes the loading of compressed database (in cvd)
> by uncompressing big blocks of compressed data using gzread, store them
> into a cache and read lines of uncompressed data from the cache.
> The patch add two new functions to initialize a cache of given size and
> one to free the allocated memory for the cache.
> cli_dbgets use a new function called cli_dbgets_cache when a cache
> pointer is found.
>
> To use the cache simply call cl_dbcache_init(size_in_kb) before calling
> cl_load() and after load release the allocated memory using
> cl_dbcache_cleanup(). Such a patch is provided for clamd.
>
> Here are some benchmark results concerning the loading time on a VIA C3
> Samuel 2 at 400.91-MHz
>
> Loading of compressed cvd (no cache): 4:13.54
> Loading of compressed cvd (cache of 16KB): 1:11.75
> Loading of uncompressed cvd: 0:53.68
>
Excellent!
> You can see that loading time is closed to be divided by a factor of 4.
>
> We would be happy if some of you could have a look at the patchs and
> test them and report any issues or remarks.
>
Please open an enhancement bugreport on bugs.clamav.net, so that we can
assign a target milestone for this.
Best regards,
--Edwin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
|
