njh at bandsman
Mar 10, 2007, 11:46 PM
Post #2 of 10
(1988 views)
Permalink
|
|
Re: [PATCH] for FreeBSD and libpthread problems, and high CPU usage
[In reply to]
|
|
Martin Blapp wrote:
>
> Hi,
>
> It looks like the codepath in do_multipart() is not threadsafe, as it
> deadlocks threads entering it at the same time. This needs more
> investigation
> but for now this quick fix (the third one) will be enough. Since I use this
> patch, CPU utilisation has go down and even a flood of scans recovered
> fast.
> In opposite, a maginal load of 3-4 mails per second without this patch
> leads
> to starvation with libpthread.so very soon.
>
> --
> Martin
>
> --- libclamav/mbox.c Tue Feb 13 14:06:57 2007
> +++ libclamav/mbox.c Sun Mar 11 08:57:56 2007
> @@ -413,6 +413,7 @@
>
> #ifdef CL_THREAD_SAFE
> static pthread_mutex_t tables_mutex = PTHREAD_MUTEX_INITIALIZER;
> +static pthread_mutex_t multipart_mutex = PTHREAD_MUTEX_INITIALIZER;
> #endif
>
> #ifndef O_BINARY
> @@ -2506,10 +2512,16 @@
> case APPLEDOUBLE:
> case KNOWBOT:
> case -1:
> +#ifdef CL_THREAD_SAFE
> + pthread_mutex_lock(&multipart_mutex);
> +#endif
> mainMessage = do_multipart(mainMessage,
> messages, multiparts,
> &rc, mctx, messageIn,
> &aText, recursion_level);
> +#ifdef CL_THREAD_SAFE
> + pthread_mutex_unlock(&multipart_mutex);
> +#endif
> --multiparts;
> if(rc == VIRUS)
> infected = TRUE;
> @@ -2686,9 +2701,15 @@
>
> cli_dbgmsg("Mixed message with %d parts\n", multiparts);
> for(i = 0; i < multiparts; i++) {
> +#ifdef CL_THREAD_SAFE
> + pthread_mutex_lock(&multipart_mutex);
> +#endif
> mainMessage = do_multipart(mainMessage,
> messages, i, &rc, mctx,
> messageIn, &aText, recursion_level + 1);
> +#ifdef CL_THREAD_SAFE
> + pthread_mutex_unlock(&multipart_mutex);
> +#endif
> if(rc == VIRUS) {
> infected = TRUE;
> break;
do_multipart() does not spawn multiple threads, or access global variables,
so there is nothing that it needs to be thread-safe for.
Perhaps it is something else that is not thread safe, that do_mulipart() is calling?
Naturally CPU usage will go down if you change an application from multiple to single threading,
that does not necessarily indicate a problem, quite the opposite, it could indicate
that all is working as it should.
Please demonstrate evidence, e.g. files which cause a deadlock when scanned. Please
also test against SVN, patches are only accepted against that version.
> Martin Blapp, <mb [at] imp> <mbr [at] FreeBSD>
-Nigel
--
Nigel Horne. Arranger, Adjudicator, Band Trainer, Composer, Tutor, Typesetter.
NJH Music, Barnsley, UK. ICQ#20252325
njh [at] bandsman http://www.bandsman.co.uk
|
signature.asc
