Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ClamAV: devel

Re: Clamav-90_2 Lockup with freebsd 6.2

 

 

ClamAV devel RSS feed   Index | Next | Previous | View Threaded


mb at imp

Mar 4, 2007, 1:16 PM

Post #1 of 6 (1782 views)
Permalink
Re: Clamav-90_2 Lockup with freebsd 6.2

Hi all,

After adding some debug stuff to clamd running with freebsd
libpthread.so I found that:

looking at the output value of 'ps -auxH | grep clamd | grep -v grep | wc -l'
is always staying at 6 threads, but the number of threadpool->thr_alive is
going higher and higher. So threadpool->thr_alive isn't decreased and
is completly incorrect.

In my tests the number of threads with libpthread.so is never going higher than
6 threads. That explains, why a higher load is going to delay all scan
operations more and more.

With libthr.so the value of threadpool->thr_alive is equal with the
output of the ps. It can go very high, but always goes back if no more
work is there to do.

After the maxthreads limit is reached, clamd becomes completly unresponsive
with libpthreads.so.

SIGKILL and SIGSTOP don't work because some (unexisting) worker threads block
on pthread_cond_timedwait(). Only kill -9 helps. Of course, the counter
threadpool->thr_alive is wrong and may cause this problem.

Maybe someone with more threads knowledge can help here. I'll now add some debug
stuff to see where threadpool->thr_alive is going to be increased and where it
is decreased. The strange thing is that this works fine with libthr.so.

Martin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


mb at imp

Mar 4, 2007, 3:06 PM

Post #2 of 6 (1640 views)
Permalink
Re: Clamav-90_2 Lockup with freebsd 6.2 [In reply to]

Hi,

> looking at the output value of 'ps -auxH | grep clamd | grep -v grep | wc -l'
> is always staying at 6 threads, but the number of threadpool->thr_alive is
> going higher and higher. So threadpool->thr_alive isn't decreased and
> is completly incorrect.

After setting IdleTimeout very low to 5 seconds, the threadpool->thr_alive gets
decreased correctly. There are still always 6 threads running, but further
threads get cleaned up correctly.

IMHO the whole clamd thread management is somehow broken.

--
Martin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


mb at imp

Mar 4, 2007, 4:02 PM

Post #3 of 6 (1636 views)
Permalink
Re: Clamav-90_2 Lockup with freebsd 6.2 [In reply to]

Hi,

> Make sure clamd is checking the return value of pthread_create()
> for errors. You can also set LIBPTHREAD_PROCESS_SCOPE=yes or
> LIBPTHREAD_SYSTEM_SCOPE=yes in your environment to force process
> scope or system scope threads (libpthread only) for clamd.

Yes, it does check the return value. And setting system or process scope
doesn't make any difference. As I said, increasing the thread count
works, decreading doesn't.

--
Martin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


mb at imp

Mar 12, 2007, 5:07 AM

Post #4 of 6 (1623 views)
Permalink
Re: Clamav-90_2 Lockup with freebsd 6.2 [In reply to]

Hi,

> [snip patch]
>
> Does this patch make the libmap.conf hack unneeded?
>

I frist thought so, but no, the more threads are running concurrently
the slower libpthreads behaves, as it gets more lock contention. Until
this is addressed somehow, use libthr.so

--
Martin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


mb at imp

Mar 16, 2007, 7:00 AM

Post #5 of 6 (1625 views)
Permalink
Re: Clamav-90_2 Lockup with freebsd 6.2 [In reply to]

Hi,

I just fixed those issues with the port.

Thanks for reporting !

Martin Blapp, <mb [at] imp> <mbr [at] FreeBSD>
------------------------------------------------------------------
ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: <finger -l mbr [at] freebsd>
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
------------------------------------------------------------------

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


mb at imp

Mar 18, 2007, 2:28 PM

Post #6 of 6 (1610 views)
Permalink
RE: Clamav-90_2 Lockup with freebsd 6.2 [In reply to]

Hi,

> I am not sure why. But my dual xeon with libthr on clamav-90.1 still gives
> very high cpu usage.

It's the same case here. What happens if you limit kern.smp.maxcpus
to 1 ? Does it still use the same amount cpu time ? What happens if
you link clamd against libc_r ?

--
Martin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

ClamAV devel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.