Mailing List Archive: ClamAV: users

ScanStream: accept timeout, unkillable

Index | Next | Previous | View Threaded

spork at bway

Feb 5, 2007, 4:17 PM

Post #1 of 3 (1484 views)
Permalink

ScanStream: accept timeout, unkillable

Hi all,

I've been having this problem for quite some time and thought it might go away
after upgrading all our spamd/clamd boxes to FreeBSD 6.2 from 4.11.

It hasn't though... We use a maildrop recipe that uses the clamd-stream-client
to send messages over to a cluster of spam/virus filtering boxes and we find
that clamd sometimes hangs and stops taking any new connections.

The logs look like this:

Feb 5 13:25:46 spamd2 clamd[36224]: ScanStream: accept timeout.
Feb 5 13:26:45 spamd2 clamd[36224]: ScanStream: accept timeout.
Feb 5 13:28:48 spamd2 last message repeated 31 times
Feb 5 13:29:46 spamd2 last message repeated 28 times
Feb 5 13:30:10 spamd2 clamd[36224]: SelfCheck: Database status OK.
Feb 5 13:30:45 spamd2 clamd[36224]: ScanStream: accept timeout.
Feb 5 13:31:16 spamd2 last message repeated 17 times
Feb 5 13:33:20 spamd2 last message repeated 31 times
Feb 5 13:43:30 spamd2 last message repeated 151 times
Feb 5 13:53:20 spamd2 last message repeated 149 times

The only thing that will stop clamd is a "kill -9".

Clamav is 0.88.7 from FreeBSD ports collection.
OS is FreeBSD 6.2-RELEASE.
This happens at least once a day.

Any other info I can provide?

I'm not too handy with gdb, but maybe this means something to someone:

[root [at] spamd /home/spork]# gdb /usr/local/sbin/clamd 36224
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols
found)...
Attaching to program: /usr/local/sbin/clamd, process 36224
Reading symbols from /usr/local/lib/libclamav.so.1...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/libclamav.so.1
Reading symbols from /lib/libz.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.3
Reading symbols from /usr/lib/libbz2.so.2...(no debugging symbols
found)...done.Loaded symbols for /usr/lib/libbz2.so.2
Reading symbols from /usr/local/lib/libgmp.so.7...(no debugging symbols
found)...done.
Loaded symbols for /usr/local/lib/libgmp.so.7
Reading symbols from /lib/libpthread.so.2...(no debugging symbols
found)...done.
warning: Unable to get location for thread creation breakpoint: generic error
[New Thread 0x805fe00 (runnable)]
[New Thread 0x9893600 (runnable)]
[New Thread 0x98b1400 (runnable)]
[New Thread 0x962d800 (runnable)]
[New Thread 0x95fea00 (runnable)]
[New Thread 0x95a6e00 (runnable)]
[New Thread 0x962d600 (runnable)]
[New Thread 0x8e61e00 (runnable)]
[New Thread 0x962d400 (runnable)]
[New Thread 0x98b1000 (runnable)]
[New Thread 0x98b1800 (runnable)]
[New Thread 0x98b1600 (runnable)]
[New Thread 0x9893e00 (runnable)]
[New Thread 0x98b1a00 (runnable)]
[New Thread 0x9893000 (runnable)]
[New Thread 0x962dc00 (runnable)]
[New Thread 0x9893c00 (runnable)]
[New Thread 0x989c200 (runnable)]
[New Thread 0x962da00 (runnable)]
[New Thread 0x98ef200 (runnable)]
[New Thread 0x95a6c00 (runnable)]
[New Thread 0x98ef000 (runnable)]
[New Thread 0x9893800 (runnable)]
[New Thread 0x9893a00 (runnable)]
[New Thread 0x9893400 (runnable)]
[New Thread 0x95fe800 (runnable)]
[New Thread 0x989c000 (runnable)]
[New Thread 0x9893200 (runnable)]
[New Thread 0x98b1e00 (runnable)]
[New Thread 0x95fec00 (runnable)]
[New Thread 0x87c2600 (LWP 100065)]
[New Thread 0x805f000 (runnable)]
[New LWP 100199]
Loaded symbols for /lib/libpthread.so.2
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols
found)...done.Loaded symbols for /libexec/ld-elf.so.1
[Switching to LWP 100199]
0x28148c6b in pthread_testcancel () from /lib/libpthread.so.2
(gdb) where
#0 0x28148c6b in pthread_testcancel () from /lib/libpthread.so.2
#1 0x28140efa in pthread_mutexattr_init () from /lib/libpthread.so.2
#2 0x08057400 in ?? ()

Thanks,

Charles
___
Charles Sprickman
NetEng/SysAdmin
Bway.net - New York's Best Internet - www.bway.net
spork [at] bway - 212.655.9344

_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html

cgreen at sentex

Feb 6, 2007, 7:05 AM

Post #2 of 3 (1359 views)
Permalink

Re: ScanStream: accept timeout, unkillable [In reply to]

Charles Sprickman wrote:
> Hi all,
>
> I've been having this problem for quite some time and thought it might
> go away after upgrading all our spamd/clamd boxes to FreeBSD 6.2 from
> 4.11.
>
> It hasn't though... We use a maildrop recipe that uses the
> clamd-stream-client to send messages over to a cluster of spam/virus
> filtering boxes and we find that clamd sometimes hangs and stops
> taking any new connections.

We've been running 6.x our scanner boxes for a while now, but it's only
been with the more recent security/clamav-devel port installs that we
noticed a problem much like this. Most connections to the daemon (made
through clamav-milter in our case) timed out, and the only way to bring
down the daemon was with a kill -9.

For us, the 20061029 devel snapshot was fine, but the current one
(20061217) has problems.

We never noticed a problem on 4.x, but it's been rather a while since we
ran clamd on that.

From the gdb trace you provided, it looks like clam is having issues
with the new libpthread threading library. This is what we figured our
boxes were having trouble with as well.

We've had success changing back to the older libthr library. Try
dropping this into /etc/libmap.conf:

------------

[clamd]
libc_r.so.5 libthr.so.2
libc_r.so.6 libthr.so.2
libthr.so.2 libthr.so.2
libpthread.so.1 libthr.so.2
libpthread.so.2 libthr.so.2

------------

If it works, great. If not, there's nothing more I can suggest. I
don't know why this works for us -- it was mostly a shot in the dark
since the issue appeared to be threading related and we knew FreeBSD 6
has seen a lot of work done on the new threading library and there may
still have been bugs to work out or something. I know Perl, not C, so
figuring out if the bug is in clamd or in libpthread will take someone
other than me.

I sort of meant to open up a bug about this; if switching to libthr
works for you, I guess I should.

Cheers,

Craig.
------
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html

spork at bway

Feb 12, 2007, 1:51 PM

Post #3 of 3 (1342 views)
Permalink

Re: ScanStream: accept timeout, unkillable [In reply to]

On Tue, 6 Feb 2007, Craig Green wrote:

> Charles Sprickman wrote:
>> Hi all,
>>
>> I've been having this problem for quite some time and thought it might go
>> away after upgrading all our spamd/clamd boxes to FreeBSD 6.2 from 4.11.
>>
>> It hasn't though... We use a maildrop recipe that uses the
>> clamd-stream-client to send messages over to a cluster of spam/virus
>> filtering boxes and we find that clamd sometimes hangs and stops taking any
>> new connections.
>
> We've been running 6.x our scanner boxes for a while now, but it's only been
> with the more recent security/clamav-devel port installs that we noticed a
> problem much like this. Most connections to the daemon (made through
> clamav-milter in our case) timed out, and the only way to bring down the
> daemon was with a kill -9.
>
> For us, the 20061029 devel snapshot was fine, but the current one (20061217)
> has problems.
>
> We never noticed a problem on 4.x, but it's been rather a while since we ran
> clamd on that.
>
> From the gdb trace you provided, it looks like clam is having issues with
> the new libpthread threading library. This is what we figured our boxes were
> having trouble with as well.
>
> We've had success changing back to the older libthr library. Try dropping
> this into /etc/libmap.conf:

That does it!

Is this something to work with the clamav people on or the FreeBSD folks?

Thanks,

Charles

> ------------
>
> [clamd]
> libc_r.so.5 libthr.so.2
> libc_r.so.6 libthr.so.2
> libthr.so.2 libthr.so.2
> libpthread.so.1 libthr.so.2
> libpthread.so.2 libthr.so.2
>
> ------------
>
> If it works, great. If not, there's nothing more I can suggest. I don't
> know why this works for us -- it was mostly a shot in the dark since the
> issue appeared to be threading related and we knew FreeBSD 6 has seen a lot
> of work done on the new threading library and there may still have been bugs
> to work out or something. I know Perl, not C, so figuring out if the bug is
> in clamd or in libpthread will take someone other than me.
>
> I sort of meant to open up a bug about this; if switching to libthr works for
> you, I guess I should.
>
>
> Cheers,
>
> Craig.
> ------
> _______________________________________________
> Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
> http://lurker.clamav.net/list/clamav-users.html
>
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com