Mailing List Archive: Linux: Kernel

2.6.27 inotify causes unkillable processes with 100% CPU usage

Index | Next | Previous | View Threaded

tss at iki

Dec 15, 2008, 3:31 AM

Post #1 of 3 (2227 views)
Permalink

2.6.27 inotify causes unkillable processes with 100% CPU usage

There appears to be a bug in 2.6.27 and inotify where it causes
processes in running state to eat 100% CPU and they can't be killed
with kill -9, requiring a reboot to get rid of them.

3 people (Cc'd) so far have reported this to me, all of them last
week. The problem shows up when running my Dovecot IMAP server. Its
imap processes start hanging, apparently it can happen after running
for only a couple of minutes:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6717 arno 20 0 2964 1608 1192 R 100 0.2 1158:05 imap

Only the imap processes hang (Dovecot has a few others). The only
thing special about it is that it uses inotify. After disabling
Dovecot's inotify support at least one of the people (Arno) reported
that he can't reprocuce the bug anymore. He also couldn't reproduce
the bug with 2.6.26 kernel.

I don't think Dovecot does anything special with its inotify code. It
just listens on two directories. You can find the code here: http://hg.dovecot.org/dovecot-1.1/file/f9a000ee400d/src/lib/ioloop-notify-inotify.c

Kernels causing the hang:

2.6.27-9.slh.1-sidux-686
Arch Linux, kernel 2.6.27.8
Gentoo with 2.6.27-gentoo-r5, amd64, 2 dual core opterons

Not causing the hang:

2.6.26-6.slh.1-sidux-686

Attachments:

PGP.sig (0.19 KB)

viro at ZenIV

Dec 15, 2008, 4:05 AM

Post #2 of 3 (2190 views)
Permalink

Re: 2.6.27 inotify causes unkillable processes with 100% CPU usage [In reply to]

On Mon, Dec 15, 2008 at 01:31:29PM +0200, Timo Sirainen wrote:
> There appears to be a bug in 2.6.27 and inotify where it causes processes
> in running state to eat 100% CPU and they can't be killed with kill -9,
> requiring a reboot to get rid of them.

> Kernels causing the hang:
>
> 2.6.27-9.slh.1-sidux-686
> Arch Linux, kernel 2.6.27.8
> Gentoo with 2.6.27-gentoo-r5, amd64, 2 dual core opterons
>
> Not causing the hang:
>
> 2.6.26-6.slh.1-sidux-686

Could you try to bisect that? Particulary interesting commits:

8f7b0ba1c853919b85b54774775f567f30006107
6ff2d39b91aec3dcae951afa982059e3dd9b49dc
711a49a07f84f914aac26a52143f6e7526571143
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

holger at wizards

Dec 15, 2008, 5:52 AM

Post #3 of 3 (2167 views)
Permalink

Re: 2.6.27 inotify causes unkillable processes with 100% CPU usage [In reply to]

On Mon, 15 Dec 2008 12:05:12 +0000, Al Viro wrote:

> On Mon, Dec 15, 2008 at 01:31:29PM +0200, Timo Sirainen wrote:
>> There appears to be a bug in 2.6.27 and inotify where it causes
>> processes in running state to eat 100% CPU and they can't be killed with
>> kill -9, requiring a reboot to get rid of them.
>
>> Kernels causing the hang:
>>
>> 2.6.27-9.slh.1-sidux-686
>> Arch Linux, kernel 2.6.27.8
>> Gentoo with 2.6.27-gentoo-r5, amd64, 2 dual core opterons
>>
>> Not causing the hang:
>>
>> 2.6.26-6.slh.1-sidux-686
>
> Could you try to bisect that? Particulary interesting commits:

Please see this posting, which also contains a pointer to the
necessary patch:
http://lkml.indiana.edu/hypermail/linux/kernel/0812.1/02305.html

According to GregKH it will be in -stable.10, but in the meantime you can
also apply the patch from http://tinyurl.com/5gpqkj on top stable .8
(which introduced the regression) or .9.

At this point it is not clear whether the changes to inotify caused this
or just fell victim to the parallel regression in idr.

-h

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads