Mailing List Archive: Apache: Dev

Interesting server-status at w.a.o

Index | Next | Previous | View Threaded

rainer.jung at kippdata

Feb 24, 2012, 2:59 AM

Post #1 of 6 (211 views)
Permalink

Interesting server-status at w.a.o

Currently server-status at w.a.o shows 4 processes with no busy requests
but all slots in state "G" or "L" and many connections in "closing". It
seems the reason for those processes to not end are the closing
connections, but if it were correct lingering close, that would have
ended long ago. The situation persists since at least 10 minutes now, so
those recycling processes seem to hang.

In addition it is strange, that we see many "L" slots for those
gracefully shutting down processes. Looks like the state reset after "L"
doesn't happen when a process is scheduled for recycling.

Note that w.a.o is still at 2.3.15.

Don't know whether we can get some thread stacks for the hanging PIDs
69762, 62850, 62438 and 63707.

Regards,

Rainer

info at apachelounge

Feb 24, 2012, 3:12 AM

Post #2 of 6 (203 views)
Permalink

Re: **** Interesting server-status at w.a.o [In reply to]

Looks like the hanging "L"'s, which I reported way back, which was
investigated by Stefan

With 2.4.1, I have not seeing the "L"'s (yet).

-----Original Message-----
From: Rainer Jung
Sent: Friday, February 24, 2012 11:59 AM
To: dev [at] httpd
Cc: Apache Infrastructure
Subject: ******** Interesting server-status at w.a.o

Currently server-status at w.a.o shows 4 processes with no busy requests
but all slots in state "G" or "L" and many connections in "closing". It
seems the reason for those processes to not end are the closing
connections, but if it were correct lingering close, that would have
ended long ago. The situation persists since at least 10 minutes now, so
those recycling processes seem to hang.

In addition it is strange, that we see many "L" slots for those
gracefully shutting down processes. Looks like the state reset after "L"
doesn't happen when a process is scheduled for recycling.

Note that w.a.o is still at 2.3.15.

Don't know whether we can get some thread stacks for the hanging PIDs
69762, 62850, 62438 and 63707.

Regards,

Rainer

rainer.jung at kippdata

Feb 24, 2012, 3:50 AM

Post #3 of 6 (202 views)
Permalink

Re: **** Interesting server-status at w.a.o [In reply to]

On 24.02.2012 12:12, Steffen wrote:
> Looks like the hanging "L"'s, which I reported way back, which was
> investigated by Stefan

Could be but not necessarily. It depends on your MaxConnectionsPerChild
setting. The situation at w.a.o is due to processes getting recycled. I
sporadically watched the server-status there and the "L" situation did
not show up for a long time. I expect the adminds did restart Apache
gracefully.

If your "L"s showed up during normal operation without restart and with
MaxConnectionsPerChild 0, then it could well be totaly unrelated. If
they suddenly show up when your child is being recycled, then it could
be the same reason.

> With 2.4.1, I have not seeing the "L"'s (yet).
>

That's good to know. Thanks!

The more important problem at w.a.o IMHO is that those gracefully
shutting down processes either do not terminate, or their scoreboard
entries are not cleared.

But maybe that's also fixed with Stefan's "L" improvements. Didn't have
time to check it. I hope infra will update to 2.4.1.

Regards,

Rainer

sf at sfritsch

Feb 24, 2012, 1:31 PM

Post #4 of 6 (193 views)
Permalink

Re: Interesting server-status at w.a.o [In reply to]

On Friday 24 February 2012, Rainer Jung wrote:
> > Looks like the hanging "L"'s, which I reported way back, which
> > was investigated by Stefan
>

>
> But maybe that's also fixed with Stefan's "L" improvements. Didn't
> have time to check it. I hope infra will update to 2.4.1.

I can't remember having made any change directly relevant to the
hanging "L" states. I only fixed various other logging issues found by
Steffen. But the changes from w.a.o's version to 2.4.1 are so large
that it makes sense to update w.a.o first and investigate the problems
afterwards.

sctemme at apache

Feb 26, 2012, 8:46 AM

Post #5 of 6 (188 views)
Permalink

Re: **** Interesting server-status at w.a.o [In reply to]

On Feb 24, 2012, at 3:50 AM, Rainer Jung wrote:

> On 24.02.2012 12:12, Steffen wrote:
>> Looks like the hanging "L"'s, which I reported way back, which was
>> investigated by Stefan
>
> Could be but not necessarily. It depends on your MaxConnectionsPerChild setting. The situation at w.a.o is due to processes getting recycled. I sporadically watched the server-status there and the "L" situation did not show up for a long time. I expect the adminds did restart Apache gracefully.

If I recall correctly, a.o httpds get a Graceful every 24h to turn over the logs. So you should never see a really long-lived scoreboard on any of those instances.

S.

--
sctemme [at] apache http://www.temme.net/sander/
PGP FP: FC5A 6FC6 2E25 2DFD 8007 EE23 9BB8 63B0 F51B B88A

View my availability: http://tungle.me/sctemme

rainer.jung at kippdata

Feb 26, 2012, 10:11 AM

Post #6 of 6 (190 views)
Permalink

Re: **** Interesting server-status at w.a.o [In reply to]

On 26.02.2012 17:46, Sander Temme wrote:
>
> On Feb 24, 2012, at 3:50 AM, Rainer Jung wrote:
>
>> On 24.02.2012 12:12, Steffen wrote:
>>> Looks like the hanging "L"'s, which I reported way back, which was
>>> investigated by Stefan
>>
>> Could be but not necessarily. It depends on your MaxConnectionsPerChild setting. The situation at w.a.o is due to processes getting recycled. I sporadically watched the server-status there and the "L" situation did not show up for a long time. I expect the adminds did restart Apache gracefully.
>
> If I recall correctly, a.o httpds get a Graceful every 24h to turn over the logs. So you should never see a really long-lived scoreboard on any of those instances.

Yes, Daniel Shahaf also wrote on the infra list, that they have a daily
restart. But the problem seems to happen often enough, e.g. today you
can see the same symptoms. I even saw for a few seconds a server-status
that had all processes with a "no" in the "accepting" column (although I
could still get the result of the server-status itself.

Nevertheless still it is still version 2.3.15 I hope we will see
something better after infra finds time to update.

Regards,

Rainer

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads