stuart at morungos
Oct 8, 2010, 11:35 AM
Post #2 of 2
As I understand it, MaxRequestPerChild and ThreadsPerChild are both
Re: Re: still struggling with fastcgi (Stuart Watt)
[In reply to]
Apache configurations which handle the Apache processes, and have no
(direct) effect in FastCGI. They may well have an indirect effect, as
they can cause requests to be queued in Apache before they are forwarded
to Perl processes. FastCGI will have a second set of process management
controls, probably somewhere else on configuration. What Apache FastCGI
system are you using? I'll see if I can run our system under Windows
Apache (it's about time I tried it) and test that out.
On 10/8/2010 12:56 PM, Joseph He wrote:
> Thank you, Stuart.
> Process Explorer really helps, at least I can see which script is
> alive, even though I still cannot figure out it is process or thread.
> Here is what I see on Process Explorer:
> 1. There is httpd.exe. Its command line is C:\Apache2.2\bin\httpd.exe
> -k runservice
> 2. Under httpd.exe, there is another httpd.exe, its command line is
> C:\Apache2.2\bin\httpd.exe -d C:/Apache2.2 -f
> C:\Apache2.2\conf\httpd.conf -d C:\Apache2.2\.
> 3. Under httpd.exe mentioned in 2, there are multiple perl.exe, their
> command lines are typically C:\Perl\bin\Perl.exe
> C:\...\fcgi-bin\index.fcgi, etc.
> ## All these perl.exe seems to be process instead of thread to me
> 'cause its memory usage is typically from 15M - 23M, which is a bit
> higher to me if it is a thread.
> I believe this is resource issue, whenever my cpu usage reaches to
> 100%, problem happens. So I studied Apache configure directives and
> find that MaxRequestPerChild and ThreadsPerChild are supported by
> mdm_winnt, no other configuration directives are useful in windows.
> Below is my testing and observation:
> MaxRequestsPerChild 6
> ThreadsPerChild 5
> Now after clicking and clicking, on Process Explorer, I can see Apache
> launches another httpd.exe under httpd.exe mentioned in 1, then the
> 'older' httpd.exe and all the perl.exe under its umbrella are killed.
> The server just repeated this for three days without problem. But,
> sorry, please bear with me that there is still but here :)
> With these two directives, perl.exe processes cannot just grow more
> and more now, but I don't know how Apache counts this
> MaxRequestPerChild, some pages I can refresh 30 times before new
> httpd.exe is launched, some others might be 15, might be 8 or 9. Some
> times I can see a new server is launched with 5 perl.exe, sometimes it
> might happen with more that 20 perl.exe processes running. This really
> puzzled me, because it is hard to optimize the server for production,
> especially our clients might use different kinds of server/hardware.
> Who can explain what exactly MaxRequestPerChild and ThreadsPerChild
> are in Windows environment? Apache document does not help me much here.
> Really appreciate all your helps!
> Today's Topics:
> 1. Re: still struggling with fastcgi (Stuart Watt)
> Message: 1
> Date: Sat, 2 Oct 2010 13:24:23 -0400
> From: Stuart Watt <stuart [at] morungos <mailto:stuart [at] morungos>>
> Subject: Re: [Catalyst-dev] still struggling with fastcgi
> To: Development of the elegant MVC web framework
> <catalyst-dev [at] lists
> <mailto:catalyst-dev [at] lists>>
> Message-ID: <33E64D3E-4005-479A-A800-5619A90ADE59 [at] morungos
> <mailto:33E64D3E-4005-479A-A800-5619A90ADE59 [at] morungos>>
> Content-Type: text/plain; charset="us-ascii"
> If you see multiple processes in the Windows task manager, these
> will be distinct Perl processes rather than threads. I haven't
> used Apache FastCGI, but under IIS, FastCGI is a single Perl
> thread per worker process, and this is likely to be the case. If
> you use the SysInternals process explorer (I'd strongly recommend
> using these tools for performance debugging in Windows) this will
> allow you to see how many threads each process is using inside. So
> it sounds like Apache may be creating Windows processes on you,
> but I could be wrong.
> I haven't seen this kind of an error with IIS/FastCGI, but we tend
> not to use that many worker processes. I've been using 20 worker
> processes at most.
> There are connection limits in MSSQL which could be a factor. That
> would be something I'd check.
> The 109 implies you are using named pipes to communicate between
> Perl and MSSQL. I tend to use TCP/IP. These are settings in the
> MSSQL Configuration Manager tool, and it might be worth looking to
> see whether changing the protocols might result in a different
> error. I'm not a low-level Windows expert, and there are many
> other resource limits you can hit in Windows apart from memory
> issues. Again, the SysInternals tools might help.
> Not having used Apache FastCGI that's about all I can say for now,
> but at the Perl/MSSQL boundary (which we hit even using
> IIS/FastCGI) these are issues I'd suggest looking into.
> All the best
> Stuart Watt
> stuart [at] morungos <mailto:stuart [at] morungos>
> On 2010-10-01, at 3:59 PM, Joseph He wrote:
> > It seems to be a resource issue, so it is a matter of how to
> configure Apache/Windows to better handle/allocate resources.
> fastCGI helps to keep scripts stay in cache instead of killing and
> loading it every time.
> > My understanding is on Windows, there is only one process with
> many threads, so my assumption is if I set limit on the threads,
> then there should not be a lot of spare threads staying in the
> cache, thus, the system will not be running to resource limit.
> > I add these to conf file.
> > ThreadLimit 5
> > ThreadsPerChild 5
> > Unfortunately, on windows process monitor, I still see the
> number of Perl processes just goes up and up with my clicking and
> then give me error when it reaches some level. This is not a
> striking test yet, just clicking 40 different pages by myself can
> exhaust all the resources ....
> > On 29 September 2010 22:15, Joseph He <joseph.he.2008 [at] gmail
> <mailto:joseph.he.2008 [at] gmail>> wrote:
> > > Thank you, T0m. Actually it is 5.8.7.
> > >
> > > I have done more testing while having windows Tast Manager
> open. I find o=
> > ut
> > > that these issues only happen when I already have about 35-38
> Perl proces=
> > ses
> > > running. Mainly two types of errors:
> > >
> > > 1. [Wed Sep 29 15:06:11 2010] [error] [client 127.0.0.1] (OS
> 109)The pipe
> > > has been ended. : FastCGI: comm with server "C:.....fcgi"
> > > GetOverlappedResult() failed, referer ......
> > >
> > > ## I searched and know that "(OS 109)The pipe has been ended"
> is SQL serv=
> > er
> > > error message.
> > >
> > > 2. [Wed Sep 29 15:10:52 2010] [error] [client 127.0.0.1]
> FastCGI: server
> > > "C:......fcgi" stderr: install_driver(ODBC) failed: Can't load
> > > 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module
> DBD::ODBC: load_file=
> > :A
> > > dynamic link library (DLL) initialization routine failed at
> > > C:/Perl/lib/DynaLoader.pm line 230., referer: ....
> > > #
> > Sounds like an out of memory condition triggering a problem.
> What does the
> > Windows monitor show with respect to memory usage?
> > I have seen random errors from FastCGI on Linux, too, which
> aren't all that
> > easy to track down.
> > How about trying it through an Apache server on Windows and see
> if that
> > works or gives more information.
> > Or running your app on Linux if that's an option.
> > Regards, Peter
> > http://perl.dragonstaff.co.uk
> Catalyst-dev mailing list
> Catalyst-dev [at] lists