Mailing List Archive: ModPerl: Dev

dual server setup with 2.0?

Index | Next | Previous | View Threaded

stas at stason

Apr 23, 2001, 7:26 PM

Post #1 of 19 (2682 views)
Permalink

dual server setup with 2.0?

Hey folks,

I thought about the mod_perl 2.0 performance setups. So, since now one
process can have pools with interpreter and without it, would it make
sense to kill the dual server setup as we endorse now (where mod_perl is
the backend) and have only one server, with different pools.

So would something like this faster than the current schema?

Threads Pool A : lots of plain Apache threads.
Threads Pool P : a few Perl interpreters.

A thread from pool A accepts the connection and decides to handle it to
a thread from pool P, pool P does what it does and returns the result to
some thread in pool A. (We have to do the second hanling because we want
to untie the thread P asap).

With a dual server setup mod_proxy talks to the back-end server.
How this actually will work within one 2.0 server? Is there going to be
some kind mod_proxy like facility that will know to dispatch the requests?

Thanks!

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

dougm at covalent

Apr 23, 2001, 9:31 PM

Post #2 of 19 (2666 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Stas Bekman wrote:

> Hey folks,
>
> I thought about the mod_perl 2.0 performance setups. So, since now one
> process can have pools with interpreter and without it, would it make
> sense to kill the dual server setup as we endorse now (where mod_perl is
> the backend) and have only one server, with different pools.
>
> So would something like this faster than the current schema?
>
> Threads Pool A : lots of plain Apache threads.
> Threads Pool P : a few Perl interpreters.

this is not the way it works. there is one thread "pool", which apache
manages, no Perl interpreters are associated with these threads.
when a request comes in, mod_perl checks to see if it should handle
it. if so, selects an interpreter from the interpreter pool and the
current thread will then use that interpreter. for how long depends on
PerlInterpScope, which is per-request by default.

> With a dual server setup mod_proxy talks to the back-end server.
> How this actually will work within one 2.0 server? Is there going to be
> some kind mod_proxy like facility that will know to dispatch the requests?

see above.

i don't know if 2.0 will obsolete the need for dual server setup. with
the right config parameters, maybe.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 23, 2001, 9:42 PM

Post #3 of 19 (2669 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Mon, 23 Apr 2001, Doug MacEachern wrote:

> On Tue, 24 Apr 2001, Stas Bekman wrote:
>
> > Hey folks,
> >
> > I thought about the mod_perl 2.0 performance setups. So, since now one
> > process can have pools with interpreter and without it, would it make
> > sense to kill the dual server setup as we endorse now (where mod_perl is
> > the backend) and have only one server, with different pools.
> >
> > So would something like this faster than the current schema?
> >
> > Threads Pool A : lots of plain Apache threads.
> > Threads Pool P : a few Perl interpreters.
>
> this is not the way it works. there is one thread "pool", which apache
> manages, no Perl interpreters are associated with these threads.
> when a request comes in, mod_perl checks to see if it should handle
> it. if so, selects an interpreter from the interpreter pool and the
> current thread will then use that interpreter. for how long depends on
> PerlInterpScope, which is per-request by default.
>
> > With a dual server setup mod_proxy talks to the back-end server.
> > How this actually will work within one 2.0 server? Is there going to be
> > some kind mod_proxy like facility that will know to dispatch the requests?
>
> see above.
>
> i don't know if 2.0 will obsolete the need for dual server setup. with
> the right config parameters, maybe.

OK, so how mod_perl can then tell apache to delegate the generated
response to some light thread to spoon feed the client? Or can you release
the Perl interpreter back into the pool, before all of the responce has
reached the client?

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

dougm at covalent

Apr 23, 2001, 9:56 PM

Post #4 of 19 (2639 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Stas Bekman wrote:

> OK, so how mod_perl can then tell apache to delegate the generated
> response to some light thread to spoon feed the client? Or can you release
> the Perl interpreter back into the pool, before all of the responce has
> reached the client?

it might depend on the length of the script output. i don't know if the
filter chain will block if output > x bytes. but the interpreter can be
released before all data has reached the client, with 'PerlInterpScope
handler' configured. we can configure the mod_perl buffer size to be any
value before it sends data down the filter chain.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 23, 2001, 10:03 PM

Post #5 of 19 (2669 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Mon, 23 Apr 2001, Doug MacEachern wrote:

> On Tue, 24 Apr 2001, Stas Bekman wrote:
>
> > OK, so how mod_perl can then tell apache to delegate the generated
> > response to some light thread to spoon feed the client? Or can you release
> > the Perl interpreter back into the pool, before all of the responce has
> > reached the client?
>
> it might depend on the length of the script output. i don't know if the
> filter chain will block if output > x bytes. but the interpreter can be
> released before all data has reached the client, with 'PerlInterpScope
> handler' configured. we can configure the mod_perl buffer size to be any
> value before it sends data down the filter chain.

So it's possible. great, gotta try that.

Thanks, Doug!

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

dougm at covalent

Apr 23, 2001, 10:12 PM

Post #6 of 19 (2644 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Stas Bekman wrote:

> So it's possible. great, gotta try that.

its compile-time only configurable right now.
to bump it from 8k to 16k, something like:
MP_CCOPTS=-DMP_IOBUFSIZE=16384

should work.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 23, 2001, 10:21 PM

Post #7 of 19 (2638 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Mon, 23 Apr 2001, Doug MacEachern wrote:

> On Tue, 24 Apr 2001, Stas Bekman wrote:
>
> > So it's possible. great, gotta try that.
>
> its compile-time only configurable right now.
> to bump it from 8k to 16k, something like:
> MP_CCOPTS=-DMP_IOBUFSIZE=16384
>
> should work.

Oops, I've replied to your commit before I saw this email :(

cool! What about the upper limit, does Apache imposes some limit on the
max buffer size. Is it a kernel setting or else?

> MP_CCOPTS=-DMP_IOBUFSIZE=16384

Isn't it confusing when you use MP_ options for Makefile.PL *and* CCOPTS?
Looks like it'll be user-error-prone, or will it go away later on.

just thoughts...

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

dougm at covalent

Apr 23, 2001, 10:26 PM

Post #8 of 19 (2652 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Stas Bekman wrote:

> cool! What about the upper limit, does Apache imposes some limit on the
> max buffer size. Is it a kernel setting or else?

shouldn't matter for the case we're talking about, mod_perl will have
released the interpreter that generated the output before the buffer is
sent down the filter chain.

> > MP_CCOPTS=-DMP_IOBUFSIZE=16384
>
> Isn't it confusing when you use MP_ options for Makefile.PL *and* CCOPTS?
> Looks like it'll be user-error-prone, or will it go away later on.

we could easily change it to just: MP_IOBUFSIZE=16384
or a PerlIoBufsize directive if we want to make it runtime configurable.
but lets worry about that later.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 23, 2001, 10:31 PM

Post #9 of 19 (2645 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Mon, 23 Apr 2001, Doug MacEachern wrote:

> On Tue, 24 Apr 2001, Stas Bekman wrote:
>
> > cool! What about the upper limit, does Apache imposes some limit on the
> > max buffer size. Is it a kernel setting or else?
>
> shouldn't matter for the case we're talking about, mod_perl will have
> released the interpreter that generated the output before the buffer is
> sent down the filter chain.

ok

> > > MP_CCOPTS=-DMP_IOBUFSIZE=16384
> >
> > Isn't it confusing when you use MP_ options for Makefile.PL *and* CCOPTS?
> > Looks like it'll be user-error-prone, or will it go away later on.
>
> we could easily change it to just: MP_IOBUFSIZE=16384
> or a PerlIoBufsize directive if we want to make it runtime configurable.
> but lets worry about that later.

ok, I'll document it in modperl_dev.pod, right?

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

dougm at covalent

Apr 23, 2001, 10:38 PM

Post #10 of 19 (2640 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Stas Bekman wrote:

> ok, I'll document it in modperl_dev.pod, right?

sure, if you've tested that it works :)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 23, 2001, 10:48 PM

Post #11 of 19 (2641 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Mon, 23 Apr 2001, Doug MacEachern wrote:

> On Tue, 24 Apr 2001, Stas Bekman wrote:
>
> > ok, I'll document it in modperl_dev.pod, right?
>
> sure, if you've tested that it works :)

I'm not sure how to test it now, but 'make test' passes for me...

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

khera at kcilink

Apr 24, 2001, 6:40 AM

Post #12 of 19 (2646 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

>>>>> "SB" == Stas Bekman <stas[at]stason.org> writes:

SB> sense to kill the dual server setup as we endorse now (where mod_perl is
SB> the backend) and have only one server, with different pools.

Another benefit of the front/back end setup is that the ligthweight
front-ends buffer the output from the heavy mod_perl processes,
freeing them up quickly to handle more requests. You don't want
mod_perl talking directly to slow clients.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 24, 2001, 6:47 AM

Post #13 of 19 (2639 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Vivek Khera wrote:

> >>>>> "SB" == Stas Bekman <stas[at]stason.org> writes:
>
> SB> sense to kill the dual server setup as we endorse now (where mod_perl is
> SB> the backend) and have only one server, with different pools.
>
> Another benefit of the front/back end setup is that the ligthweight
> front-ends buffer the output from the heavy mod_perl processes,
> freeing them up quickly to handle more requests. You don't want
> mod_perl talking directly to slow clients.

Heh, Vivek :) That's exactly the gist of this thread... That's why we are
talking about buffer sizes :)

with 2.0 you can get the same effeciency as you have had before with a
dual server setup.

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://logilune.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

dougm at covalent

Apr 24, 2001, 9:05 PM

Post #14 of 19 (2660 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Tue, 24 Apr 2001, Stas Bekman wrote:

> with 2.0 you can get the same effeciency as you have had before with a
> dual server setup.

i would make that claim just yet, until its proven. in theory though, it
will be possible.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

prcotter at bellsouth

Apr 25, 2001, 3:23 AM

Post #15 of 19 (2651 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

> >
> > Another benefit of the front/back end setup is that the ligthweight
> > front-ends buffer the output from the heavy mod_perl processes,
> > freeing them up quickly to handle more requests. You don't want
> > mod_perl talking directly to slow clients.
>

Hi - Can you point me at a document that talks about the splitting between
mod-perl and
non-mp request handling.?

Regards - Paul Cotter

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 25, 2001, 4:14 AM

Post #16 of 19 (2646 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Wed, 25 Apr 2001, Paul Cotter wrote:

>
> > >
> > > Another benefit of the front/back end setup is that the ligthweight
> > > front-ends buffer the output from the heavy mod_perl processes,
> > > freeing them up quickly to handle more requests. You don't want
> > > mod_perl talking directly to slow clients.
> >
>
> Hi - Can you point me at a document that talks about the splitting between
> mod-perl and
> non-mp request handling.?

http://perl.apache.org/guide/strategy.html
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://eXtropia.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

khera at kcilink

Apr 25, 2001, 7:16 AM

Post #17 of 19 (2644 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

>>>>> "PC" == Paul Cotter <prcotter[at]bellsouth.net> writes:

>> > Another benefit of the front/back end setup is that the ligthweight
>> > front-ends buffer the output from the heavy mod_perl processes,
>> > freeing them up quickly to handle more requests. You don't want
>> > mod_perl talking directly to slow clients.
>>

PC> Hi - Can you point me at a document that talks about the splitting between
PC> mod-perl and
PC> non-mp request handling.?

The mod_perl guide, and my original tuning docs which are part of your
mod_perl distribution called, appropriately enough,
mod_perl_tuning.pod. Just run it through perldoc and read away.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

stas at stason

Apr 25, 2001, 8:02 AM

Post #18 of 19 (2663 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

On Wed, 25 Apr 2001, Vivek Khera wrote:

> >>>>> "PC" == Paul Cotter <prcotter[at]bellsouth.net> writes:
>
> >> > Another benefit of the front/back end setup is that the ligthweight
> >> > front-ends buffer the output from the heavy mod_perl processes,
> >> > freeing them up quickly to handle more requests. You don't want
> >> > mod_perl talking directly to slow clients.
> >>
>
> PC> Hi - Can you point me at a document that talks about the splitting between
> PC> mod-perl and
> PC> non-mp request handling.?
>
> The mod_perl guide, and my original tuning docs which are part of your
> mod_perl distribution called, appropriately enough,
> mod_perl_tuning.pod. Just run it through perldoc and read away.

Yup, Vivek is the godfather of this cool setup. At least I've learned
about it from his pod!

_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas[at]stason.org http://apachetoday.com http://logilune.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

khera at kcilink

Apr 25, 2001, 8:06 AM

Post #19 of 19 (2637 views)
Permalink

Re: dual server setup with 2.0? [In reply to]

>>>>> "SB" == Stas Bekman <stas[at]stason.org> writes:

SB> Yup, Vivek is the godfather of this cool setup. At least I've learned
SB> about it from his pod!

Most of the ideas came from discussions on the mod-perl list; I can't
really take creadit for inventing them, just formalizing them and
writing up examples, and testing them in production in desperation for
getting some sites working ;-)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe[at]perl.apache.org
For additional commands, e-mail: dev-help[at]perl.apache.org

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com