Mailing List Archive: Catalyst: Users

Multiple instances of same app with 5.80 under mod_perl

Index | Next | Previous | View Threaded

stephenclouse at gmail

May 27, 2009, 10:12 AM

Post #1 of 13 (1717 views)
Permalink

Multiple instances of same app with 5.80 under mod_perl

Hello all,

I am fairly new to Catalyst and am currently evaluating it for use on
several projects, a couple of them being conversions of an existing system
to use Catalyst instead of the hacked-together in-house framework currently
in use. Those apps have the old per-customer/mass-blog-hosting model going
-- the exact same app deployed with different database/filesystem path
configurations (currently set using mod_perl configuration directives inside
<VirtualHost> sections in Apache config), previously accomplished with
Catalyst either with FastCGI deployment or ACCEPT_CONTEXT magic that I don't
quite understand yet.

I have seen hints dropped in places such as
http://osdir.com/ml/lang.perl.modules.dbix-class/2006-08/msg00188.html that
Catalyst 5.80 has gained more explicit support for such a deployment model
under mod_perl, but so far I have come up empty on finding references or
examples of how to actually do it.

Some guidance would be greatly appreciated.

--
Stephen Clouse <stephenclouse [at] gmail>

swatt at infobal

May 27, 2009, 11:18 AM

Post #2 of 13 (1665 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

I did a simple tweak to the configuration file management so environment
variables could be substituted into configuration files. All I added to
MyApp.pm was:

__PACKAGE__->config('Plugin::ConfigLoader' =>
{ file => 'MyApp.yaml',
substitutions => {
ENV => sub {
my ($c, $v) = @_;
if (! defined($ENV{$v})) {
die("Missing environment variable: $v");
} else {
return $ENV{$v};
}
}
}
}
);

Then, I could use YAML settings like:

session:
storage: '__ENV(TEMP)__/sessions'

It was pretty easy for us to get web servers to set environment
variables, so this trick allowed us to have multiple systems with
different database and file directories set pretty straightforwardly.

This may be one approach you could use, but there are likely better
ones. Especially since I'm still stuck at 5.70 waiting for a good
opportunity to upgrade without anyone (even my colleagues) noticing.
This is a 5.70 solution, but should be OK in 5.80, I would guess.

All the best
Stuart

Stephen Clouse wrote:
> Hello all,
>
> I am fairly new to Catalyst and am currently evaluating it for use on
> several projects, a couple of them being conversions of an existing
> system to use Catalyst instead of the hacked-together in-house
> framework currently in use. Those apps have the old
> per-customer/mass-blog-hosting model going -- the exact same app
> deployed with different database/filesystem path configurations
> (currently set using mod_perl configuration directives inside
> <VirtualHost> sections in Apache config), previously accomplished with
> Catalyst either with FastCGI deployment or ACCEPT_CONTEXT magic that I
> don't quite understand yet.
>
> I have seen hints dropped in places such as
> http://osdir.com/ml/lang.perl.modules.dbix-class/2006-08/msg00188.html
> that Catalyst 5.80 has gained more explicit support for such a
> deployment model under mod_perl, but so far I have come up empty on
> finding references or examples of how to actually do it.
>
> Some guidance would be greatly appreciated.
>
> --
> Stephen Clouse <stephenclouse [at] gmail <mailto:stephenclouse [at] gmail>>
>
> --
> This message was scanned by ESVA and is believed to be clean.
> Click here to report this message as spam.
> <http://antispam.infobal.com/cgi-bin/learn-msg.cgi?id=64AE2280C1.6F32B>
> ------------------------------------------------------------------------
>
> _______________________________________________
> List: Catalyst [at] lists
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
> Dev site: http://dev.catalyst.perl.org/
>

--
Stuart Watt
ARM Product Developer
Information Balance

mdietrich at cpan

May 27, 2009, 4:40 PM

Post #3 of 13 (1663 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Hi Stuart,

do you use Log::Log4perl in your Catalyst apps? Months ago I wrote a
message to the list about problems with Log::Log4perl and multiple
instances of Catalyst within one Apache. Nobody answered and because
it's currently not very important to me (not in production environment
yet) I didn't follow-up.

The problem was to separate each VirtualHosts Catalyst into different
log files which didn't worked. Catalyst always logged into the file
from the latest VirtualHost. Maybe I have some short time tomorrow to
create a local test case (I use the included server now so I have to
switch to Apache again).

Regards,
matt

--
rainboxx Matthias Dietrich
Freier Software Engineer

rainboxx | Tel.: +49 (0) 151 / 50 60 78 64
T�lzer Str. 19 | Mail: matt [at] rainboxx
70372 Stuttgart | WWW : http://www.rainboxx.de

XING: https://www.xing.com/profile/Matthias_Dietrich18
GULP: http://www.gulp.de/profil/rainboxx.html

Attachments:

PGP.sig (0.19 KB)

bobtfish at bobtfish

May 28, 2009, 10:00 AM

Post #4 of 13 (1656 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Stephen Clouse wrote:
> I have seen hints dropped in places such as
> http://osdir.com/ml/lang.perl.modules.dbix-class/2006-08/msg00188.html
> that Catalyst 5.80 has gained more explicit support for such a
> deployment model under mod_perl, but so far I have come up empty on
> finding references or examples of how to actually do it.
>
> Some guidance would be greatly appreciated.

Hiya.

Sorry to let you down, but this mail was written in 2006, and contains
the plan as it was then.

Unfortunately, nobody has stepped up to the plate to do the requisite
work to split the application class and context (which is what that mail
is hinting at).

The work done in 5.80 makes this _significantly_ easier than it used to
be if someone wants to step up to help tackle it - I think more work is
involved munging the test suite than actually hacking Catalyst itself..

It _is_ still possible to have one application instance serving many
vhosts, and many of the parts are there - supporting this inside your
application isn't too difficult, but unfortunately you can't yet get it
'out of the box'.

Cheers
t0m

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

bobtfish at bobtfish

May 28, 2009, 10:03 AM

Post #5 of 13 (1645 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Stuart Watt wrote:
> I did a simple tweak to the configuration file management so environment
> variables could be substituted into configuration files. All I added to
> MyApp.pm was:
<snip>
> Then, I could use YAML settings like:
>
> session:
> storage: '__ENV(TEMP)__/sessions'
>
> It was pretty easy for us to get web servers to set environment
> variables, so this trick allowed us to have multiple systems with
> different database and file directories set pretty straightforwardly.

That's a nice trick, it would be great to see it documented. Do you
fancy writing up a wiki page, or adding something to the manual, or even
patching ConfigLoader so that it supports a syntax like this by default?

> This may be one approach you could use, but there are likely better
> ones. Especially since I'm still stuck at 5.70 waiting for a good
> opportunity to upgrade without anyone (even my colleagues) noticing.
> This is a 5.70 solution, but should be OK in 5.80, I would guess.

Yes, that'll work in 5.80, but still won't solve the issue at hand.

If you try to run multiple copies of the same application in the same
perl process, it won't work like you expect - the setting will come from
the first application loaded..

Your solution just makes it much easier to have multiple deployment
environments with different configurations, but these environments still
have to have a different perl interpreter each.

Cheers
t0m

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

bobtfish at bobtfish

May 28, 2009, 10:05 AM

Post #6 of 13 (1643 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Matthias Dietrich wrote:
> do you use Log::Log4perl in your Catalyst apps? Months ago I wrote a
> message to the list about problems with Log::Log4perl and multiple
> instances of Catalyst within one Apache. Nobody answered and because
> it's currently not very important to me (not in production environment
> yet) I didn't follow-up.
>
> The problem was to separate each VirtualHosts Catalyst into different
> log files which didn't worked. Catalyst always logged into the file
> from the latest VirtualHost. Maybe I have some short time tomorrow to
> create a local test case (I use the included server now so I have to
> switch to Apache again).

This is to be expected.

The instance of the logging class to use is stored in class data inside
your application class.

One solution to this would be to keep a hash of log class instances per
vhost, and then ensure to replace the logger being used at the start of
each hit - this should do what you're looking for..

Cheers
t0m

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

swatt at infobal

May 28, 2009, 10:42 AM

Post #7 of 13 (1652 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Tomas Doran wrote:
> If you try to run multiple copies of the same application in the same
> perl process, it won't work like you expect - the setting will come
> from the first application loaded..
>
> Your solution just makes it much easier to have multiple deployment
> environments with different configurations, but these environments
> still have to have a different perl interpreter each.
True, we did need to keep each application using processes which are
separate. This was an issue when we used PerlEx (the ActivePerl style
mod_perl approach) but we now use FastCGI (clients use Windows, etc.)
and IIS freely creates many processes, and makes it really easy to set
environment variables for them from its configuration files. The IIS
FastCGI mechanism was developed closely with Zend, and I guess PHP faced
the same kind of issue, and this was their solution.

I'll wikify the config trick for now; we're close to production
deadlines right now, but I do want to get this out there!

--S
--
Stuart Watt
ARM Product Developer
Information Balance

stephenclouse at gmail

May 28, 2009, 3:48 PM

Post #8 of 13 (1643 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

On Thu, May 28, 2009 at 12:00 PM, Tomas Doran <bobtfish [at] bobtfish> wrote:

> Sorry to let you down, but this mail was written in 2006, and contains the
> plan as it was then.
>
> Unfortunately, nobody has stepped up to the plate to do the requisite work
> to split the application class and context (which is what that mail is
> hinting at).

/sadpanda :(

> The work done in 5.80 makes this _significantly_ easier than it used to be
> if someone wants to step up to help tackle it - I think more work is
> involved munging the test suite than actually hacking Catalyst itself..

Hmm....well, something I might get to eventually, but probably not a project
for Still-Reading-Tutorials Guy.

> It _is_ still possible to have one application instance serving many
> vhosts, and many of the parts are there - supporting this inside your
> application isn't too difficult, but unfortunately you can't yet get it 'out
> of the box'.

It seems to me it could be done almost entirely within mod_perl using some
interpreter pool trickery and by deferring AppClass->setup(). I assume that
is the call in the base package that instantiates everything for the
Catalyst app, yes?

Commenting that out, then theoretically (note that this is totally untested
at the moment, but I intend to tomorrow):

PerlModule AppClass

<VirtualHost foo>
PerlOptions +Clone
PerlSetVar catalyst_instance foo
SetHandler modperl
PerlResponseHandler AppClass
</VirtualHost>

<VirtualHost bar>
PerlOptions +Clone
PerlSetVar catalyst_instance bar
SetHandler modperl
PerlResponseHandler AppClass
</VirtualHost>

And then a custom mp handler that does AppClass->setup() with an appropriate
config (based on catalyst_instance) on the first request and then passes off
to AppClass->handle_request. By deferring the instantiation you restrict
the package globals to the virtual host's interpreter pool (so multiple apps
won't collide), but still gain the optree sharing by cloning from the
top-level interpreter with all the Catalyst code pre-loaded.

I welcome anyone crazy enough to try this before I do :)

--
Stephen Clouse <stephenclouse [at] gmail>

bobtfish at bobtfish

May 29, 2009, 2:29 AM

Post #9 of 13 (1622 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Stuart Watt wrote:
> True, we did need to keep each application using processes which are
> separate. This was an issue when we used PerlEx (the ActivePerl style
> mod_perl approach) but we now use FastCGI (clients use Windows, etc.)
> and IIS freely creates many processes, and makes it really easy to set
> environment variables for them from its configuration files. The IIS
> FastCGI mechanism was developed closely with Zend, and I guess PHP faced
> the same kind of issue, and this was their solution.

So I guess that IIS starts a different set of FCGI processes per vhost.

Which will fix all your problems nicely as everything exists in it's own
process space, and each process only serves the vhost it was brought up
to serve.

I can't see this strategy working so good if you had several hundred
active vhosts at the same time however...

200x copies of the same application per server won't work so nice...

Or IIS would be tearing down and starting new FCGI processes per request
if the max processes limit is < no of active vhosts.. That'd be, well,
CGI then..

So yes, for most uses, this is a 'get out of jail free' card, but
doesn't in any way solve the core architectural issue at hand.

> I'll wikify the config trick for now; we're close to production
> deadlines right now, but I do want to get this out there!

That's great, thank you very much!

Cheers
t0m

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

stephenclouse at gmail

May 29, 2009, 9:49 AM

Post #10 of 13 (1622 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

On Thu, May 28, 2009 at 5:48 PM, Stephen Clouse <stephenclouse [at] gmail>wrote:

> Commenting that out, then theoretically (note that this is totally untested
> at the moment, but I intend to tomorrow):
>

Well, it was worth a try, but obviously too much voodoo. Catalyst complains
about not being able to find //_DISPATCH. Something internal gets fried
with the delayed setup call.

This, on the other hand, works perfectly, although less memory-efficient --
one full compile of Catalyst per vhost. Even so, this should be a win over
the traditional FastCGI deployment (since the optree will at least still be
shared within the vhost interpreter pool).

<VirtualHost foo>
PerlOptions +Parent
PerlSwitches -I/path/to/myapp/lib
PerlSetEnv MYAPP_CONFIG foo.conf
SetHandler modperl
PerlResponseHandler MyApp
</VirtualHost>

<VirtualHost bar>
PerlOptions +Parent
PerlSwitches -I/path/to/myapp/lib
PerlSetEnv MYAPP_CONFIG bar.conf
SetHandler modperl
PerlResponseHandler MyApp
</VirtualHost>

--
Stephen Clouse <stephenclouse [at] gmail>

mdietrich at cpan

May 31, 2009, 8:19 AM

Post #11 of 13 (1599 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Hi,

> One solution to this would be to keep a hash of log class instances
> per vhost, and then ensure to replace the logger being used at the
> start of each hit - this should do what you're looking for..

yes, I thought I have to do it in such a way. Where would be the best
point to get the right logger?

matt

--
rainboxx Matthias Dietrich
Freier Software Engineer

rainboxx | Tel.: +49 (0) 151 / 50 60 78 64
T�lzer Str. 19 | Mail: matt [at] rainboxx
70372 Stuttgart | WWW : http://www.rainboxx.de

XING: https://www.xing.com/profile/Matthias_Dietrich18
GULP: http://www.gulp.de/profil/rainboxx.html

Attachments:

PGP.sig (0.19 KB)

bobtfish at bobtfish

Jun 1, 2009, 5:08 AM

Post #12 of 13 (1582 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

Matthias Dietrich wrote:
> Hi,
>
>> One solution to this would be to keep a hash of log class instances
>> per vhost, and then ensure to replace the logger being used at the
>> start of each hit - this should do what you're looking for..
>
> yes, I thought I have to do it in such a way. Where would be the best
> point to get the right logger?

I'm thinking something like this:

{
my %loggers;
after prepare => sub {
my $class = shift;
my $host = $c->req->hoatname
my $logger = $loggers{host} ||= My::Logger->new( logfile =>
$class->path_to('logs', "$host.log") );
$class->log( $logger );
};
}

(untested)

Cheers
t0m

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

dbix-class at trout

Jun 8, 2009, 11:46 AM

Post #13 of 13 (1450 views)
Permalink

Re: Multiple instances of same app with 5.80 under mod_perl [In reply to]

On Fri, May 29, 2009 at 11:49:45AM -0500, Stephen Clouse wrote:
> On Thu, May 28, 2009 at 5:48 PM, Stephen Clouse <stephenclouse [at] gmail>wrote:
>
> > Commenting that out, then theoretically (note that this is totally untested
> > at the moment, but I intend to tomorrow):
> >
>
> Well, it was worth a try, but obviously too much voodoo. Catalyst complains
> about not being able to find //_DISPATCH. Something internal gets fried
> with the delayed setup call.
>
> This, on the other hand, works perfectly, although less memory-efficient --
> one full compile of Catalyst per vhost. Even so, this should be a win over
> the traditional FastCGI deployment (since the optree will at least still be
> shared within the vhost interpreter pool).

I doubt it.

FastCGI isn't "traditional" for Catalyst - it's what people switched to
because mod_perl almost never has a better memory/performance profile.

Especially since most of the mod_perl cleverness relies on interpreter
cloning, which unshares a huge amount of stuff that -can- be shared across
a fork.

--
Matt S Trout Catalyst and DBIx::Class consultancy with a clue
Technical Director and a commit bit: http://shadowcat.co.uk/catalyst/
Shadowcat Systems Limited
mst (@) shadowcat.co.uk http://shadowcat.co.uk/blog/matt-s-trout/

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads