Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Catalyst: Users

Using Catalyst with mod_per or FastCGI on heavy traffic web application

 

 

Catalyst users RSS feed   Index | Next | Previous | View Threaded


shanu_040 at yahoo

Jan 29, 2010, 3:05 AM

Post #1 of 14 (2604 views)
Permalink
Using Catalyst with mod_per or FastCGI on heavy traffic web application

Does anyone have any advice on what will the best option mod_perl or FastCGI or something else. if I have the following development/production environment for my web application which is a search engine.
1. Linux(RHEL5)
2. Apache 2.2.x
3. Perl 5.10
4. mod_perl 2.0.x
5. mysql 5.1.x
6. Catalyst 5.8.x










Genius at work               


The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/


nabble at mackler

Jan 29, 2010, 4:32 PM

Post #2 of 14 (2510 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Well, I'm sure no expert, but that doesn't stop me from having opinions.

The reasons I stopped using mod_perl are: safer when perl encounters
errors, easier for learning, easier for development, better error
messages when restarting production applications, and the ability to
have each application run as a different system user.

Perhaps the most compelling reason that I stopped using mod_perl is
because with both perl and the web server combined in one executable,
a problem with perl can cause not only the whole website to go down,
but also all the other websites handled by that same webserver to go
down as well.

Second, setting up staging servers so you can develop without touching
your production site is easier using fastcgi. Even having one staging
server with mod_perl is difficult. Will you run two whole apaches
simultaneously? What if you want to have more than one staging
server? Adding more staging servers gets ridiculous fast.

(BTW, I know the Catalyst Book says that you're supposed to use the
myapp_server.pl for development, but I want to have SSL turned on all
the time.)

With fastcgi, I literally had dozens of different applications running
at once, and they can all crash and burn and the webserver keeps
running. In fact, that's how I learned to use Catalyst. For every
tutorial I found, I made its own fastcgi process, and then I set up
the web server to know about each one. http://myhost.com/alpha was
the authorization tutorial, /beta was the CRUD totorial, etc. I could
look at and play with any or all of them running at once; it made
learning from examples much easier.

In addition to making it easier to learn and to development, in the
production environment fastcgi has significant advantages as relates
to error messages, restarting, and security: I can have two levels of
error messages: one is the replacement for the "Internal Server Error"
that results from a problem with perl running--the same error you
arleady have, for example if your database server crashes, generated
by the perl application. But with fastcgi I have another, separate
error page that is a nice-looking static page served by the web server
when the fasccgi server is not there. So during the time when I am
restarting my fastcgi application, visitors see that nice static page
rather than getting a browser error message, which is what happens
when you restart a server with mod_perl. And as your application(s)
grow in size (and number), restarting them takes longer and longer, so
that feature becomes more important.

Finally, a wonderful benefit of using fastcgi is that each one of my
fastcgi applications runs as a separate user, and none of those
fastcgi users is the user that the web server runs as. I sleep that
much better at night knowing that the web server cannot read the files
that have database passwords in them, and so on.

Anyway, that's my conclusion after doing things both ways. My current
setup is similar to yours: FreeBSD, lighttpd, Catalyst, and
PostgreSQL.

Adam

PS, I'm curious why you're using mysql. Is there a way in which its
better than Postgres?

On Fri, Jan 29, 2010 at 04:35:10PM +0530, Amit Jha wrote:
> Does anyone have any advice on what will the best option mod_perl or
> FastCGI or something else. if I have the following
> development/production environment for my web application which is a
> search engine.
> 1. Linux(RHEL5)
> 2. Apache 2.2.x
> 3. Perl 5.10
> 4. mod_perl 2.0.x
> 5. mysql 5.1.x
> 6. Catalyst 5.8.x
> _______________________________________________
> List: Catalyst [at] lists
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
> Dev site: http://dev.catalyst.perl.org/


_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


xenoterracide at gmail

Jan 29, 2010, 5:22 PM

Post #3 of 14 (2512 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Enlightening how do you feel about fastcgi vs psgi?


_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


hdp.perl.catalyst.users at weftsoar

Jan 29, 2010, 5:48 PM

Post #4 of 14 (2507 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Excerpts from xenoterracide [at] gmail's message of Fri Jan 29 20:22:48 -0500 2010:
> Enlightening how do you feel about fastcgi vs psgi?

This question makes no sense. How do you feel about HTML vs. HTTP?

PSGI is an interface for Perl code. FastCGI is an interface for network
communications. They aren't in the same problem space, except to the degree
that there are FastCGI servers that then run PSGI applications.

hdp.

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


moseley at hank

Jan 29, 2010, 6:14 PM

Post #5 of 14 (2514 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Just so it's not one-sided, I moved from FastCGI to mod_perl some years
back. Start up and restart time was one issue, IIRC, but mod_perl was
trivial to configure and solved stability issues we were seeing. I never
went back, so maybe it's better now. Is there now a manager that will spawn
more FastCGI processes based on load?

The Catalyst server is used for development. The app knows it's running
with that engine and disables the SSL requirements, but logs that the page
would be blocked if not in SSL.

Starting different configurations is no problem at all. I have an app
startup script that generates the httpd.conf file from configuration files
at startup time. It's basically start_app --app=foo --mode=testing. The
apps have dev, testing, qa, stating, and production modes, so as you can see
it's not a big deal. All those are on different machines, but different
modes can use different ports, of course, so can be on the same machine.

I'm usingCronolog so there's no log rotation/restart needs.

mod_perl does seem a bit heavy weight just to serve requests, especially
since all we use is the response handler. We also don't use many modules
(SSL is not handled by Apache). I've meant to look at other server options,
but there's not pressing need.

I don't really see how what server you use effects error messages. Stderr
is stderr. I would never use the canned Apache error responses anyway for
a site.

True, Apache can run as only one user. But, that's normally the apache (or
whatever) user which has very limited access, which is what you want. So, I
don't see a benefit of running as different users. Apps may need to run on
multiple machines so only part of the file system need access to would be
for temp files.

I think mod_perl is daunting to some due to all the possibilities (and
default httpd.conf Apache provides) but really the config required to get it
to serve a Catalyst app is just a few lines. My entire files are pretty
small:

$ fgrep -v '#' httpd.conf | grep -v '^$' |grep -v LoadModule | grep -v
'Module>' | wc -l
40




--
Bill Moseley
moseley [at] hank


xenoterracide at gmail

Jan 29, 2010, 7:36 PM

Post #6 of 14 (2513 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Hmm. Perhaps i misunderstand the concept. I was thinking there was the third option of using a psgi server or mod psgi
-----Original Message-----
Date: Friday, January 29, 2010 8:50:07 pm
To: "catalyst" <catalyst [at] lists>
From: "Hans Dieter Pearcey" <hdp.perl.catalyst.users [at] weftsoar>
Subject: Re: [Catalyst] Using Catalyst with mod_per or FastCGI on heavy traffic web application

Excerpts from xenoterracide [at] gmail's message of Fri Jan 29 20:22:48 -0500 2010:
> Enlightening how do you feel about fastcgi vs psgi?

This question makes no sense. How do you feel about HTML vs. HTTP?

PSGI is an interface for Perl code. FastCGI is an interface for network
communications. They aren't in the same problem space, except to the degree
that there are FastCGI servers that then run PSGI applications.

hdp.

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/



_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


nabble at mackler

Jan 29, 2010, 10:59 PM

Post #7 of 14 (2496 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

On Fri, Jan 29, 2010 at 06:14:09PM -0800, Bill Moseley wrote:
> I don't really see how what server you use effects error messages. Stderr
> is stderr. I would never use the canned Apache error responses anyway for
> a site.
> --
> Bill Moseley

Oops, I see did not explain that point clearly.

By error messages, I was refering to error messages that visitors to
my website see, not the kind that I see in my log.

With mod_perl, if your script has an error, for example if your
database server goes away, you'll either get the default "Internal
Server Error" message, or (hopefully) a customized error message that
looks like it's part of your website. Same using fastcgi.

But with mod_perl, when you're restarting your application, you're
starting the whole web server, so during that time...which can be
longer than you expect for a number of reasons...people attempting to
reach your site will get browser errors saying your site is down or
unreachable.

But with fastcgi, the web server keeps running and can serve a static
error page while the fastcgi server is not available. And with no
perl in the mix, restarting the web server itself takes less time and
can be done more gracefully.

Adam Mackler

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


hdp.perl.catalyst.users at weftsoar

Jan 30, 2010, 4:28 AM

Post #8 of 14 (2488 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Excerpts from xenoterracide [at] gmail's message of Fri Jan 29 22:36:53 -0500 2010:
> Hmm. Perhaps i misunderstand the concept. I was thinking there was the third
> option of using a psgi server or mod psgi

I didn't see any reference to either, but it doesn't matter; comparing either
of those as "PSGI" to FastCGI still doesn't make sense. "A psgi server" would
probably mean a standalone http server, roughly equivalent to
Catalyst::Engine::HTTP or HTTP::Prefork, and mod_psgi is an Apache module.

Your email client is misconfigured or something, btw; you're starting new
threads every time you reply to this one.

hdp.

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


xenoterracide at gmail

Jan 30, 2010, 6:39 AM

Post #9 of 14 (2473 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

No it's just a crappy one on a cell phone. it's my only way to to access my email and internet on a regular basis. unfortately i can't configure it correctly.
-----Original Message-----
Date: Saturday, January 30, 2010 7:30:03 am
To: "catalyst" <catalyst [at] lists>
From: "Hans Dieter Pearcey" <hdp.perl.catalyst.users [at] weftsoar>
Subject: Re: [Catalyst] Using Catalyst with mod_per or FastCGI on heavy traffic web application

Excerpts from xenoterracide [at] gmail's message of Fri Jan 29 22:36:53 -0500 2010:
> Hmm. Perhaps i misunderstand the concept. I was thinking there was the third
> option of using a psgi server or mod psgi

I didn't see any reference to either, but it doesn't matter; comparing either
of those as "PSGI" to FastCGI still doesn't make sense. "A psgi server" would
probably mean a standalone http server, roughly equivalent to
Catalyst::Engine::HTTP or HTTP::Prefork, and mod_psgi is an Apache module.

Your email client is misconfigured or something, btw; you're starting new
threads every time you reply to this one.

hdp.

_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/



_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


moseley at hank

Jan 30, 2010, 7:30 AM

Post #10 of 14 (2472 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

On Fri, Jan 29, 2010 at 10:59 PM, Adam Mackler <nabble [at] mackler> wrote:

>
>
> But with mod_perl, when you're restarting your application, you're
> starting the whole web server, so during that time...which can be
> longer than you expect for a number of reasons...people attempting to
> reach your site will get browser errors saying your site is down or
> unreachable.


> But with fastcgi, the web server keeps running and can serve a static
> error page while the fastcgi server is not available. And with no
> perl in the mix, restarting the web server itself takes less time and
> can be done more gracefully.
>

That makes no sense. If fastcgi processes are down, then, well they are
down. If all your mod_perl servers are down, they are, well, down.

Maybe you are assuming the Apache that is running mod_perl also serves
static content? Or that there's only one web server? Or there's no load
balancer? Or it's not run with a reverse proxy as is the standard approach?

We found that restarting with FastCGI took longer. Maybe that was because
each process has to compile the app (instead of compile and fork to many
processes?). Is that true? I can't remember.




--
Bill Moseley
moseley [at] hank


shanu_040 at yahoo

Jan 31, 2010, 8:36 PM

Post #11 of 14 (2295 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Let me tell you that I am going to deploy 3-tire architecture and
inbuilt application load balancer for my production environment. And In
future I may have h/w load balancer as well. So Its clear thal I am looking at
very fast and stable environment to deliver my content. So Please help me to identify the best better
option.

Amit
               

--- On Sat, 30/1/10, Adam Mackler <nabble [at] mackler> wrote:

From: Adam Mackler <nabble [at] mackler>
Subject: Re: [Catalyst] Using Catalyst with mod_per or FastCGI on heavy traffic web application
To: shanu_040 [at] yahoo, "The elegant MVC web framework" <catalyst [at] lists>
Date: Saturday, 30 January, 2010, 6:02 AM

Well, I'm sure no expert, but that doesn't stop me from having opinions.

The reasons I stopped using mod_perl are: safer when perl encounters
errors, easier for learning, easier for development, better error
messages when restarting production applications, and the ability to
have each application run as a different system user.

Perhaps the most compelling reason that I stopped using mod_perl is
because with both perl and the web server combined in one executable,
a problem with perl can cause not only the whole website to go down,
but also all the other websites handled by that same webserver to go
down as well.

Second, setting up staging servers so you can develop without touching
your production site is easier using fastcgi.  Even having one staging
server with mod_perl is difficult.  Will you run two whole apaches
simultaneously?  What if you want to have more than one staging
server?  Adding more staging servers gets ridiculous fast.

(BTW, I know the Catalyst Book says that you're supposed to use the
myapp_server.pl for development, but I want to have SSL turned on all
the time.)

With fastcgi, I literally had dozens of different applications running
at once, and they can all crash and burn and the webserver keeps
running.  In fact, that's how I learned to use Catalyst.  For every
tutorial I found, I made its own fastcgi process, and then I set up
the web server to know about each one.  http://myhost.com/alpha was
the authorization tutorial, /beta was the CRUD totorial, etc.  I could
look at and play with any or all of them running at once; it made
learning from examples much easier.

In addition to making it easier to learn and to development, in the
production environment fastcgi has significant advantages as relates
to error messages, restarting, and security: I can have two levels of
error messages: one is the replacement for the "Internal Server Error"
that results from a problem with perl running--the same error you
arleady have, for example if your database server crashes, generated
by the perl application.  But with fastcgi I have another, separate
error page that is a nice-looking static page served by the web server
when the fasccgi server is not there.  So during the time when I am
restarting my fastcgi application, visitors see that nice static page
rather than getting a browser error message, which is what happens
when you restart a server with mod_perl.  And as your application(s)
grow in size (and number), restarting them takes longer and longer, so
that feature becomes more important.

Finally, a wonderful benefit of using fastcgi is that each one of my
fastcgi applications runs as a separate user, and none of those
fastcgi users is the user that the web server runs as.  I sleep that
much better at night knowing that the web server cannot read the files
that have database passwords in them, and so on.

Anyway, that's my conclusion after doing things both ways.  My current
setup is similar to yours:  FreeBSD, lighttpd, Catalyst, and
PostgreSQL.

Adam

PS, I'm curious why you're using mysql.  Is there a way in which its
better than Postgres?

On Fri, Jan 29, 2010 at 04:35:10PM +0530, Amit Jha wrote:
> Does anyone have any advice on what will the best option mod_perl or
> FastCGI or something else. if I have the following
> development/production environment for my web application which is a
> search engine.
> 1. Linux(RHEL5)
> 2. Apache 2.2.x
> 3. Perl 5.10
> 4. mod_perl 2.0.x
> 5. mysql 5.1.x
> 6. Catalyst 5.8.x
> _______________________________________________
> List: Catalyst [at] lists
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
> Dev site: http://dev.catalyst.perl.org/




Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/


catalyst at fadetoblack

Feb 1, 2010, 2:07 AM

Post #12 of 14 (2274 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavytraffic web application [In reply to]

Adam Mackler wrote:
> Finally, a wonderful benefit of using fastcgi is that each one of my
> fastcgi applications runs as a separate user, and none of those
> fastcgi users is the user that the web server runs as. I sleep that
> much better at night knowing that the web server cannot read the files
> that have database passwords in them, and so on.

It's more likely that any security hole will be in the perl application
rather than the web server, so your database password is equally exposed
with either method.

In any case you should be able to make your app root read-only - which will
mean that the apache child processes can't read the files.

Carl


_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


catalyst at fadetoblack

Feb 1, 2010, 3:01 AM

Post #13 of 14 (2273 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Another mod_perl user here! I've looked at FastCGI, but ongoing management
has always looked to be more complecated than just altering the apache
config.

We have a multi-server setup with hardware load-balancers. They balance
between two threaded apache servers which serve all static files and then
using mod_proxy, pass other requests via the load balancer again to the
three apache/mod_perl/catalyst servers. (We did it with mod_proxy_balancer
for a while, but found the hardware load-balancer did a better job).

The front apache servers are configured for maximum threads, with Keep-Alive
on and very high connection timeouts. The mod_perl servers are configured
with the usual pre-fork and Keep-Alive off. We maximise buffers in the
front-end to release the mod_perl process as soon as they can to handle
another request.

We generally restart the mod_perl servers in turn, so although there's a
slow restart that's invisible to end users. Should there be a problem with
all the servers the front apache gets a 502 proxy error, so we replace that
with a pretty page.

One of the advantages of mod_perl is that you only have a single instance of
perl running, so you only have code in common to your apps loaded once (so
that covers everything from core and CPAN as well as bespoke common code.)

If we want to run a slightly different version of a codebase we use
"PerlOptions +Parent" which gives you a separate perl interpreter for a
particular vhost.

Now I wouldn't say this is a good setup for *everybody* to use, but
certainly there's a class of users where this type of setup will work very
well.

Carl


_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/


igavus at gmail

Feb 1, 2010, 11:12 AM

Post #14 of 14 (2227 views)
Permalink
Re: Using Catalyst with mod_per or FastCGI on heavy traffic web application [In reply to]

Hi,

I'm quite happy with nginx load-balancing in front on FastCGI servers - it
handles node failures gracefully by just hopping to the next one.

--
Nothing is ever easy

Catalyst users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.