Mailing List Archive: Request Tracker: Devel

webmux connecting to the database; causing mod_perl startup to fail

Index | Next | Previous | View Threaded

dominic.hargreaves at oucs

Sep 11, 2010, 6:45 AM

Post #1 of 4 (729 views)
Permalink

webmux connecting to the database; causing mod_perl startup to fail

Hello,

I'm investigating Debian bug report
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595054>
concerning the complete failure of startup of Apache when webmux.pl
is being PerlRequire'd and the database server is unavailable.

I've been able to reproduce this myself with both MySQL and PostgreSQL.

In this particular instance, the reason the database server is not
running is because the user is using parallel bootup script, but the
same would apply if, for example, the database server was on a separate
system from the web server. However it's worth noting that parallel
booting will be the default for the next version of Debian so it's going
to become a common problem.

From a look at the code in webmux.pl (3.8.8) I don't see a simple
solution to this problem; the database connection is established by
RT::Init which is needed before launching into the Mason handler.

The parts of the Init call which use the database appear to be:

- InitSystemObjects()
- InitPlugins(); (possibly plugins talk to the database in init...)

The explicit use of the database handle is to call CheckCompatibility.
This looks like a "nice to have" function rather than an essential -
it is after all also called by rt-setup-database.

Maybe it would be possible to split the Init into two stages and only
do the database connection in the second stage, which could be
run by the Mason handler the first time a request comes in.

I couldn't find an open bug about this, which is a bit surprising
because it seems relatively likely to happen in all sort of scenarios.

Do people agree that this is a problem that should be fixed? Any obvious
(minimally-invasive, given that this is something that should be fixed
in a frozen Debian release) methods?

Cheers,
Dominic.

--
Dominic Hargreaves, Systems Development and Support Team
Computing Services, University of Oxford

Attachments:

signature.asc (0.18 KB)

jesse at bestpractical

Sep 11, 2010, 8:38 AM

Post #2 of 4 (703 views)
Permalink

Re: webmux connecting to the database; causing mod_perl startup to fail [In reply to]

> Maybe it would be possible to split the Init into two stages and only
> do the database connection in the second stage, which could be
> run by the Mason handler the first time a request comes in.
>

That's going to become impossible as we continue to move RT's configuration into the database. I consider having your database around before application startup to be a fairly serious requirement. Parallel startup scripts don't have a way to declare dependencies? How do you deal with, say, starting up apps that are mounted on NFS filesystems?
_______________________________________________
List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel

dominic.hargreaves at oucs

Sep 11, 2010, 9:10 AM

Post #3 of 4 (701 views)
Permalink

Re: webmux connecting to the database; causing mod_perl startup to fail [In reply to]

On Sat, Sep 11, 2010 at 11:38:32AM -0400, Jesse Vincent wrote:
> That's going to become impossible as we continue to move RT's configuration into the database. I consider having your database around before application startup to be a fairly serious requirement. Parallel startup scripts don't have a way to declare dependencies? How do you deal with, say, starting up apps that are mounted on NFS filesystems?

They do, but the problem is that I'm not in a position to mandate
the dependencies declared by Apache, and it's not just RT that fails
to start up but Apache in total, which may be used in other ways on the
same system.

There are network filesystem dependencies already included in the
parallel startup stuff, I believe. But as pointed out already, there
are some requirements you can't satisfy as easily: lack of a database
server on a remote host being the obvious one, so even if there was
sufficient information in the dependency information of the init
scripts, that doesn't solve the underlying problem.

I should have pointed out the short discussion on debian-devel on this
matter, which was fairly strongly of the opinion that RT shouldn't
break Apache in this way, and was also fairly negative about trying
to persuade the Apache maintainers to add some dependencies so Apache
will always start before the database servers:

http://lists.debian.org/debian-devel/2010/08/msg00786.html
and continuing:
http://lists.debian.org/debian-devel/2010/09/msg00011.html

If you think it's impossible to fix this in RT, then probably the
only thing we can do is deprecate support for mod_perl, and suggest
that people use FastCGI instead; because FastCGI doesn't turn fatal
errors in the application into a fatal error to start the web server
(certainly the Apache modules both seem to have a retry-with-backoff
approach to the problem). Or maybe there's a mod_perl trick to do a
similar thing which I'm not aware of.

Cheers,
Dominic.

--
Dominic Hargreaves, Systems Development and Support Team
Computing Services, University of Oxford

Attachments:

signature.asc (0.18 KB)

jesse at bestpractical

Sep 11, 2010, 9:42 AM

Post #4 of 4 (707 views)
Permalink

Re: webmux connecting to the database; causing mod_perl startup to fail [In reply to]

I suspect there's a way to make RT under modperl fail more gracefully. I'd support such a change.

"Dominic Hargreaves" <dominic.hargreaves [at] oucs> wrote:

>On Sat, Sep 11, 2010 at 11:38:32AM -0400, Jesse Vincent wrote:
>> That's going to become impossible as we continue to move RT's configuration into the database. I consider having your database around before application startup to be a fairly serious requirement. Parallel startup scripts don't have a way to declare dependencies? How do you deal with, say, starting up apps that are mounted on NFS filesystems?
>
>They do, but the problem is that I'm not in a position to mandate
>the dependencies declared by Apache, and it's not just RT that fails
>to start up but Apache in total, which may be used in other ways on the
>same system.
>
>There are network filesystem dependencies already included in the
>parallel startup stuff, I believe. But as pointed out already, there
>are some requirements you can't satisfy as easily: lack of a database
>server on a remote host being the obvious one, so even if there was
>sufficient information in the dependency information of the init
>scripts, that doesn't solve the underlying problem.
>
>I should have pointed out the short discussion on debian-devel on this
>matter, which was fairly strongly of the opinion that RT shouldn't
>break Apache in this way, and was also fairly negative about trying
>to persuade the Apache maintainers to add some dependencies so Apache
>will always start before the database servers:
>
>http://lists.debian.org/debian-devel/2010/08/msg00786.html
>and continuing:
>http://lists.debian.org/debian-devel/2010/09/msg00011.html
>
>If you think it's impossible to fix this in RT, then probably the
>only thing we can do is deprecate support for mod_perl, and suggest
>that people use FastCGI instead; because FastCGI doesn't turn fatal
>errors in the application into a fatal error to start the web server
>(certainly the Apache modules both seem to have a retry-with-backoff
>approach to the problem). Or maybe there's a mod_perl trick to do a
>similar thing which I'm not aware of.
>
>Cheers,
>Dominic.
>
>--
>Dominic Hargreaves, Systems Development and Support Team
>Computing Services, University of Oxford

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
_______________________________________________
List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads