Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ModPerl: Dev

Re: Apache::DBI

 

 

ModPerl dev RSS feed   Index | Next | Previous | View Threaded


adam.prime at utoronto

Jul 21, 2009, 12:01 PM

Post #1 of 6 (3308 views)
Permalink
Re: Apache::DBI

Continuing discussion from off list.

Philip M. Gollucci wrote:
> Fred Moyer wrote:
>> On 7/21/09 6:58 AM, Adam Prime wrote:
>>> If it get a new release ready, can / will one of you guys release it?
>>> You both have co-maint on cpan.
>>
>> I'd like to see some patches hit the list with test results first. I
>> saw the bug reports come in but haven't had a chance to review them.
>
> Not sure how much time you want to spend, but I overhauled AuthDBI on
> the branch 100% its not quite done yet though, but it should actually be
> testable with Apache-Test now.

What I wanted to do was fix this issue:

http://rt.cpan.org/Public/Bug/Display.html?id=36346

either by checking the various server starting variables for each of the
apis, or simply throwing an eval around the push_handler.

and incorporate this patch from the list:

http://marc.info/?l=apache-modperl&m=124810297118194&w=2

with a bit more error checking.

There are a number of other issues in RT that could be looked at too.

If AuthDBI has been rewritten in a branch, i would propose that it stay
there for the moment, and we get the new version of Apache::DBI out,
with the existing version of AuthDBI, then merge and release that later.

In either case I'd probably want to push a dev version to CPAN, and get
feedback from the list before pushing a full release to CPAN. I'm not
exactly familiar with voting and release managing and all that stuff,
and to be honest, I didn't even realize that Apache::DBI was in the
mod_perl svn repo.

Adam

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl


adam.prime at utoronto

Jul 21, 2009, 8:39 PM

Post #2 of 6 (3104 views)
Permalink
Re: Apache::DBI [In reply to]

Adam Prime wrote:
> What I wanted to do was fix this issue:
>
> http://rt.cpan.org/Public/Bug/Display.html?id=36346
>
> either by checking the various server starting variables for each of the
> apis, or simply throwing an eval around the push_handler.

I've been looking into this bug a bit tonight, and it seems to me that
the problem is maybe with restart_count.

This code:

if (Apache2::ServerUtil::restart_count() == 1) {
debug(2, "$prefix skipping connection during server startup, read
the docu !!");
return $drh->connect(@args);
}

appears earlier in the connect function, and based on the comments, it
would appear that any connection to the database that happens during
startup (which to me should mean any time before the end of the
ChildInitPhase) should not be cached. The problem is PostConfig is run
twice on startup, once with restart_count == 1, and once with
restart_count == 2. So the second time through, it gets past that if
statement, and blows up when it hits the code that tries to push_handler().

The documentation here is also no longer true:

http://perl.apache.org/docs/2.0/api/Apache2/ServerUtil.html#C_restart_count_

If I put that code into my startup.pl, i get:

cnt: 1
cnt: 2
cnt: 3
cnt: 4
cnt: 5
cnt: 6
cnt: 7

I'm going to fool around with things a bit more.

Adam



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl


fred at redhotpenguin

Jul 22, 2009, 1:42 AM

Post #3 of 6 (3121 views)
Permalink
Re: Apache::DBI [In reply to]

On Tue, Jul 21, 2009 at 8:39 PM, Adam Prime<adam.prime [at] utoronto> wrote:
> Adam Prime wrote:
>> What I wanted to do was fix this issue:
>>
>> http://rt.cpan.org/Public/Bug/Display.html?id=36346
>>
>> either by checking the various server starting variables for each of the
>> apis, or simply throwing an eval around the push_handler.
>
> I've been looking into this bug a bit tonight, and it seems to me that
> the problem is maybe with restart_count.
>
> This code:
>
> if (Apache2::ServerUtil::restart_count() == 1) {
>    debug(2, "$prefix skipping connection during server startup, read
> the docu !!");
>    return $drh->connect(@args);
> }
>
> appears earlier in the connect function, and based on the comments, it
> would appear that any connection to the database that happens during
> startup (which to me should mean any time before the end of the
> ChildInitPhase) should not be cached. The problem is PostConfig is run
> twice on startup, once with restart_count == 1, and once with
> restart_count == 2.  So the second time through, it gets past that if
> statement, and blows up when it hits the code that tries to push_handler().
>
> The documentation here is also no longer true:
>
> http://perl.apache.org/docs/2.0/api/Apache2/ServerUtil.html#C_restart_count_
>
> If I put that code into my startup.pl, i get:
>
> cnt: 1
> cnt: 2
> cnt: 3
> cnt: 4
> cnt: 5
> cnt: 6
> cnt: 7
>
> I'm going to fool around with things a bit more.

Looks like we have a unit test for part of restart_count:


phred [at] harpu ~/dev/svn/modperl/mod_perl-2.0 $ ack restart_count t/
t/conf/post_config_startup.pl
110: # my $cnt = Apache2::ServerUtil::restart_count();

t/htdocs/vhost/post_config.pl
6:$TestVhost::config::restart_count = Apache2::ServerUtil::restart_count();

t/response/TestAPI/server_util.pm
59: ok t_cmp Apache2::ServerUtil::restart_count, 2, "restart count";

t/response/TestVhost/config.pm
21:our $restart_count;
37: ok t_cmp($restart_count, 2, "PerlPostConfigRequire");

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl


fred at redhotpenguin

Jul 22, 2009, 1:52 AM

Post #4 of 6 (3106 views)
Permalink
Re: Apache::DBI [In reply to]

On Tue, Jul 21, 2009 at 8:39 PM, Adam Prime<adam.prime [at] utoronto> wrote:
> Adam Prime wrote:
>> What I wanted to do was fix this issue:
>>
>> http://rt.cpan.org/Public/Bug/Display.html?id=36346

I was just looking over your original code where you connect() in
startup.pl. Won't that cause issues since the database handle will be
forked also? Here's what I have in my startup.pl:

use Apache::DBI ();
use Some::Database::Modules ();
use DBI ();
use DBD::Pg ();
DBI->install_driver('Pg');

$Apache::DBI::DEBUG = $config->sl_db_debug;
my $db_connect_params = SL::Model->connect_params;
Apache::DBI->connect_on_init( @{$db_connect_params} );
Apache::DBI->setPingTimeOut( $db_connect_params->[0],
$config->sl_db_ping_timeout );

# delete this line and I will beat you with a stick
# we need to disconnect before the fork
SL::Model->connect->disconnect;
$DBI::connect_via = 'Apache::DBI::connect';

print STDOUT "Startup.pl finished...\n";


I had to leave myself a threatening message in my code as a warning
not to have an open database handle in startup.pl. I'd been told by
Perrin and others why it was necessary, but it wasn't until I screwed
up my own code and experienced the full wrath of a forked httpd child
with a shared db handle cause database transactions to fail at random.

I


>>
>> either by checking the various server starting variables for each of the
>> apis, or simply throwing an eval around the push_handler.
>
> I've been looking into this bug a bit tonight, and it seems to me that
> the problem is maybe with restart_count.
>
> This code:
>
> if (Apache2::ServerUtil::restart_count() == 1) {
>    debug(2, "$prefix skipping connection during server startup, read
> the docu !!");
>    return $drh->connect(@args);
> }
>
> appears earlier in the connect function, and based on the comments, it
> would appear that any connection to the database that happens during
> startup (which to me should mean any time before the end of the
> ChildInitPhase) should not be cached. The problem is PostConfig is run
> twice on startup, once with restart_count == 1, and once with
> restart_count == 2.  So the second time through, it gets past that if
> statement, and blows up when it hits the code that tries to push_handler().
>
> The documentation here is also no longer true:
>
> http://perl.apache.org/docs/2.0/api/Apache2/ServerUtil.html#C_restart_count_
>
> If I put that code into my startup.pl, i get:
>
> cnt: 1
> cnt: 2
> cnt: 3
> cnt: 4
> cnt: 5
> cnt: 6
> cnt: 7
>
> I'm going to fool around with things a bit more.
>
> Adam
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe [at] perl
> For additional commands, e-mail: dev-help [at] perl
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl


adam.prime at utoronto

Jul 22, 2009, 5:41 AM

Post #5 of 6 (3101 views)
Permalink
Re: Apache::DBI [In reply to]

Fred Moyer wrote:
> On Tue, Jul 21, 2009 at 8:39 PM, Adam Prime<adam.prime [at] utoronto> wrote:
>> Adam Prime wrote:
>>> What I wanted to do was fix this issue:
>>>
>>> http://rt.cpan.org/Public/Bug/Display.html?id=36346
>
> I was just looking over your original code where you connect() in
> startup.pl. Won't that cause issues since the database handle will be
> forked also? Here's what I have in my startup.pl:

The code in the ticket was just to illustrate the failure. I ran into
this problem because I wanted to preload a large read-only datastructure
out of the database prior to the fork to get it completely shared
between all the children. The handle itself is then disconnected and
discarded.

Adam



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl


adam.prime at utoronto

Jul 26, 2009, 9:06 PM

Post #6 of 6 (3079 views)
Permalink
Re: Apache::DBI [In reply to]

Adam Prime wrote:
> Fred Moyer wrote:
>> On Tue, Jul 21, 2009 at 8:39 PM, Adam Prime<adam.prime [at] utoronto> wrote:
>>> Adam Prime wrote:
>>>> What I wanted to do was fix this issue:
>>>>
>>>> http://rt.cpan.org/Public/Bug/Display.html?id=36346
>> I was just looking over your original code where you connect() in
>> startup.pl. Won't that cause issues since the database handle will be
>> forked also? Here's what I have in my startup.pl:
>
> The code in the ticket was just to illustrate the failure. I ran into
> this problem because I wanted to preload a large read-only datastructure
> out of the database prior to the fork to get it completely shared
> between all the children. The handle itself is then disconnected and
> discarded.
>
> Adam
>

The patch attached ads code to bail out of connect() calls called prior
to the fork by setting a package global during the ChildInit phase,
along with the other patch which was submitted to the users mailing list.

It works fine for me in extremely limited testing.

Thoughts?

Adam
Attachments: Apache-DBI.patch (2.65 KB)

ModPerl dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.