Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ModPerl: ModPerl

Apache2::SubProcess subprocess not subprocessing properly

 

 

ModPerl modperl RSS feed   Index | Next | Previous | View Threaded


tosh at 1200group

Feb 9, 2010, 5:21 AM

Post #1 of 6 (895 views)
Permalink
Apache2::SubProcess subprocess not subprocessing properly

Hi after much trial and all error I am seeing that the browser
connection closing is also stopping my subprocess.

The main ModPerl::Registry program looks like this:
### file.pl ###
use Apache2::SubProcess ();
use JSON();

&main(shift);

sub main {
my $r = shift;
...
$r->spawn_proc_prog ('/web/html/file_fork.pl', \@argv);
return print $cgi->redirect('index.pl');
}
###

And

### file_fork.pl ###
use JSON();
use warnings;
use POSIX 'setsid';
chdir '/' or die "Can't chdir to /: $!";
open STDIN, '/dev/null' or die "Can't read /dev/null: $!";
open STDOUT, '+>>', '/tmp/debug.txt' or die "Can't write to
/tmp/debug.txt: $!";
open STDERR, '>&STDOUT' or die "Can't dup stdout: $!";
setsid or die "Can't start a new session: $!";

# run your code here or call exec to another program
foreach my $num (1..10) {
warn $num;
sleep(1);
}
###

When file.pl is executed it calls file_fork.pl and file_fork starts to
print 1, 2, 3 and then stops around 2 or 3 depending on how long my
browser is connected.

Are there any glaring errors that anyone sees? It should work shouldn't
it, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 after 10 seconds...

Thank-you all!

Tosh


--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/


torsten.foertsch at gmx

Feb 10, 2010, 6:07 AM

Post #2 of 6 (830 views)
Permalink
Re: Apache2::SubProcess subprocess not subprocessing properly [In reply to]

On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
> use POSIX 'setsid';
if( fork ) { POSIX::_exit 0; CORE::exit 0 }
> chdir '/' or die "Can't chdir to /: $!";
>
simplest is to fork() another time to break the parent-child relationship.

Keep in mind that you make your service vulnerable if you simply fork off long
running processes.

What prevents a user from testing it:

ab -n 100000 -c 100 http://...

This will quite fast create a *lot* of processes.

I tend to use some kind of queue for such processing. But, of course, it
depends.

Torsten


tosh at 1200group

Feb 11, 2010, 3:07 AM

Post #3 of 6 (822 views)
Permalink
Re: Apache2::SubProcess subprocess not subprocessing properly [In reply to]

Actually my solution to this problem was pretty rational. I just turned
mod_perl off for this particular program.

<Files ~ "(?<!myfork)\.pl$">
PerlHandler ModPerl::Registry
</Files>

Will I a have possible performance problems? Sure, but nothing a few
more $25/month servers can't fix, and it saves me all the aggravation of
the past months trying to get this to work.

Problem solved!

Tosh


Torsten Förtsch wrote:
> On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
>> use POSIX 'setsid';
> if( fork ) { POSIX::_exit 0; CORE::exit 0 }
>> chdir '/' or die "Can't chdir to /: $!";
>>
> simplest is to fork() another time to break the parent-child relationship.
>
> Keep in mind that you make your service vulnerable if you simply fork off long
> running processes.
>
> What prevents a user from testing it:
>
> ab -n 100000 -c 100 http://...
>
> This will quite fast create a *lot* of processes.
>
> I tend to use some kind of queue for such processing. But, of course, it
> depends.
>
> Torsten
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/


eric.berg at barclayscapital

Feb 16, 2010, 9:00 AM

Post #4 of 6 (800 views)
Permalink
RE: Apache2::SubProcess subprocess not subprocessing properly [In reply to]

We are configured to have mod_perl serve the .pl scripts and leave the .cgi scripts to be served as regular old CGI, just as a point of interest.

Eric

> -----Original Message-----
> From: Tosh Cooey [mailto:tosh [at] 1200group]
> Sent: Thursday, February 11, 2010 6:08 AM
> To: Torsten Förtsch
> Cc: modperl [at] perl
> Subject: Re: Apache2::SubProcess subprocess not subprocessing properly
>
> Actually my solution to this problem was pretty rational. I
> just turned
> mod_perl off for this particular program.
>
> <Files ~ "(?<!myfork)\.pl$">
> PerlHandler ModPerl::Registry
> </Files>
>
> Will I a have possible performance problems? Sure, but nothing a few
> more $25/month servers can't fix, and it saves me all the
> aggravation of
> the past months trying to get this to work.
>
> Problem solved!
>
> Tosh
>
>
> Torsten Förtsch wrote:
> > On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
> >> use POSIX 'setsid';
> > if( fork ) { POSIX::_exit 0; CORE::exit 0 }
> >> chdir '/' or die "Can't chdir to /: $!";
> >>
> > simplest is to fork() another time to break the
> parent-child relationship.
> >
> > Keep in mind that you make your service vulnerable if you
> simply fork off long
> > running processes.
> >
> > What prevents a user from testing it:
> >
> > ab -n 100000 -c 100 http://...
> >
> > This will quite fast create a *lot* of processes.
> >
> > I tend to use some kind of queue for such processing. But,
> of course, it
> > depends.
> >
> > Torsten
> >
>
> --
> McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
>
_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________


torsten.foertsch at gmx

Feb 20, 2010, 6:19 AM

Post #5 of 6 (764 views)
Permalink
Re: Apache2::SubProcess subprocess not subprocessing properly [In reply to]

Tosh,

On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
> Hi after much trial and all error I am seeing that the browser
> connection closing is also stopping my subprocess.
>

I don't know what you are trying to achieve and I don't use
Apache2::Subprocess much. What I do to fork off processes instead is this:

my $close_fd=sub {
use POSIX ();
my %save=(2=>1); # keep STDERR
undef @save{@_} if( @_ );
opendir my $d, "/proc/self/fd" or do {
warn "Cannot open directory /proc/self/fd: $!\n";
POSIX::_exit -1; # try this first to avoid buffer flushing
CORE::exit -1;
};

while (defined(my $fd=readdir $d)) {
next unless $fd=~/^\d+$/;
POSIX::close $fd unless exists $save{$fd};
}
};

my $spawn=sub {
use POSIX ();
my ($daemon_should_survive_apache_restart, @args)=@_;

local $SIG{CHLD}='IGNORE';
my $pid;
# yes, even fork can fail
select undef, undef, undef, .1 while( !defined($pid=fork) );
unless( $pid ) { # child
# 2nd fork to cut parent relationship with a mod_perl apache
select undef, undef, undef, .1 while( !defined($pid=fork) );
if( $pid ) {
POSIX::_exit 0;
CORE::exit 0;
} else {
if( ref($daemon_should_survive_apache_restart) ) {
$close_fd->($daemon_should_survive_apache_restart->{fd});
POSIX::setsid if( $daemon_should_survive_apache_restart->{survive} );
} else {
$close_fd->();
POSIX::setsid if( $daemon_should_survive_apache_restart );
}

if( 'CODE' eq ref $args[0] ) {
my $f=shift @args;
# TODO: restore %ENV and exit() behavior
eval {$f->(@args)};
CORE::exit 0;
} else {
{exec @args;} # extra block to suppress a warning
POSIX::_exit -1;
CORE::exit -1;
}
}
}
waitpid $pid, 0; # avoid a zombie on some OS
};

And this is how to use it:

$spawn->({survive=>1}, sub {
my ($n, $sleep)=@_;
for(1..$n) {
warn "sleeping $sleep\n";
sleep $sleep;
}
warn "Done\n";
}, 10, 2 );

or

$spawn->
(undef, # don't survive apache stop
qw/bash -c/,'for (( i=0; i<10; i++ )); do echo "opi" >&2; sleep 2; done')

The first argument to $spawn is an options hash with 2 possible options:

survive=>BOOLEAN # should the subprocess survive an apache stop or not
fd=>\@list_of_open_file_descriptors_to_preserve

Normally the $spawn will close all files save for STDERR. With the fd option
one can preserve other files such as database connections or other.

With the survive option true the child process will be the leader of a new
process group.

NOTES:

- The $close_fd function is Linux-specific. I couldn't think of a really
portable (at least across UNICES) way to get all open file descriptors of the
current process.

- The waitpid at the end of $spawn is not necessary under Linux since
$SIG{CHLD}='IGNORE' already takes care of zombies. But other systems need it.

- Keep in mind that %ENV and exit() work different under mod_perl. If you pass
a subroutine this behavior remains.

Torsten


tosh at 1200group

Feb 20, 2010, 7:01 AM

Post #6 of 6 (760 views)
Permalink
Re: Apache2::SubProcess subprocess not subprocessing properly [In reply to]

You aren't going to like this, and maybe what you're doing is for a
really complex situation, but it really shouldn't be so complex to just
spawn off a long running process that you don't care to ever hear back
from, honestly.

Tosh


Torsten Förtsch wrote:
> Tosh,
>
> On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
>> Hi after much trial and all error I am seeing that the browser
>> connection closing is also stopping my subprocess.
>>
>
> I don't know what you are trying to achieve and I don't use
> Apache2::Subprocess much. What I do to fork off processes instead is this:
>
> my $close_fd=sub {
> use POSIX ();
> my %save=(2=>1); # keep STDERR
> undef @save{@_} if( @_ );
> opendir my $d, "/proc/self/fd" or do {
> warn "Cannot open directory /proc/self/fd: $!\n";
> POSIX::_exit -1; # try this first to avoid buffer flushing
> CORE::exit -1;
> };
>
> while (defined(my $fd=readdir $d)) {
> next unless $fd=~/^\d+$/;
> POSIX::close $fd unless exists $save{$fd};
> }
> };
>
> my $spawn=sub {
> use POSIX ();
> my ($daemon_should_survive_apache_restart, @args)=@_;
>
> local $SIG{CHLD}='IGNORE';
> my $pid;
> # yes, even fork can fail
> select undef, undef, undef, .1 while( !defined($pid=fork) );
> unless( $pid ) { # child
> # 2nd fork to cut parent relationship with a mod_perl apache
> select undef, undef, undef, .1 while( !defined($pid=fork) );
> if( $pid ) {
> POSIX::_exit 0;
> CORE::exit 0;
> } else {
> if( ref($daemon_should_survive_apache_restart) ) {
> $close_fd->($daemon_should_survive_apache_restart->{fd});
> POSIX::setsid if( $daemon_should_survive_apache_restart->{survive} );
> } else {
> $close_fd->();
> POSIX::setsid if( $daemon_should_survive_apache_restart );
> }
>
> if( 'CODE' eq ref $args[0] ) {
> my $f=shift @args;
> # TODO: restore %ENV and exit() behavior
> eval {$f->(@args)};
> CORE::exit 0;
> } else {
> {exec @args;} # extra block to suppress a warning
> POSIX::_exit -1;
> CORE::exit -1;
> }
> }
> }
> waitpid $pid, 0; # avoid a zombie on some OS
> };
>
> And this is how to use it:
>
> $spawn->({survive=>1}, sub {
> my ($n, $sleep)=@_;
> for(1..$n) {
> warn "sleeping $sleep\n";
> sleep $sleep;
> }
> warn "Done\n";
> }, 10, 2 );
>
> or
>
> $spawn->
> (undef, # don't survive apache stop
> qw/bash -c/,'for (( i=0; i<10; i++ )); do echo "opi" >&2; sleep 2; done')
>
> The first argument to $spawn is an options hash with 2 possible options:
>
> survive=>BOOLEAN # should the subprocess survive an apache stop or not
> fd=>\@list_of_open_file_descriptors_to_preserve
>
> Normally the $spawn will close all files save for STDERR. With the fd option
> one can preserve other files such as database connections or other.
>
> With the survive option true the child process will be the leader of a new
> process group.
>
> NOTES:
>
> - The $close_fd function is Linux-specific. I couldn't think of a really
> portable (at least across UNICES) way to get all open file descriptors of the
> current process.
>
> - The waitpid at the end of $spawn is not necessary under Linux since
> $SIG{CHLD}='IGNORE' already takes care of zombies. But other systems need it.
>
> - Keep in mind that %ENV and exit() work different under mod_perl. If you pass
> a subroutine this behavior remains.
>
> Torsten
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

ModPerl modperl RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.