Mailing List Archive: ModPerl: Dev

[mp2] mod_perl closes apache's stdin and/or stdout

Index | Next | Previous | View Threaded

dmitry at karasik

Oct 1, 2007, 7:49 AM

Post #1 of 19 (4687 views)
Permalink

[mp2] mod_perl closes apache's stdin and/or stdout

Hello,

As a follow-up to my previous bug report to users@,
( http://www.gossamer-threads.com/lists/modperl/modperl/94912 )
I'd like to re-post the same issue to dev@, because I'm unsure
what is the status of this bugreport, and basically if it
was recognized as one.

Just for the sake if the test wasn't reproduced correctly, I'll rerun:

1) test setup:

add to httpd.conf:

<LocationMatch "^/cgi-bin/m.cgi$">
SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
</LocationMatch>

<LocationMatch "^/cgi-bin/a.cgi$">
SetHandler cgi-script
</LocationMatch>

both cgis are identical, and are:

#!/usr/bin/perl -w
print "Content-type: text/html\n\n", $ENV{MOD_PERL}||'', "\n", <STDIN>, "\n";

test.pl:

open F, "|nc localhost 80"; # this requires netcat because I'm lazy
print F <<R;
POST /cgi-bin/m.cgi HTTP/1.1
Host: localhost
Content-Type: application/x-www-form-urlencoded
Content-Length: 13
Keep-Alive: 300

submit=Submit

POST /cgi-bin/a.cgi HTTP/1.1
Host: localhost
Content-Type: application/x-www-form-urlencoded
Content-Length: 13
Keep-Alive: 0
Connection: close

submit=Submit
R

2)

- now, stop apache, run it from console as 'httpd -X' ( or apache2 -X on some linux distros)
- run 'perl test.pl'

The bug manifests itself in that the following apache response is printed out (some lines are skipped):

HTTP/1.1 200 OK
1d
mod_perl/2.0.3
submit=Submit
0

HTTP/1.1 200 OK
Connection: close
Content-Type: text/html
2
0

and the second request doesn't contain 'submit=Submit' line.

This can be fixed by applying the following patch:

--- modperl_io.c.orig 2007-09-25 15:36:02.000000000 +0200
+++ modperl_io.c 2007-09-25 15:35:51.000000000 +0200
@@ -129,6 +129,15 @@
Perl_croak(aTHX_ "Failed to dup STDIN: %" SVf, get_sv("!", TRUE));
}

+ /* In mixed environment of mod_perl and cgi scripts, cgi scripts may read content of
+ * POST requests off STDIN. do_close() calls actual close(0), freeing the descriptor 0
+ * for reuse, and creating havoc for anyone reading from file descriptor 0.
+ * This hack changes the IO type to IoTYPE_STD, because do_close() does not call
+ * underlying close() on IO handles of these types, but does free the associated
+ * resources. */
+ if ( IoIFP(io) && PerlIO_fileno(IoIFP(io)) == 0)
+ IoTYPE(io) = IoTYPE_STD;
+
/* similar to PerlIO::scalar, the PerlIO::Apache layer doesn't
* have file descriptors, so STDIN must be closed before it can
* be reopened */

So, I'm not sure what other information I can provide, please tell me if there's anything
more, because I'd love to see that bug fixed.

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

fred at taperfriendlymusic

Oct 1, 2007, 9:53 AM

Post #2 of 19 (4524 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Dmitry Karasik wrote:
> Hello,

Hi Dmitry,

> As a follow-up to my previous bug report to users@,
> ( http://www.gossamer-threads.com/lists/modperl/modperl/94912 )
> I'd like to re-post the same issue to dev@, because I'm unsure
> what is the status of this bugreport, and basically if it
> was recognized as one.

Can you try the updated test patch that Torsten replied to the list with:

http://marc.info/?l=apache-modperl&m=119074979901161&w=2

you can run it via:

cd ModPerl-Registry
perl Makefile.PL && make && make test

once 'make test' has been run once, you can run that test individually with:

./t/TEST t/cgi-bin/stdin.t

It's really important to show a failing regression test here to prove
that it fails on your setup, but passes on other users' setups. I
looked through the thread and it looks like you are using a perlio
enabled perl, so that's probably not the problem.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

torsten.foertsch at gmx

Oct 1, 2007, 10:12 AM

Post #3 of 19 (4522 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

On Monday 01 October 2007 18:53, Fred Moyer wrote:
> It's really important to show a failing regression test here to prove
> that it fails on your setup, but passes on other users' setups. šI
> looked through the thread and it looks like you are using a perlio
> enabled perl, so that's probably not the problem.

Dimitry said the problem occurs only for the very first request after Apache
has been started. I suspect the first request a newly born Apache child
handles is affected.

I think a test for that situation is difficult to write.

Torsten

dmitry at karasik

Oct 1, 2007, 10:33 AM

Post #4 of 19 (4514 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

> Dimitry said the problem occurs only for the very first request after Apache
> has been started. I suspect the first request a newly born Apache child
> handles is affected.

Not only on the first request, that effect also happens on subsequent requests,
but the very first request is also the one that gives an always reproducible
case.

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

dmitry at karasik

Oct 1, 2007, 11:54 PM

Post #5 of 19 (4522 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

> Can you try the updated test patch that Torsten replied to the list with:
>
> http://marc.info/?l=apache-modperl&m=119074979901161&w=2

The test succeeds in boths cases, on both patched and unpatched mod_perl.
I don't know why is that, but I think that shouldn't be a reason to ignore
the error that is reproduced using the test case I proposed (and
of course because of the Pesticide Paradox :)

/dk

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

torsten.foertsch at gmx

Oct 2, 2007, 11:16 AM

Post #6 of 19 (4496 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

On Tuesday 02 October 2007 08:54, Dmitry Karasik wrote:
> > Can you try the updated test patch that Torsten replied to the list with:
> >
> > http://marc.info/?l=apache-modperl&m=119074979901161&w=2
>
> The test succeeds in boths cases, on both patched and unpatched mod_perl.
> I don't know why is that, but I think that shouldn't be a reason to ignore
> the error that is reproduced using the test case I proposed (and
> of course because of the Pesticide Paradox :)

Try the following (assuming you are using prefork):

0) cd ModPerl-Registry/
1) start the testsuite with t/TEST -start-httpd
2) kill all apache children (workers). The supervisor apache should respawn
new children.
3) Then run your test: t/TEST -run-tests -verbose t/stdin.t

I hope it fails this way. If yes then maybe a way to construct a proper test
is to add a new vhost with very limited MaxRequestsPerChild (1 or 2)

Torsten

dmitry at karasik

Oct 3, 2007, 12:07 AM

Post #7 of 19 (4499 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

>
> 0) cd ModPerl-Registry/
> 1) start the testsuite with t/TEST -start-httpd
> 2) kill all apache children (workers). The supervisor apache should respawn
> new children.
> 3) Then run your test: t/TEST -run-tests -verbose t/stdin.t

No, it doesn't, it passes. I tried both with and without 2), same results,
both succeed.

> I hope it fails this way. If yes then maybe a way to construct a proper test
> is to add a new vhost with very limited MaxRequestsPerChild (1 or 2)

The key here is to make apache to fork a cgi process from the process (or its
parent) that has called modperl_response_handler_cgi() before.

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

geoff at modperlcookbook

Oct 3, 2007, 5:07 AM

Post #8 of 19 (4515 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Dmitry Karasik wrote:
>> 0) cd ModPerl-Registry/
>> 1) start the testsuite with t/TEST -start-httpd
>> 2) kill all apache children (workers). The supervisor apache should respawn
>> new children.
>> 3) Then run your test: t/TEST -run-tests -verbose t/stdin.t
>
> No, it doesn't, it passes. I tried both with and without 2), same results,
> both succeed.
>
>> I hope it fails this way. If yes then maybe a way to construct a proper test
>> is to add a new vhost with very limited MaxRequestsPerChild (1 or 2)
>
> The key here is to make apache to fork a cgi process from the process (or its
> parent) that has called modperl_response_handler_cgi() before.

have you tried

$ t/TEST -one-process t/yourtest.t

?

--Geoff

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

dmitry at karasik

Oct 3, 2007, 5:43 AM

Post #9 of 19 (4492 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

>
> have you tried
>
> $ t/TEST -one-process t/yourtest.t

same result, the bug does not show.

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

dmitry at karasik

Oct 8, 2007, 2:09 AM

Post #10 of 19 (4444 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

I found the change that triggers the bug with mod_perl tests:
This line

Apache::TestRequest::user_agent( keep_alive => 1);

added in stdin.t does it. I hope it is convincing enough now. I still don't
understand why it was so hard to try to reproduce my test setup with netcat.

/dk

My log:

$ ./t/TEST -start-httpd -one-process
/usr/local/sbin/httpd -D ONE_PROCESS -d /usr/ports/www/mod_perl2/work/mod_perl-2.0.3/ModPerl-Registry/t -f /usr/ports/www/mod_perl2/work/mod_perl-2.0.3/ModPerl-Registry/t/conf/httpd.conf -D APACHE2
using Apache/2.2.6 (prefork MPM)

waiting 90 seconds for server to start: ...
waiting 90 seconds for server to start: ok (waited 2 secs)
server localhost:8529 started

$ t/TEST -run-tests -verbose t/stdin.t
t/stdin....1..2
# Running under perl version 5.008008 for freebsd
# Current time local: Mon Oct 8 11:05:12 2007
# Current time GMT: Mon Oct 8 09:05:12 2007
# Using Test.pm version 1.25
# Using Apache/Test.pm version 1.29
# testing : form submission ok under mod_perl
# expected: (?-xism:submitted)
# received: submitted / mod_perl/2.0.3
ok 1
# testing : stdin reset, form submission ok under cgi
# expected: (?-xism:submitted)
# received:
not ok 2
# Failed test 2 in t/stdin.t at line 25
FAILED test 2
Failed 1/2 tests, 50.00% okay
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
t/stdin.t 2 1 2
Failed 1/1 test scripts. 1/2 subtests failed.
Files=1, Tests=2, 2 wallclock secs ( 0.30 cusr + 0.05 csys = 0.36 CPU)
Failed 1/1 test programs. 1/2 subtests failed.
[ error] error running tests (please examine t/logs/error_log)
[1] 86918 exit 1 t/TEST -run-tests -verbose t/stdin.t

On Tue, Oct 02, 2007 at 08:16:20PM +0200, Torsten Foertsch wrote:
> On Tuesday 02 October 2007 08:54, Dmitry Karasik wrote:
> > > Can you try the updated test patch that Torsten replied to the list with:
> > >
> > > http://marc.info/?l=apache-modperl&m=119074979901161&w=2
> >
> > The test succeeds in boths cases, on both patched and unpatched mod_perl.
> > I don't know why is that, but I think that shouldn't be a reason to ignore
> > the error that is reproduced using the test case I proposed (and
> > of course because of the Pesticide Paradox :)
>
> Try the following (assuming you are using prefork):
>
> 0) cd ModPerl-Registry/
> 1) start the testsuite with t/TEST -start-httpd
> 2) kill all apache children (workers). The supervisor apache should respawn
> new children.
> 3) Then run your test: t/TEST -run-tests -verbose t/stdin.t
>
> I hope it fails this way. If yes then maybe a way to construct a proper test
> is to add a new vhost with very limited MaxRequestsPerChild (1 or 2)
>
> Torsten

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

fred at taperfriendlymusic

Oct 8, 2007, 9:04 PM

Post #11 of 19 (4466 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Dmitry Karasik wrote:
> I found the change that triggers the bug with mod_perl tests:
> This line
>
> Apache::TestRequest::user_agent( keep_alive => 1);
>
> added in stdin.t does it. I hope it is convincing enough now. I still don't
> understand why it was so hard to try to reproduce my test setup with netcat.

Does this test pass for you when you apply the patch in this thread?

http://marc.info/?l=apache-modperl&m=119072925228529&w=2

> /dk
>
>
>
> My log:
>
> $ ./t/TEST -start-httpd -one-process
> /usr/local/sbin/httpd -D ONE_PROCESS -d /usr/ports/www/mod_perl2/work/mod_perl-2.0.3/ModPerl-Registry/t -f /usr/ports/www/mod_perl2/work/mod_perl-2.0.3/ModPerl-Registry/t/conf/httpd.conf -D APACHE2
> using Apache/2.2.6 (prefork MPM)
>
> waiting 90 seconds for server to start: ...
> waiting 90 seconds for server to start: ok (waited 2 secs)
> server localhost:8529 started
>
> $ t/TEST -run-tests -verbose t/stdin.t
> t/stdin....1..2
> # Running under perl version 5.008008 for freebsd
> # Current time local: Mon Oct 8 11:05:12 2007
> # Current time GMT: Mon Oct 8 09:05:12 2007
> # Using Test.pm version 1.25
> # Using Apache/Test.pm version 1.29
> # testing : form submission ok under mod_perl
> # expected: (?-xism:submitted)
> # received: submitted / mod_perl/2.0.3
> ok 1
> # testing : stdin reset, form submission ok under cgi
> # expected: (?-xism:submitted)
> # received:
> not ok 2
> # Failed test 2 in t/stdin.t at line 25
> FAILED test 2
> Failed 1/2 tests, 50.00% okay
> Failed Test Stat Wstat Total Fail List of Failed
> -------------------------------------------------------------------------------
> t/stdin.t 2 1 2
> Failed 1/1 test scripts. 1/2 subtests failed.
> Files=1, Tests=2, 2 wallclock secs ( 0.30 cusr + 0.05 csys = 0.36 CPU)
> Failed 1/1 test programs. 1/2 subtests failed.
> [ error] error running tests (please examine t/logs/error_log)
> [1] 86918 exit 1 t/TEST -run-tests -verbose t/stdin.t
>
>
>
>
>
> On Tue, Oct 02, 2007 at 08:16:20PM +0200, Torsten Foertsch wrote:
>> On Tuesday 02 October 2007 08:54, Dmitry Karasik wrote:
>>>> Can you try the updated test patch that Torsten replied to the list with:
>>>>
>>>> http://marc.info/?l=apache-modperl&m=119074979901161&w=2
>>> The test succeeds in boths cases, on both patched and unpatched mod_perl.
>>> I don't know why is that, but I think that shouldn't be a reason to ignore
>>> the error that is reproduced using the test case I proposed (and
>>> of course because of the Pesticide Paradox :)
>> Try the following (assuming you are using prefork):
>>
>> 0) cd ModPerl-Registry/
>> 1) start the testsuite with t/TEST -start-httpd
>> 2) kill all apache children (workers). The supervisor apache should respawn
>> new children.
>> 3) Then run your test: t/TEST -run-tests -verbose t/stdin.t
>>
>> I hope it fails this way. If yes then maybe a way to construct a proper test
>> is to add a new vhost with very limited MaxRequestsPerChild (1 or 2)
>>
>> Torsten
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

dmitry at karasik

Oct 8, 2007, 9:58 PM

Post #12 of 19 (4442 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

> Does this test pass for you when you apply the patch in this thread?
> http://marc.info/?l=apache-modperl&m=119072925228529&w=2

You mean my own patch? Yes, it does pass with the patch applied.

>
> >/dk
> >
> >
> >
> >My log:
> >
> >$ ./t/TEST -start-httpd -one-process
> >/usr/local/sbin/httpd -D ONE_PROCESS -d
> >/usr/ports/www/mod_perl2/work/mod_perl-2.0.3/ModPerl-Registry/t -f
> >/usr/ports/www/mod_perl2/work/mod_perl-2.0.3/ModPerl-Registry/t/conf/httpd.conf -D APACHE2
> >using Apache/2.2.6 (prefork MPM)
> >
> >waiting 90 seconds for server to start: ...
> >waiting 90 seconds for server to start: ok (waited 2 secs)
> >server localhost:8529 started
> >
> >$ t/TEST -run-tests -verbose t/stdin.t
> >t/stdin....1..2
> ># Running under perl version 5.008008 for freebsd
> ># Current time local: Mon Oct 8 11:05:12 2007
> ># Current time GMT: Mon Oct 8 09:05:12 2007
> ># Using Test.pm version 1.25
> ># Using Apache/Test.pm version 1.29
> ># testing : form submission ok under mod_perl
> ># expected: (?-xism:submitted)
> ># received: submitted / mod_perl/2.0.3
> >ok 1
> ># testing : stdin reset, form submission ok under cgi
> ># expected: (?-xism:submitted)
> ># received:
> >not ok 2
> ># Failed test 2 in t/stdin.t at line 25
> >FAILED test 2
> > Failed 1/2 tests, 50.00% okay
> >Failed Test Stat Wstat Total Fail List of Failed
> >-------------------------------------------------------------------------------
> >t/stdin.t 2 1 2
> >Failed 1/1 test scripts. 1/2 subtests failed.
> >Files=1, Tests=2, 2 wallclock secs ( 0.30 cusr + 0.05 csys = 0.36 CPU)
> >Failed 1/1 test programs. 1/2 subtests failed.
> >[ error] error running tests (please examine t/logs/error_log)
> >[1] 86918 exit 1 t/TEST -run-tests -verbose t/stdin.t
> >
> >
> >
> >
> >
> >On Tue, Oct 02, 2007 at 08:16:20PM +0200, Torsten Foertsch wrote:
> >>On Tuesday 02 October 2007 08:54, Dmitry Karasik wrote:
> >>>>Can you try the updated test patch that Torsten replied to the list
> >>>>with:
> >>>>
> >>>>http://marc.info/?l=apache-modperl&m=119074979901161&w=2
> >>>The test succeeds in boths cases, on both patched and unpatched mod_perl.
> >>>I don't know why is that, but I think that shouldn't be a reason to
> >>>ignore
> >>>the error that is reproduced using the test case I proposed (and
> >>>of course because of the Pesticide Paradox :)
> >>Try the following (assuming you are using prefork):
> >>
> >>0) cd ModPerl-Registry/
> >>1) start the testsuite with t/TEST -start-httpd
> >>2) kill all apache children (workers). The supervisor apache should
> >>respawn new children.
> >>3) Then run your test: t/TEST -run-tests -verbose t/stdin.t
> >>
> >>I hope it fails this way. If yes then maybe a way to construct a proper
> >>test is to add a new vhost with very limited MaxRequestsPerChild (1 or 2)
> >>
> >>Torsten
> >
> >
> >

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

gozer at ectoplasm

Oct 9, 2007, 12:47 AM

Post #13 of 19 (4453 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Dmitry Karasik wrote:
>> Does this test pass for you when you apply the patch in this thread?
>> http://marc.info/?l=apache-modperl&m=119072925228529&w=2
>
> You mean my own patch? Yes, it does pass with the patch applied.

Here is what I am testing with:

Index: ModPerl-Registry/t/stdin.t
===================================================================
--- ModPerl-Registry/t/stdin.t (revision 0)
+++ ModPerl-Registry/t/stdin.t (revision 0)
@@ -0,0 +1,26 @@
+use strict;
+use warnings FATAL => 'all';
+
+use Apache::Test;
+use Apache::TestUtil;
+use Apache::TestRequest qw(POST);
+
+Apache::TestRequest::user_agent(keep_alive => 1);
+
+plan tests => 2, need [qw(CGI)],
+ need_min_module_version CGI => 3.08;
+
+{
+ my $url = "/stdin_override/mod_perl";
+ my $res = POST $url, [submit=>1];
+
+ ok t_cmp($res->content,
+ qr{submitted},
+ "form submission ok under mod_perl");
+
+ $url = "/stdin_override/cgi";
+ $res = POST $url, [submit=>1];
+ ok t_cmp($res->content,
+ qr{submitted},
+ "stdin reset, form submission ok under cgi");
+}
Index: ModPerl-Registry/t/conf/extra.conf.in
===================================================================
--- ModPerl-Registry/t/conf/extra.conf.in (revision 583059)
+++ ModPerl-Registry/t/conf/extra.conf.in (working copy)
@@ -183,6 +183,24 @@
PerlResponseHandler ModPerl::Registry
</Location>

+### test for mixed mod_perl/cgi environment ###
+<IfModule mod_alias.c>
+ Alias /stdin_override/mod_perl @ServerRoot@/cgi-bin/stdin.pl
+ Alias /stdin_override/cgi @ServerRoot@/cgi-bin/stdin.pl
+</IfModule>
+
+<Location /stdin_override/mod_perl>
+ SetHandler perl-script
+ PerlResponseHandler ModPerl::Registry
+ PerlOptions +ParseHeaders
+ Options +ExecCGI
+</Location>
+
+<Location /stdin_override/cgi>
+ SetHandler cgi-script
+ Options +ExecCGI
+</Location>
+
### deflate tests ###
<IfModule mod_alias.c>
Alias /registry_bb_deflate/ @ServerRoot@/cgi-bin/
Index: ModPerl-Registry/t/cgi-bin/stdin.pl
===================================================================
--- ModPerl-Registry/t/cgi-bin/stdin.pl (revision 0)
+++ ModPerl-Registry/t/cgi-bin/stdin.pl (revision 0)
@@ -0,0 +1,18 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use CGI qw/:standard/;
+
+print <<END;
+Content-type: text/plain
+
+END
+
+if (param()) {
+ my $submit = param('submit');
+ if ($submit) {
+ print "submitted";
+ }
+}

$> t/TEST -start-httpd -one-process
$> t/TEST -run-tests -verbose t/stdin.t
[...]
All tests successful.

That's without the IO patch, and I tried with :

Perl-5.8.8 (Redhat Fedora Core 7)
httpd/2.2.2/worker
httpd/2.2.2/prefork

I, for one, suspect there is some important variable we are not narrowing
down on here.

The IO patch itself, which avoids calling close() on the underlying FD, worries
me a little. Are we really fixing the problem, or just dodging it for this
particular case ?

With the patch, I see this test failing:
t/modperl/io_nested_with_cled_stds

I agree with the spirit of the patch. Avoiding the call to close() on
the underlying FD for STDIN makes sure nobody else calling open() could
accidentally grab it. (BTW, shouldn't the same fix be applied to STDOUT
as well?).

Looking at modperl_io_perlio_restore_stdin however, isn't this where the
real problem lies? It does restore Perl's stdin handle back to it's original
state, but I don't see where/how that code would guarantee STDIN to re-acquire
the same FD. Isn't that the bug ?

And avoiding the close() in the sugested patch would still need to undo
the override in modperl_io_perlio_restore_stdin() to allow the real, underlying
STDIN to be eventially closed, no ?

------------------------------------------------------------------------
Philippe M. Chiasson GPG: F9BFE0C2480E7680 1AE53631CB32A107 88C3A5A5
http://gozer.ectoplasm.org/ m/gozer\@(apache|cpan|ectoplasm)\.org/

Attachments:

signature.asc (0.24 KB)

dmitry at karasik

Oct 9, 2007, 2:45 AM

Post #14 of 19 (4423 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

There's one thing, it might be important, and might be not. The 'Alias'
setting didn't work on my setup, so instead of finding what's wrong with Alias,
I changed the config so the setup has two different but identical files perform
mod_perl and cgi requests. So this section was (to be pedantic) different:

> +<IfModule mod_alias.c>
> + Alias /stdin_override/mod_perl @ServerRoot@/cgi-bin/stdin.pl
> + Alias /stdin_override/cgi @ServerRoot@/cgi-bin/stdin.pl
> +</IfModule>
> +
> +<Location /stdin_override/mod_perl>
> + SetHandler perl-script
> + PerlResponseHandler ModPerl::Registry
> + PerlOptions +ParseHeaders
> + Options +ExecCGI
> +</Location>
> +
> +<Location /stdin_override/cgi>
> + SetHandler cgi-script
> + Options +ExecCGI
> +</Location>

Instead it was

+<Location /cgi-bin/stdin_mod_perl.pl>
+ SetHandler perl-script
+ PerlResponseHandler ModPerl::Registry
+ PerlOptions +ParseHeaders
+ Options +ExecCGI
+</Location>
+
+<Location /cgi-bin/stdin_cgi.pl>
+ SetHandler cgi-script
+ Options +ExecCGI
+</Location>

where stdin_cgi.pl and stdin_mod_perl.pl are copies of stdin.pl .
May I ask you to try that altered setup?

> Looking at modperl_io_perlio_restore_stdin however, isn't this where the
> real problem lies? It does restore Perl's stdin handle back to it's original
> state, but I don't see where/how that code would guarantee STDIN to re-acquire
> the same FD. Isn't that the bug ?

The problem is that modperl_io_perlio_restore_stdin() is called much later,
after possibly several further mod_perl requests are executed. I agree that
making sure to reassign FD to 0 (and stdout etc) would be a good idea, however,
by the time when the cgi script in question is being run, the FD 0 would stay
closed disregarding of whether modperl_io_perlio_restore_stdin() does the right
thing or not.

> And avoiding the close() in the sugested patch would still need to undo
> the override in modperl_io_perlio_restore_stdin() to allow the real, underlying
> STDIN to be eventially closed, no ?

I do not understand that need to close FD 0 at all. All the overriding business
is done here so that PerlIO-level STDIN glob would be something else than FD 0,
which is fine. My point is that FD 0 is a shared resource within apache process
space, and it is neither mod_perl nor anyone else's business to close it.

--
Sincerely,
Dmitry Karasik

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

swp5jhu02 at sneakemail

Dec 17, 2007, 1:31 AM

Post #15 of 19 (4113 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Hi,

Dmitry Karasik dmitry-at-karasik.eu.org wrote:
> mod_perl during the initialization process closes file descriptor 0,
> and frees it for further reuse in other processes. Therefore, any cgi script
> executing after mod_perl is initialized, and in the same process tree, will
> have file descriptor 0 closed. The concrete example where this behavior leads
> to impossibility of processing POST requests is reported earlier and is located
> at
>
> http://marc.info/?l=apache-modperl&m=119062450730646&w=2

Dmitry's patch from Sep 25, 2007
http://www.gossamer-threads.com/lists/modperl/modperl/94912 has still
not been applied to SVN. May I humbly ask that this patch is applied?

On the dev list
http://marc.info/?l=apache-modperl-dev&m=119125026607247&w=2 it seems
that whether or not this really is a bug is drawn into question. I can
confirm that it, really, really is a bug!

I'm now getting emails from others experiencing the same problem,
querying for a solution. Dmitry's patch fixes our problem. I'm now
forwarding links to the patch to other users. Wouldn't it be easier to
have this or another patch applied or another that also fixes the problem???

The patch didn't apply completely cleanly here against HEAD, but this
one does ( also at http://demo.capmon.dk/~pvm/modperl_io.patch ) :

~/mod_perl-2.0/src/modules/perl> svn diff
Index: modperl_io.c
===================================================================
--- modperl_io.c (revision 604795)
+++ modperl_io.c (working copy)
@@ -116,6 +116,7 @@
/* if STDIN is open, dup it, to be restored at the end of response */
if (handle && SvTYPE(handle) == SVt_PVGV &&
IoTYPE(GvIO(handle)) != IoTYPE_CLOSED) {
+ IO * io = GvIO(handle);
handle_save = gv_fetchpv(Perl_form(aTHX_
"Apache2::RequestIO::_GEN_%ld",
(long)PL_gensym++),
@@ -128,6 +129,17 @@
Perl_croak(aTHX_ "Failed to dup STDIN: %" SVf, get_sv("!",
TRUE));
}

+ /* In mixed environment of mod_perl and cgi scripts, cgi
+ * scripts may read content of POST requests off
+ * STDIN. do_close() calls actual close(0), freeing the
+ * descriptor 0 for reuse, and creating havoc for anyone
+ * reading from file descriptor 0. This hack changes the IO
+ * type to IoTYPE_STD, because do_close() does not call
+ * underlying close() on IO handles of these types, but does
+ * free the associated resources. */
+ if ( IoIFP(io) && PerlIO_fileno(IoIFP(io)) == 0)
+ IoTYPE(io) = IoTYPE_STD;
+
/* similar to PerlIO::scalar, the PerlIO::Apache layer doesn't
* have file descriptors, so STDIN must be closed before it can
* be reopened */

Thanks,

Peter
--
Peter Valdemar MĂ¸rch
http://www.morch.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

gozer at ectoplasm

Dec 17, 2007, 10:56 PM

Post #16 of 19 (4131 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Fred Moyer wrote:
> Peter Valdemar MĂ¸rch wrote:
>> Hi,
>>
>> Dmitry Karasik dmitry-at-karasik.eu.org wrote:
>>> mod_perl during the initialization process closes file descriptor 0,
>>> and frees it for further reuse in other processes. Therefore, any cgi
>>> script
>>> executing after mod_perl is initialized, and in the same process tree,
>>> will
>>> have file descriptor 0 closed. The concrete example where this
>>> behavior leads
>>> to impossibility of processing POST requests is reported earlier and
>>> is located
>>> at
>>>
>>> http://marc.info/?l=apache-modperl&m=119062450730646&w=2
>> Dmitry's patch from Sep 25, 2007
>> http://www.gossamer-threads.com/lists/modperl/modperl/94912 has still
>> not been applied to SVN. May I humbly ask that this patch is applied?
>
> I remember that Toersten and I put together a unit test for this but I
> don't think there was any change in behavior with this patch, so I'm not
> sure if anyone was able to demonstrate that it changed the behavior.

It certainly breaks t/modperl/io_nested_with_closed_stds
[error] [client 127.0.0.1] Filehandle STDOUT reopened as STDIN only for input at t/response/TestModperl/io_nested_with_closed_stds.pm line 49.\n

> If you have the tuits and could track down that unit test, show that it
> exercises the problem in question, we probably wouldn't need much else
> to apply this patch. I'd try to do this but am overextended on my tuits
> right now.

Yup, without a test case that correctly demonstrates the bug, and a patch
that doesn't break existing tests, not very likely to get applied.

--
Philippe M. Chiasson GPG: F9BFE0C2480E7680 1AE53631CB32A107 88C3A5A5
http://gozer.ectoplasm.org/ m/gozer\@(apache|cpan|ectoplasm)\.org/

Attachments:

signature.asc (0.24 KB)

torsten.foertsch at gmx

Dec 17, 2007, 11:56 PM

Post #17 of 19 (4113 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

On Tue 18 Dec 2007, Philippe M. Chiasson wrote:
> >> Dmitry's patch from Sep 25, 2007
> >> http://www.gossamer-threads.com/lists/modperl/modperl/94912 has still
> >> not been applied to SVN. May I humbly ask that this patch is applied?
> >
> > I remember that Toersten and I put together a unit test for this but I
> > don't think there was any change in behavior with this patch, so I'm not
> > sure if anyone was able to demonstrate that it changed the behavior.
>
> It certainly breaks t/modperl/io_nested_with_closed_stds
> [error] [client 127.0.0.1] Filehandle STDOUT reopened as STDIN only for
> input at t/response/TestModperl/io_nested_with_closed_stds.pm line 49.\n
>
> > If you have the tuits and could track down that unit test, show that it
> > exercises the problem in question, we probably wouldn't need much else
> > to apply this patch. I'd try to do this but am overextended on my tuits
> > right now.
>
> Yup, without a test case that correctly demonstrates the bug, and a patch
> that doesn't break existing tests, not very likely to get applied.

This patch is something I wanted to investigate more thoroughly but as ever
there is no time left for interesting problems.

I think Dmitrys argument about file descriptors being a shared/global resource
and hence must not be closed is very strong. But on the other hand existing
tests must not be broken. (or otherwise clearly explained why the test is
buggy) And of course a test that demonstate the bug is necessary.

Torsten

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

swp5jhu02 at sneakemail

Dec 19, 2007, 4:50 AM

Post #18 of 19 (4117 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Torsten Foertsch torsten.foertsch-at-gmx.net |Lists| wrote:
> I think Dmitrys argument about file descriptors being a shared/global resource
> and hence must not be closed is very strong. But on the other hand existing
> tests must not be broken. (or otherwise clearly explained why the test is
> buggy) And of course a test that demonstate the bug is necessary.

Thanks for your input.

I'll ask Dmitry if he'll provide relevant test case coverage so that the
patch can be applied. It may be a little while, though, christmas and
all. Just wanted to post that this is still breathing and that I now
realize we have the ball.

Peter

--
Peter Valdemar Mřrch
http://www.morch.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

roderich.schupp at googlemail

Dec 30, 2007, 6:15 AM

Post #19 of 19 (4053 views)
Permalink

Re: [mp2] mod_perl closes apache's stdin and/or stdout [In reply to]

Hi,
I just stumbled over this threat and think we see the effects of this
bug in the "real world". We're currently porting a medium sized
Web application from Apache 1.3 and mod_perl 1 to Apache 2.0
and mod_perl 2. The application is mostly written in Mason.

The first strange behaviour we encountered was that
authentication using mod_authnz_external would fail intermittently.
mod_authnz_external does authentication by invoking an
external program (presumably running setuid), passing it the
remote user user and password and interpreting the program's exit
code. I traced system calls for the Apache processes and
file descriptor 0 in the Apache process would sometimes be closed
when mod_authnz_external was invoked. It basically works like this
(note that the external_prog expects user+password on its stdin):

pipe(rd, wr);
if (!(pid = fork())) {
// child
close(wr);
dup2(rd, 0);
close(rd);
execve(external_prog,...)
}
// parent
close(rd);
write(wr, "user\npassword\n");
wait(pid);

Now, if file descriptor 0 is closed prior to the pipe call then rd = 0,
dup2(rd, 0) is a no-op and close(rd) closes stdin. The external program
inherits this closed stdin and will fail.
At that point I just accepted the fact that stdin may be closed in an
Apache process and patched mod_authnz_external to skip dup2/close when
rd == 0 already.

Later on, we discovered other failures with mod_authnz_external that
were similarly caused by a closed file descriptor 1 prior to its call.

We also have still unexplained intermittent problems to capture the
stdout of external programs run (with system()) from Mason scripts.
I suspect the reason may also be closed file descriptors 0, 1, or 2.

Our Apache only serves static content, some classic CGI scripts
and lots of Mason pages (i.e. we only deal with mod_perl indirectly
thru Mason) and uses only standard Apache modules except for
mod_authnz_external. I can ruled out the letter to cause this
file descriptor corruption and a pristine Apache process has fd 0 and fd 1
on /dev/null and f2 on its error_log as expected.

Cheers, Roderich

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] perl
For additional commands, e-mail: dev-help [at] perl

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads