Mailing List Archive: Perl: porters

Enable IO::Pipe Tests on Windows?

Index | Next | Previous | View Threaded

david at justatheory

May 13, 2012, 3:37 PM

Post #1 of 8 (96 views)
Permalink

Enable IO::Pipe Tests on Windows?

Porters,

I've been looking into IPC modules, and noticed that IO::Pipe's tests are skipped on Windows:

http://perl5.git.perl.org/perl.git/blob/HEAD:/dist/IO/t/io_pipe.t#l40

However, @BrowserUK tells me [via Perlmonks](http://www.perlmonks.org/?node_id=970255) that IO::Pipe works fine on Windows, and has for a while. Should the tests be re-enabled?

Best,

David

david at justatheory

Aug 9, 2012, 3:08 PM

Post #2 of 8 (75 views)
Permalink

Re: Enable IO::Pipe Tests on Windows? [In reply to]

On May 13, 2012, at 3:37 PM, David E. Wheeler wrote:

> Porters,
>
> I've been looking into IPC modules, and noticed that IO::Pipe's tests are skipped on Windows:
>
> http://perl5.git.perl.org/perl.git/blob/HEAD:/dist/IO/t/io_pipe.t#l40
>
> However, @BrowserUK tells me [via Perlmonks](http://www.perlmonks.org/?node_id=970255) that IO::Pipe works fine on Windows, and has for a while. Should the tests be re-enabled?

Here’s a patch that does this. Maybe someone can throw it into a branch somewhere to be smoked on Windows?

Best,

David

Attachments:

0001-Enable-IO-Pipe-tests-on-Windows.patch (2.96 KB)

Steve.Hay at verosoftware

Aug 10, 2012, 12:38 AM

Post #3 of 8 (75 views)
Permalink

RE: Enable IO::Pipe Tests on Windows? [In reply to]

David E. Wheeler wrote on 2012-08-09:
> On May 13, 2012, at 3:37 PM, David E. Wheeler wrote:
>
>> Porters,
>>
>> I've been looking into IPC modules, and noticed that IO::Pipe's tests

>> are skipped on Windows:
>>
>>
>> http://perl5.git.perl.org/perl.git/blob/HEAD:/dist/IO/t/io_pipe.t#l40
>>
>> However, @BrowserUK tells me [via
> Perlmonks](http://www.perlmonks.org/?node_id=970255) that IO::Pipe
> works fine on Windows, and has for a while. Should the tests be re-
enabled?
>
> Here's a patch that does this. Maybe someone can throw it into a
> branch somewhere to be smoked on Windows?

I put it into my current blead workspace and tried it out, but it
doesn't work :-(

Process Explorer shows me that the perl.exe running io_pipe.t has
spawned a child perl.exe like this:

perl.exe -pe "BEGIN{$SIG{ALRM} = sub {print qq(not ok 4\n); exit}; alarm
10} s/not //"

and it's now just sat there doing nothing. I killed the child process to
see if the parent (../../t/perl -I../.. -MTestInit=U2T t/io_pipe.t)
would continue with other things, but that's also now hung too.

Verbose output showed ok 1 and ok 2, then the child hung. After I killed
the child I got ok 4, ok 5 and ok 6 but then had to kill the parent.

So it seems like IO::Pipe->reader() works (as BrowserUk said, and his
example works for me too) but IO::Pipe->writer() doesn't?...

bulk88 at hotmail

Aug 10, 2012, 3:34 PM

Post #4 of 8 (75 views)
Permalink

RE: Enable IO::Pipe Tests on Windows? [In reply to]

> Subject: RE: Enable IO::Pipe Tests on Windows?
> Date: Fri, 10 Aug 2012 08:38:53 +0100
> From: Steve.Hay [at] verosoftware
> To: david [at] justatheory; perl5-porters [at] perl
> I put it into my current blead workspace and tried it out, but it
> doesn't work :-(
>
> Process Explorer shows me that the perl.exe running io_pipe.t has
> spawned a child perl.exe like this:
>
> perl.exe -pe "BEGIN{$SIG{ALRM} = sub {print qq(not ok 4\n); exit}; alarm
> 10} s/not //"
>
> and it's now just sat there doing nothing. I killed the child process to
> see if the parent (../../t/perl -I../.. -MTestInit=U2T t/io_pipe.t)
> would continue with other things, but that's also now hung too.
>
> Verbose output showed ok 1 and ok 2, then the child hung. After I killed
> the child I got ok 4, ok 5 and ok 6 but then had to kill the parent.
>
> So it seems like IO::Pipe->reader() works (as BrowserUk said, and his
> example works for me too) but IO::Pipe->writer() doesn't?...

Child process cmd line is "C:\p517\bin\perl.exe -pe "BEGIN{$SIG{ALRM} = sub {print qq(not ok 4\n); exit}; alarm 10} s/not //"", parent cmd line is "perl io_pipe.t". Child perl curcop file is "-e", curcop line is 0. Child callstack is
___________________________________________________
     ntdll.dll!ZwReadFile() + 0xa bytes
     kernel32.dll!ReadFile() + 0xb6 bytes
     msvcr90.dll!_read_nolock(int fh=0, void * inputbuf=0x0000000001c888e0, unsigned int cnt=0) Line 233 + 0x1f bytes    C
     msvcr90.dll!_read(int fh=0, void * buf=0x0000000000000000, unsigned int cnt=0) Line 93 + 0xd bytes    C
     perl517.dll!win32_read(int fd=0, void * buf=0x0000000001c888e0, unsigned int cnt=8192) Line 3209    C
     perl517.dll!PerlLIORead(IPerlLIO * piPerl=0x00000000003d0f28, int handle=0, void * buffer=0x0000000001c888e0, unsigned int count=8192) Line 1034    C++
     perl517.dll!PerlIOUnix_read(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c47010, void * vbuf=0x0000000001c888e0, unsigned __int64 count=8192) Line 2789 + 0x2c bytes    C
     perl517.dll!Perl_PerlIO_read(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c47010, void * vbuf=0x0000000001c888e0, unsigned __int64 count=8192) Line 1679 + 0x54 bytes    C
     perl517.dll!PerlIOBuf_fill(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c51a00) Line 4033 + 0x21 bytes    C
     perl517.dll!Perl_PerlIO_fill(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c51a00) Line 1776 + 0x50 bytes    C
     perl517.dll!PerlIOBase_read(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c51a00, void * vbuf=0x000000000012d4d8, unsigned __int64 count=1) Line 2170 + 0xf bytes    C
     perl517.dll!PerlIOBuf_read(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c51a00, void * vbuf=0x000000000012d4d8, unsigned __int64 count=1) Line 4054 + 0x19 bytes    C
     perl517.dll!Perl_PerlIO_read(interpreter * my_perl=0x0000000001c269f8, _PerlIO * * f=0x0000000001c51a00, void * vbuf=0x000000000012d4d8, unsigned __int64 count=1) Line 1679 + 0x54 bytes    C
     perl517.dll!PerlIO_getc(_PerlIO * * f=0x0000000001c51a00) Line 4911 + 0x1a bytes    C
>    perl517.dll!Perl_sv_gets(interpreter * my_perl=0x0000000001c269f8, sv * const sv=0x0000000001c28db0, _PerlIO * * const fp=0x0000000001c51a00, long append=0) Line 7761 + 0xd bytes    C
     perl517.dll!Perl_do_readline(interpreter * my_perl=0x0000000001c269f8) Line 1689 + 0x1f bytes    C
     perl517.dll!Perl_pp_readline(interpreter * my_perl=0x0000000001c269f8) Line 348    C
     perl517.dll!Perl_runops_debug(interpreter * my_perl=0x0000000001c269f8) Line 2129 + 0x11 bytes    C
     perl517.dll!S_run_body(interpreter * my_perl=0x0000000001c269f8, long oldscope=1) Line 2389    C
     perl517.dll!perl_run(interpreter * my_perl=0x0000000001c269f8) Line 2307    C
     perl517.dll!RunPerl(int argc=3, char * * argv=0x00000000003c0d20, char * * env=0x000000000034bdd0) Line 270 + 0xa bytes    C++
     perl.exe!main(int argc=3, char * * argv=0x00000000003c0d20, char * * env=0x000000000034bc30) Line 24    C
     perl.exe!__tmainCRTStartup() Line 586 + 0x19 bytes    C
     kernel32.dll!BaseProcessStart() + 0x2c bytes
_____________________________________________________
In parent perl, callstack is
_____________________________________________________
     ntdll.dll!ZwWaitForMultipleObjects() + 0xa bytes
     kernel32.dll!WaitForMultipleObjectsEx() - 0x65 bytes
     user32.dll!RealMsgWaitForMultipleObjectsEx() + 0x147 bytes
     user32.dll!MsgWaitForMultipleObjects() + 0x40 bytes
>    perl517.dll!win32_msgwait(interpreter * my_perl=0x0000000001c269f8, unsigned long count=1, void * * handles=0x000000000012f768, unsigned long timeout=4294967295, unsigned long * resultp=0x000000000012f770) Line 2162 + 0x26 bytes    C
     perl517.dll!win32_waitpid(int pid=22584, int * status=0x000000000012f830, int flags=0) Line 2292    C
     perl517.dll!PerlProcWaitpid(IPerlProc * piPerl=0x00000000003cf1b0, int pid=22584, int * status=0x000000000012f830, int flags=0) Line 1678    C++
     perl517.dll!Perl_wait4pid(interpreter * my_perl=0x0000000001c269f8, int pid=22584, int * statusp=0x000000000012f830, int flags=0) Line 3178 + 0x2c bytes    C
     perl517.dll!Perl_pp_waitpid(interpreter * my_perl=0x0000000001c269f8) Line 4115 + 0x18 bytes    C
     perl517.dll!Perl_runops_debug(interpreter * my_perl=0x0000000001c269f8) Line 2129 + 0x11 bytes    C
     perl517.dll!S_run_body(interpreter * my_perl=0x0000000001c269f8, long oldscope=1) Line 2389    C
     perl517.dll!perl_run(interpreter * my_perl=0x0000000001c269f8) Line 2307    C
     perl517.dll!RunPerl(int argc=2, char * * argv=0x00000000003b0660, char * * env=0x000000000034bde0) Line 270 + 0xa bytes    C++
     perl.exe!main(int argc=2, char * * argv=0x00000000003b0660, char * * env=0x000000000034bc40) Line 24    C
     perl.exe!__tmainCRTStartup() Line 586 + 0x19 bytes    C
     kernel32.dll!BaseProcessStart() + 0x2c bytes
_______________________________________________________
curcop file is "C:/p517/lib/IO/Pipe.pm", curcop line is 152. That is it for now. I didn't analyze the problem any further right now.

bulk88 at hotmail

Aug 10, 2012, 8:09 PM

Post #5 of 8 (74 views)
Permalink

RE: Enable IO::Pipe Tests on Windows? [In reply to]

It is a buffering problem.

Here is the chunk of io_pipe.t that deadlocks. The child Perl process is reading stdin, thats the fh=0 to CRT you see above.
______________________________________________________
if ($is_win32) {
    print "ok $_ # skipped: $is_win32\n" for 1..4;
} else {
    $pipe = new IO::Pipe->reader($perl, '-e', 'print qq(not ok 1\n)');
    while (<$pipe>) {
      s/^not //;
      print;
    }
    $pipe->close or print "# \$!=$!\nnot ";
    print "ok 2\n";
    $cmd = 'BEGIN{$SIG{ALRM} = sub {print qq(not ok 4\n); exit}; alarm 10} s/not //';
    $pipe = new IO::Pipe->writer($perl, '-pe', $cmd);
    print $pipe "not ok 3\n" ;
    $pipe->close or print "# \$!=$!\nnot ";
    print "ok 4\n";
}
__________________________________________________________

The parent Perl process is supposed to do "    print $pipe "not ok 3\n" ;" which will unlock the child process (or atleast that ReadFile will succeed). I traced pp_print op. The "not ok" string got copied to a buffer in PerlIO and control was returned, pp_print or its child functions, never touch/call the CRT or Kernel32 IO functions for that "print $pipe "not ok 3\n" ;". How to fix this, someone else will know better than me.

bulk88 at hotmail

Aug 10, 2012, 8:20 PM

Post #6 of 8 (74 views)
Permalink

RE: Enable IO::Pipe Tests on Windows? [In reply to]

It might also be a newline problem rather than a buffering problem, or both, or a problem with alarm (even though alarm is a failure path, and probably has its own set of problems on Windows since alarm isn't Win32 native, since it doesn't kick in after 10 seconds for me).
______________________________________________________________
C:\p517\perl\dist\IO\t>perl -MO=Deparse -pe "BEGIN{$SIG{ALRM} = sub {print qq(no
t ok 4\n); exit}; alarm 10} s/not //"
sub BEGIN {
$SIG{'ALRM'} = sub {
print "not ok 4\n";
exit;
}
;
alarm 10;
}
LINE: while (defined($_ = <ARGV>)) {
s/not //;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK

C:\p517\perl\dist\IO\t>
______________________________________________________________

bulk88 at hotmail

Aug 11, 2012, 12:38 AM

Post #7 of 8 (74 views)
Permalink

RE: Enable IO::Pipe Tests on Windows? [In reply to]

Its not a buffering problem, completely (it might be a buffering problem if the other problems are fixed). If I add "    $pipe->autoflush(1);", I step through pp_print and PerlIO and it does call the CRT which calls WriteFile eventually from a callee of pp_print at the marked ">>>>>" line. Still, the child never exits.
_____________________________
    }
    if (MARK <= SP)
        goto just_say_no;
    else {
        if (PL_op->op_type == OP_SAY) {
        if (PerlIO_write(fp, "\n", 1) == 0 || PerlIO_error(fp))
            goto just_say_no;
        }
            else if (PL_ors_sv && SvOK(PL_ors_sv))
        if (!do_print(PL_ors_sv, fp)) /* $\ */
            goto just_say_no;

        if (IoFLAGS(io) & IOf_FLUSH)
>>>>>>>>>>>>>>>        if (PerlIO_flush(fp) == EOF)
            goto just_say_no;
    }
    }
    SP = ORIGMARK;
    XPUSHs(&PL_sv_yes);
    RETURN;

just_say_no:
    SP = ORIGMARK;
    XPUSHs(&PL_sv_undef);
    RETURN;
_________________________

bulk88 at hotmail

Aug 12, 2012, 8:29 PM

Post #8 of 8 (73 views)
Permalink

RE: Enable IO::Pipe Tests on Windows? [In reply to]

Handle 7A4 is the CRT's fh=0 handle that ReadFile and NtReadFile are trying to read from CRT from PerlIO from pp_readline. Handle 7AC has the same name as 7A4 but I don't know its purpose. On real Unix, Perl diamond operator is supposed to return undef after the parent process closes() the STDIN handle of the child process, right? The parent perl process doesn't have any "\Device\NamedPipe\Win32Pipes.*.*" handles at this point, but it is doing a waitpid on the child as described earlier.
From IO::Pipe.
________________________________
sub close {
my $fh = shift;
my $r = $fh->SUPER::close(@_);
waitpid(${*$fh}{'io_pipe_pid'},0)
if(defined ${*$fh}{'io_pipe_pid'});
$r;
}
_________________________________
Hmmm, I did a Close Handle from Process Explorer, on Handle 7AC, child Perl unfroze from diamond operator, diamond operator returned undef (I forgot to step through stdio/PerlIO to see exactly what CRT read() returned), and child Perl process exited. Sounds like a resource leak, somewhere in the child process. I dont know what Handle 7AC is.

Attachments:

pp_readline handles.PNG (62.7 KB)

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads