Mailing List Archive: Python: Python

Read STDIN as bytes rather than a string

Index | Next | Previous | View Threaded

jason at powerpull

Jun 18, 2012, 4:13 PM

Post #1 of 6 (467 views)
Permalink

Read STDIN as bytes rather than a string

I tried this:

Python 3.2.2 (default, Feb 24 2012, 20:07:04)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import io
>>> fh = io.open(sys.stdin)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: invalid file: <_io.TextIOWrapper name='<stdin>' mode='r'
encoding='UTF-8'>
>>> fh = io.open(sys.stdin, "rb")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: invalid file: <_io.TextIOWrapper name='<stdin>' mode='r'
encoding='UTF-8'>
--
http://mail.python.org/mailman/listinfo/python-list

benjamin.kaplan at case

Jun 18, 2012, 4:18 PM

Post #2 of 6 (467 views)
Permalink

Re: Read STDIN as bytes rather than a string [In reply to]

On Mon, Jun 18, 2012 at 4:13 PM, Jason Friedman <jason [at] powerpull> wrote:
> I tried this:
>
> Python 3.2.2 (default, Feb 24 2012, 20:07:04)
> [GCC 4.6.1] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys
>>>> import io
>>>> fh = io.open(sys.stdin)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: invalid file: <_io.TextIOWrapper name='<stdin>' mode='r'
> encoding='UTF-8'>
>>>> fh = io.open(sys.stdin, "rb")
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: invalid file: <_io.TextIOWrapper name='<stdin>' mode='r'
> encoding='UTF-8'>
> --

You want to read from sys.stdin.buffer
--
http://mail.python.org/mailman/listinfo/python-list

lists at cheimes

Jun 18, 2012, 4:35 PM

Post #3 of 6 (463 views)
Permalink

Re: Read STDIN as bytes rather than a string [In reply to]

Am 19.06.2012 01:13, schrieb Jason Friedman:
> I tried this:

sys.stdin wraps a buffered reader which itself wraps a raw file reader.

>>> sys.stdin
<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>
>>> sys.stdin.buffer
<_io.BufferedReader name='<stdin>'>
>>> sys.stdin.buffer.raw
<_io.FileIO name='<stdin>' mode='rb'>

You should read from sys.stdin.buffer unless you really need the bare
metal.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

jason at powerpull

Jun 18, 2012, 4:49 PM

Post #4 of 6 (461 views)
Permalink

Re: Read STDIN as bytes rather than a string [In reply to]

> sys.stdin wraps a buffered reader which itself wraps a raw file reader.
>
>>>> sys.stdin
> <_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>
>>>> sys.stdin.buffer
> <_io.BufferedReader name='<stdin>'>
>>>> sys.stdin.buffer.raw
> <_io.FileIO name='<stdin>' mode='rb'>
>
> You should read from sys.stdin.buffer unless you really need the bare
> metal.
>

Thank you, and just to close the loop:

$ cat ~/my-input.py
#!/opt/python/bin/python3
import sys
print(sys.stdin.buffer.read())

$ echo hello | ~/my-input.py
b'hello\n'
--
http://mail.python.org/mailman/listinfo/python-list

jason at powerpull

Jun 18, 2012, 4:53 PM

Post #5 of 6 (466 views)
Permalink

Re: Read STDIN as bytes rather than a string [In reply to]

Which leads me to another question ... how can I debug these things?

$ echo 'hello' | python3 -m pdb ~/my-input.py
> /home/jason/my-input.py(2)<module>()
-> import sys
(Pdb) *** NameError: name 'hello' is not defined
--
http://mail.python.org/mailman/listinfo/python-list

oscar.j.benjamin at gmail

Jun 19, 2012, 5:24 AM

Post #6 of 6 (457 views)
Permalink

Re: Read STDIN as bytes rather than a string [In reply to]

On 19 June 2012 00:53, Jason Friedman <jason [at] powerpull> wrote:

> Which leads me to another question ... how can I debug these things?
>
> $ echo 'hello' | python3 -m pdb ~/my-input.py
> > /home/jason/my-input.py(2)<module>()
> -> import sys
> (Pdb) *** NameError: name 'hello' is not defined
> --
> http://mail.python.org/mailman/listinfo/python-list
>

It's difficult to debug problems that are related to reading from stdin. I
don't know of any good way, so I just end up doing things like adding print
statements and checking the output rather than using a debugger. Tools like
hd can help with checking the input/output files that you're using. If
there were a debugger it would probably need to be one with a GUI - the
only one I know is spyder but I don't think that will allow you to pipe
anything on stdin.

One thing I wanted to say is that if your script is intended to work on
Windows you'll need to use msvcrt.setmode() to disable newline translation
on stdin (I haven't tested with Python 3.x, but it's definitely necessary
with Python 2.x). See Frazil's post here:
http://stackoverflow.com/questions/2850893/reading-binary-data-from-stdin

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads