Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue15180] Cryptic traceback from os.path.join when mixing str & bytes

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Jun 25, 2012, 4:45 AM

Post #1 of 15 (233 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes

New submission from Nick Coghlan <ncoghlan [at] gmail>:

As seen in #4489, the traceback when mixing str and bytes in os.path.join is rather cryptic and hard to decipher if you've never encountered it before:

>>> import os.path
>>> os.path.join(b'', '')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.2/posixpath.py", line 78, in join
if b.startswith(sep):
TypeError: startswith first arg must be str or a tuple of str, not bytes

>>> os.path.join('', b'')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.2/posixpath.py", line 78, in join
if b.startswith(sep):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

While it's slightly less cryptic with a real source file (since you can at least see the os.path.join call), you have to have some how idea of how os.path.join works to realise that:
- type(sep) == type(args[0])
- b in args[1:]

The challenge is to generate a more user friendly error message without making the normal case of correct types any slower.

----------
components: Library (Lib)
messages: 163945
nosy: ncoghlan
priority: normal
severity: normal
stage: needs patch
status: open
title: Cryptic traceback from os.path.join when mixing str & bytes
type: enhancement
versions: Python 3.3

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jun 25, 2012, 4:47 AM

Post #2 of 15 (233 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

Catch TypeError, check whether bytes & str are being mixed, re-raise if not, say something user-friendly if yes?

----------
nosy: +hynek

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jun 25, 2012, 4:57 AM

Post #3 of 15 (232 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Nick Coghlan <ncoghlan [at] gmail> added the comment:

Yeah, that should do it - I just hadn't looked at the structure of the code to see how annoying it will be to separate the check out from the if statement.

Good use case for PEP 409, too :)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jun 25, 2012, 6:35 AM

Post #4 of 15 (226 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

Here's a fix. It would be nice if it could make for 3.3 final I guess. Tests pass for Linux and OS X.

----------
keywords: +needs review, patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file26156/nicer-error-on-str-bytes-mix.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jun 25, 2012, 7:19 AM

Post #5 of 15 (227 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Changes by Arfrever Frehtes Taifersar Arahesis <Arfrever.FTA [at] GMail>:


----------
nosy: +Arfrever

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jun 25, 2012, 8:31 AM

Post #6 of 15 (228 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

Just a nit, you can use double quotes instead of escaping apostrophes ;)

----------
nosy: +pitrou

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jun 30, 2012, 6:12 PM

Post #7 of 15 (224 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Terry J. Reedy <tjreedy [at] udel> added the comment:

This strikes me as a bugfix that does not get backported because code might depend on the bug. If the policy for exception messages, such as it is, documented somewhere?

----------
nosy: +terry.reedy

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 2, 2012, 12:51 PM

Post #8 of 15 (225 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Éric Araujo <merwok [at] netwok> added the comment:

> This strikes me as a bugfix that does not get backported because code might depend on
> the bug. If the policy for exception messages, such as it is, documented somewhere?

I don’t know if it’s written anywhere, but the rule I follow is that even though exact error messages are not specified by the Python language, they should not be changed in bugfix releases (I seem to remember bugs for argparse and tarfile where this was invoked). The commit policy seems in flux these days however, with code cleanups going in stable versions (urllib and collections recently).

----------
nosy: +eric.araujo, georg.brandl, r.david.murray

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 2, 2012, 12:53 PM

Post #9 of 15 (224 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

> If the policy for exception messages, such as it is, documented
> somewhere?

I think we are quite free to change exception messages, as long as the change is motivated. They are not part of the API definitions, and people shouldn't rely on them (the exception may for KeyError and such exceptions where the message is actually the value of the failed key).

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 2, 2012, 1:13 PM

Post #10 of 15 (225 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Éric Araujo <merwok [at] netwok> added the comment:

Patch looks good. Left some comments on Rietveld and +1 to Antoine’s note about quotes :)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 2, 2012, 4:02 PM

Post #11 of 15 (225 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Terry J. Reedy <tjreedy [at] udel> added the comment:

This is definitely a motivated change ;-). I would like this re-typed as behavior and applied to 2.7, 3.2, and 3.3. I see the current message as being much like a garbled sentence in the docs that people can only understand if they already understand the background.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 3, 2012, 5:32 AM

Post #12 of 15 (222 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

Cool, I'll finish it up in the sprints on Saturday.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 12, 2012, 4:25 AM

Post #13 of 15 (177 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

I didn’t, because I pulled an Antoine and enjoyed weather, people and Florence. :)

I understand it’s okay to apply this towards 3.2 & 3.3 (with the proposed fixes)?

2.7 is not necessary because this problem doesn’t exist there:

> os.path.join(b'foo', u'bar')
u'foo/bar'

Introducing explicit sanity checks there would cause people wanting to eat our hearts. :)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 15, 2012, 7:47 AM

Post #14 of 15 (171 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset cf1ac0c9e753 by Hynek Schlawack in branch '3.2':
#15180: Clarify posixpath.join() error message when mixing str & bytes
http://hg.python.org/cpython/rev/cf1ac0c9e753

New changeset 1462b963e5ce by Hynek Schlawack in branch 'default':
#15180: Clarify posixpath.join() error message when mixing str & bytes
http://hg.python.org/cpython/rev/1462b963e5ce

----------
nosy: +python-dev

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Jul 15, 2012, 8:04 AM

Post #15 of 15 (171 views)
Permalink
[issue15180] Cryptic traceback from os.path.join when mixing str & bytes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

Fixed for 3.2 & default.

----------
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15180>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.