Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue14082] shutil doesn't copy extended attributes

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Apr 29, 2012, 8:46 AM

Post #1 of 17 (298 views)
Permalink
[issue14082] shutil doesn't copy extended attributes

Changes by Hynek Schlawack <hs [at] ox>:


----------
stage: -> patch review

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 30, 2012, 4:09 PM

Post #2 of 17 (286 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

Hynek, did you get a notification of my review on Rietveld?

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 30, 2012, 5:32 PM

Post #3 of 17 (287 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

I didn't. :/ I'll look into it tomorrow.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 1, 2012, 4:23 AM

Post #4 of 17 (287 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

I have answered to the (two weeks old :-/) review. There are three open questions in there we'll have to figure out before I fix the patch:

- should copyxattr() remove xattrs in dst that aren't present in src? Make it an option like `remove_missing_xattr`?

- use "None" for `namespaces` in copyxattrs() to indicate we want to copy all of them?

- add a ignore_errors option?

ISTM, that "all namespaces" don't make much sense without ignore_errors as there seem to be some internal xattr etc.

Suggestion: copyxattrs() has ignore_errors as default and returns a list of xattr it couldn't copy as (xattr, exception) tuples? Or an "on error" handler like in rmtree? I'd prefer the first one as ISTM that failures happen more often than not.

----------
assignee: -> hynek

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 2, 2012, 5:00 AM

Post #5 of 17 (286 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Charles-François Natali <neologix [at] free> added the comment:

I really like the idea of adding extended attributes copy to shutil.cop2().
However, I'm not convinced that exposing the raw copyxattr() is
necessary (I can't really come up with a use case for this).

So I'd suggest make copyxattr() private, and wait until someone asks
for the functionality.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 2, 2012, 5:07 AM

Post #6 of 17 (285 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

> I really like the idea of adding extended attributes copy to shutil.cop2().
> However, I'm not convinced that exposing the raw copyxattr() is
> necessary (I can't really come up with a use case for this).
>
> So I'd suggest make copyxattr() private, and wait until someone asks
> for the functionality.

I agree. If someone needs it, (s)he will be able to advice on useful
semantics re replacement. For copy2() it’s a non-issue.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 5, 2012, 8:45 AM

Post #7 of 17 (286 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

So here's an updated patch. I've stripped everything away we don't need for copy2 integration. It catches EPERM, ENOTSUP and ENODATA (for race conditions).

Happy reviewing. :)

----------
Added file: http://bugs.python.org/file25462/copy2-xattr.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 5, 2012, 10:51 AM

Post #8 of 17 (285 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

The copyxattr() function should be private (_copyxattr()).
For some reason the tests are failing here:

======================================================================
ERROR: test_copy2_xattr (test.test_shutil.TestShutil)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/antoine/cpython/default/Lib/test/test_shutil.py", line 410, in test_copy2_xattr
os.setxattr(src, 'user.foo', b'42')
OSError: [Errno 95] Operation not supported

======================================================================
ERROR: test_copyxattr (test.test_shutil.TestShutil)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/antoine/cpython/default/Lib/test/test_shutil.py", line 296, in test_copyxattr
os.setxattr(src, 'user.foo', b'42')
OSError: [Errno 95] Operation not supported

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 5, 2012, 11:54 AM

Post #9 of 17 (289 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

> The copyxattr() function should be private (_copyxattr()).

Ok. I presumed that not adding it to __all__ is "private enough".

> For some reason the tests are failing here:
>
> ======================================================================
> ERROR: test_copy2_xattr (test.test_shutil.TestShutil)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/antoine/cpython/default/Lib/test/test_shutil.py", line 410, in test_copy2_xattr
> os.setxattr(src, 'user.foo', b'42')
> OSError: [Errno 95] Operation not supported
>
> ======================================================================
> ERROR: test_copyxattr (test.test_shutil.TestShutil)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/antoine/cpython/default/Lib/test/test_shutil.py", line 296, in test_copyxattr
> os.setxattr(src, 'user.foo', b'42')
> OSError: [Errno 95] Operation not supported

Looks like your file system Python uses for tmp files doesn't support
xattr. That's bad because you can't verify. How should I cope with that?
try/catch on the first setxattr() and skip the test if it fails? Is the
an official way to do that?

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 5, 2012, 12:24 PM

Post #10 of 17 (283 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

> Looks like your file system Python uses for tmp files doesn't support
> xattr. That's bad because you can't verify. How should I cope with that?
> try/catch on the first setxattr() and skip the test if it fails? Is the
> an official way to do that?

Well, apparently the extended attributes tests in test_os get skipped
with the following message: "no non-broken extended attribute support".
Perhaps you need the same condition somewhere.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 5, 2012, 1:34 PM

Post #11 of 17 (287 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

Ok, I've extracted the xattr checker from test_os.py and transformed it into a caching decorator like skip_unless_symlinks.

_copyxattr rename is also done.

----------
Added file: http://bugs.python.org/file25477/copy2-xattr-with-better-protection.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 10, 2012, 11:28 AM

Post #12 of 17 (265 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

The patch looks ok, but it would be nice if someone with non-broken xattr support could test it. Although I suppose you did run the test suite, Hynek?

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 10, 2012, 12:06 PM

Post #13 of 17 (269 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Hynek Schlawack <hs [at] ox> added the comment:

I did. The only error I got was

ERROR: test_idna (test.test_socket.GeneralModuleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/p2/Lib/test/test_socket.py", line 1182, in test_idna
socket.gethostbyname('испытание.python.org')
socket.gaierror: [Errno -3] Temporary failure in name resolution

I presume it's unrelated. :)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 12, 2012, 10:05 AM

Post #14 of 17 (267 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset 85824b819bcb by Antoine Pitrou in branch 'default':
Issue #14082: shutil.copy2() now copies extended attributes, if possible.
http://hg.python.org/cpython/rev/85824b819bcb

----------
nosy: +python-dev

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 12, 2012, 10:05 AM

Post #15 of 17 (267 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

Pushed now. Hopefully the buildbots won't moan.

----------
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 12, 2012, 10:25 AM

Post #16 of 17 (258 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

Looks like we have our first buildbot failure:


======================================================================
FAIL: test_copyxattr (test.test_shutil.TestShutil)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/test_shutil.py", line 346, in test_copyxattr
self.assertEqual(['user.bar'], os.listxattr(dst))
AssertionError: Lists differ: ['user.bar'] != [.'security.selinux', 'user.bar...

First differing element 0:
user.bar
security.selinux

Second list contains 1 additional elements.
First extra element 1:
user.bar

- ['user.bar']
+ ['security.selinux', 'user.bar']

----------
status: closed -> open

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 12, 2012, 2:59 PM

Post #17 of 17 (268 views)
Permalink
[issue14082] shutil doesn't copy extended attributes [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

Hynek's patch (communicated over IRC, committed in 8d85f9920878) seems to have fixed the failure.

----------
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14082>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.