Mailing List Archive: Python: Bugs

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows

Index | Next | Previous | View Threaded

report at bugs

Oct 10, 2009, 6:57 PM

Post #1 of 24 (446 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows

Jason R. Coombs <jaraco [at] jaraco> added the comment:

Thanks to some hints by eric.smith, I've run the regression tests with
this patch applied. It turns out there are still some outstanding issues
with draft 9.

One issue was in test_glob.py where broken symlinks would fail to be
matched by glob. This was due to the fact that lexists still needed to
be implemented in ntpath.py. Draft 10 includes this capability.

Another problem the regression tests elicited is more insidious. The
patch breaks test_os.test_1686475, which references issue1686475. The
new symlink-aware stat must traverse symlinks. Unfortunately, the
current method of resolving the symlink target (as recommended by
Microsoft) fails to work if the symlink target is in use (or otherwise a
handle cannot be obtained). Unfortunately, it's not possible to call
GetFinalPathNameByHandle without a handle, the same way as issue1686475
was addressed by using a different API call.

I'm still devising a workaround for this undesirable behavior, but I
wanted to report that progress is still being made and I wanted to
capture the intermediate patch for posterity.

Stay tuned for more info.

----------
Added file: http://bugs.python.org/file15100/windows symlink draft 10.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 23, 2009, 3:08 PM

Post #2 of 24 (393 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

After some in-depth analysis, I determined the following:

1) The new symlink-aware os.stat has to find the target of a symlink to
properly function.
2) The previously-proposed patch uses GetFilenameByHandle in os.stat to
find the target, but this fails when a handle can't be obtained (such as
when a symlink points to pagefile.sys.
3) Explorer doesn't use GetFilenameByHandle. I used ProcMon to monitor
the API calls when it looks up the properties for a symlink to
pagefile.sys. It appears to use CreateFile and DeviceIoControl to trace
through the symlink chain.

So, I've written a proof of concept that determines a symlink target
using these API calls. The code is in jaraco.windows.filesystem at
https://svn.jaraco.com/jaraco/python/jaraco.windows/jaraco/windows/filesystem.py:trace_symlink_target
. For convenience, I've pasted the most relevant code segments at
http://paste.turbogears.org/paste/122986 .

I'd like to get comments on this implementation before porting this to
the C code and adding it to the os.stat call. One of the aspects that
makes me uncomfortable is the fact that I have to call relpath between
each iteration.

I have tested this approach fairly thoroughly. It appears to work under
all circumstances (through multiple symlinks across drives and
ultimately pointing at pagefile.sys).

If no one can suggest a better approach, I plan to code this into the patch.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 6:29 PM

Post #3 of 24 (380 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

This new version of the patch provides a workaround for the issue1686475
regression. It simply falls back to the traditional (symlink-naive) stat
behavior when a handle cannot be obtained for the specified filename.
My intention is that we can commit this patch (as it's still a strict
improvement over the previous behavior), and work out a fully-symlink
aware stat method in a subsequent patch.

There are still regression tests in test_posixpath.py that are failing
due to the non-implementation of realpath. I'll tackle this task next.

----------
Added file: http://bugs.python.org/file15198/windows symlink draft 11.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:36 PM

Post #4 of 24 (381 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14009/windows symlink draft 1.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:36 PM

Post #5 of 24 (381 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14039/windows symlink draft 2.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:36 PM

Post #6 of 24 (382 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14050/windows symlink draft 3.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:36 PM

Post #7 of 24 (382 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14130/windows symlink draft 4.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:36 PM

Post #8 of 24 (379 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14195/windows symlink draft 5.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:36 PM

Post #9 of 24 (381 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14206/windows symlink draft 6.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:37 PM

Post #10 of 24 (382 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14837/python-core_rev4274.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:37 PM

Post #11 of 24 (378 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14838/python-core_rev4275.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:37 PM

Post #12 of 24 (382 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14840/windows symlink draft 7.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:37 PM

Post #13 of 24 (381 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14877/windows symlink draft 8.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:37 PM

Post #14 of 24 (380 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file14878/windows symlink draft 9.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 25, 2009, 10:37 PM

Post #15 of 24 (380 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

Removed file: http://bugs.python.org/file15100/windows symlink draft 10.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 26, 2009, 11:22 AM

Post #16 of 24 (376 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

This latest patch (12) adds os.readlink(). Unfortunately, this causes
more tests in test_posixpath.py to break. I'll continue to investigate.

----------
Added file: http://bugs.python.org/file15204/windows symlink draft 12.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 26, 2009, 2:31 PM

Post #17 of 24 (373 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Jason R. Coombs <jaraco [at] jaraco>:

Removed file: http://bugs.python.org/file15198/windows symlink draft 11.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 26, 2009, 2:47 PM

Post #18 of 24 (373 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

This latest patch (13) addresses the regressions in test_posixpath.py by
repeating a technique found in the new lstat. That is, if a handle can't
be obtained for GetFinalPathNameByHandle (because the target is in use),
it is assumed that the target does in fact exist and the attributes are
retrieved from the parent directory.

----------
Added file: http://bugs.python.org/file15208/windows symlink draft 13.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 26, 2009, 2:49 PM

Post #19 of 24 (375 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Jason R. Coombs <jaraco [at] jaraco>:

Removed file: http://bugs.python.org/file15204/windows symlink draft 12.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 26, 2009, 3:12 PM

Post #20 of 24 (375 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

The problem I'm encountering now is tests for posixpath.realpath are
failing on Windows. These tests were previously skipped under Windows
because there was no os.symlink. Now that Windows has os.symlink, these
regression tests fail (test_realpath_basic,
test_realpath_resolve_before_normalizing, test_realpath_resolve_first,
and test_realpath_resolve_parent).

What is the correct fix for this? Should posixpath.realpath work on
Windows? If not, should the tests just be ignored under Windows?
Otherwise, should posixpath.realpath be rewritten to be robust under
both operating systems? Or is there another approach?

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 27, 2009, 12:00 PM

Post #21 of 24 (366 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

This latest patch (14) makes one minor change over the previous -
updates the documentation to include "Availability: Windows" for readlink.

----------
Added file: http://bugs.python.org/file15214/windows symlink draft 14.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 27, 2009, 12:00 PM

Post #22 of 24 (364 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Jason R. Coombs <jaraco [at] jaraco>:

Removed file: http://bugs.python.org/file15208/windows symlink draft 13.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 28, 2009, 5:59 PM

Post #23 of 24 (364 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Jason R. Coombs <jaraco [at] jaraco> added the comment:

This patch (15) implements ntpath.samefile, which corrects the
regression in test_shutil.

By my tests, this corrects all regressions caused by this patch except
for those in test_posixpath.py previously mentioned. I believe these
failing tests are due to the fact that posixpath tests are inappropriate
on the Windows platform.

I propose we integrate this patch so it can have time to get feedback
from the community.

----------
Added file: http://bugs.python.org/file15219/windows symlink draft 15.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Oct 28, 2009, 6:00 PM

Post #24 of 24 (364 views)
Permalink

[issue1578269] Add os.link() and os.symlink() and os.path.islink() support for Windows [In reply to]

Changes by Jason R. Coombs <jaraco [at] jaraco>:

Removed file: http://bugs.python.org/file15214/windows symlink draft 14.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue1578269>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads