Mailing List Archive: Python: Dev

Add os.path.resolve to simplify the use of os.readlink

Index | Next | Previous | View Threaded

armin.ronacher at active-4

Jun 21, 2012, 3:23 AM

Post #1 of 12 (515 views)
Permalink

Add os.path.resolve to simplify the use of os.readlink

Due to an user error on my part I was not using os.readlink correctly.
Since links can be relative to their location I think it would make sense
to provide an os.path.resolve helper that automatically returns the
absolute path:

def resolve(filename):
try:
target = os.readlink(filename)
except OSError as e:
if e.errno == errno.EINVAL:
return abspath(filename)
raise
return normpath(join(dirname(filename), target))

The above implementation also does not fail if an entity exists but is not
a link and just returns the absolute path of the given filename in that
case.

Regards,
Armin

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

lists at cheimes

Jun 21, 2012, 3:52 AM

Post #2 of 12 (509 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

Am 21.06.2012 12:23, schrieb Armin Ronacher:
> Due to an user error on my part I was not using os.readlink correctly.
> Since links can be relative to their location I think it would make sense
> to provide an os.path.resolve helper that automatically returns the
> absolute path:
>
> def resolve(filename):
> try:
> target = os.readlink(filename)
> except OSError as e:
> if e.errno == errno.EINVAL:
> return abspath(filename)
> raise
> return normpath(join(dirname(filename), target))
>
> The above implementation also does not fail if an entity exists but is not
> a link and just returns the absolute path of the given filename in that
> case.

+1

Does the code handle a chain of absolute and relative symlinks
correctly, for example a relative symlink that points to another
relative symlink in a different directory that points to a file in a
third directry?

Christian
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

armin.ronacher at active-4

Jun 21, 2012, 4:10 AM

Post #3 of 12 (514 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

Hi,

> Am 21.06.2012 12:23, schrieb Armin Ronacher:
> Does the code handle a chain of absolute and relative symlinks
> correctly, for example a relative symlink that points to another
> relative symlink in a different directory that points to a file in a
> third directry?
No, but that's a good point. It should attempt to resolve these in a loop
until it either loops too often (would have to check the POSIX spec for a
reasonable value) or until it terminates by finding an actual file or
directory.

Regards,
Armin

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Jun 21, 2012, 4:18 AM

Post #4 of 12 (505 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On Thu, 21 Jun 2012 11:10:44 -0000
"Armin Ronacher" <armin.ronacher [at] active-4> wrote:
> Hi,
>
> > Am 21.06.2012 12:23, schrieb Armin Ronacher:
> > Does the code handle a chain of absolute and relative symlinks
> > correctly, for example a relative symlink that points to another
> > relative symlink in a different directory that points to a file in a
> > third directry?
> No, but that's a good point. It should attempt to resolve these in a loop
> until it either loops too often (would have to check the POSIX spec for a
> reasonable value) or until it terminates by finding an actual file or
> directory.

You could take a look at the resolve() algorithm in pathlib:
http://pypi.python.org/pypi/pathlib/

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

lists at cheimes

Jun 21, 2012, 4:26 AM

Post #5 of 12 (504 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

Am 21.06.2012 13:10, schrieb Armin Ronacher:
Hello Armin,

> No, but that's a good point. It should attempt to resolve these in a loop
> until it either loops too often (would have to check the POSIX spec for a
> reasonable value) or until it terminates by finding an actual file or
> directory.

The specs mention sysconf(SYMLOOP_MAX) / _POSIX_SYMLOOP_MAX for the
maximum count of lookups. The limit is lower than I expected. On my
system it's defined as 8 in
/usr/include/x86_64-linux-gnu/bits/posix1_lim.h. The limit would also
handle self referencing loops correctly.

BTW Is there a better way than raise OSError(errno.ELOOP,
os.strerror(errno.ELOOP), filename) to raise a correct OSError with
errno, errno message and filename? A classmethod like
"OSError.from_errno(errno, filename=None) -> proper subclass auf OSError
with sterror() set" would reduce the burden for developers. PEP mentions
the a similar idea at
http://www.python.org/dev/peps/pep-3151/#implementation but this was
never implemented.

Christian
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

phd at phdru

Jun 21, 2012, 4:34 AM

Post #6 of 12 (502 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On Thu, Jun 21, 2012 at 11:10:44AM -0000, Armin Ronacher <armin.ronacher [at] active-4> wrote:
> would have to check the POSIX spec for a
> reasonable value

POSIX allows 8 links:

http://infohost.nmt.edu/~eweiss/222_book/222_book/0201433079/ch02lev1sec5.html

_POSIX_SYMLOOP_MAX - number of symbolic links that can be traversed
during pathname resolution: 8

The constant _POSIX_SYMLOOP_MAX from unistd.h:

#define _POSIX_SYMLOOP_MAX 8

Oleg.
--
Oleg Broytman http://phdru.name/ phd [at] phdru
Programmers don't die, they just GOSUB without RETURN.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ncoghlan at gmail

Jun 21, 2012, 5:55 AM

Post #7 of 12 (502 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On Thu, Jun 21, 2012 at 9:26 PM, Christian Heimes <lists [at] cheimes> wrote:
> BTW Is there a better way than raise OSError(errno.ELOOP,
> os.strerror(errno.ELOOP), filename) to raise a correct OSError with
> errno, errno message and filename? A classmethod like
> "OSError.from_errno(errno, filename=None) -> proper subclass auf OSError
> with sterror() set" would reduce the burden for developers. PEP mentions
> the a similar idea at
> http://www.python.org/dev/peps/pep-3151/#implementation but this was
> never implemented.

According to the C code, it should be working at least for recognised
errno values:

http://hg.python.org/cpython/file/009ac63759e9/Objects/exceptions.c#l890

I can't get it to trigger properly in my local build, though :(

Cheers,
Nick.

--
Nick Coghlan�� |�� ncoghlan [at] gmail�� |�� Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

lists at cheimes

Jun 21, 2012, 6:04 AM

Post #8 of 12 (505 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

Am 21.06.2012 14:55, schrieb Nick Coghlan:
> On Thu, Jun 21, 2012 at 9:26 PM, Christian Heimes <lists [at] cheimes> wrote:
>> BTW Is there a better way than raise OSError(errno.ELOOP,
>> os.strerror(errno.ELOOP), filename) to raise a correct OSError with
>> errno, errno message and filename? A classmethod like
>> "OSError.from_errno(errno, filename=None) -> proper subclass auf OSError
>> with sterror() set" would reduce the burden for developers. PEP mentions
>> the a similar idea at
>> http://www.python.org/dev/peps/pep-3151/#implementation but this was
>> never implemented.
>
> According to the C code, it should be working at least for recognised
> errno values:
>
> http://hg.python.org/cpython/file/009ac63759e9/Objects/exceptions.c#l890
>
> I can't get it to trigger properly in my local build, though :(

Me neither with the one argument variant:

Python 3.3.0a4+ (default:c3616595dada+, Jun 19 2012, 23:12:25)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import errno
[73872 refs]
>>> type(OSError(errno.ENOENT))
<class 'OSError'>
[73877 refs]

It works work two arguments but it doesn't set strerror and filename
correctly:

>>> exc = OSError(errno.ENOENT, "filename")
[73948 refs]
>>> exc
FileNotFoundError(2, 'filename')
[73914 refs]
>>> exc.strerror
'filename'
[73914 refs]
>>> exc.filename
[73914 refs]

OSError doesn't accept keyword args:

>>> OSError(errno.ENOENT, filename="filename")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: OSError does not take keyword arguments

How about adding keyword support to OSError and derive the strerror from
errno if the second argument is not given?

Christian
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Jun 21, 2012, 6:16 AM

Post #9 of 12 (507 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On Thu, 21 Jun 2012 15:04:17 +0200
Christian Heimes <lists [at] cheimes> wrote:
>
> How about adding keyword support to OSError and derive the strerror from
> errno if the second argument is not given?

That's not the original behaviour:

Python 3.2.2+ (3.2:9ef20fbd340f, Oct 15 2011, 21:22:07)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> e = OSError(5)
>>> e.errno
>>> e.strerror
>>> str(e)
'5'

I don't mind making this particular compatibility-breaking change,
though.

Regards

Antoine.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

vandry at TZoNE

Jun 21, 2012, 6:46 AM

Post #10 of 12 (503 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On 2012-06-21 06:23, Armin Ronacher wrote:
> Due to an user error on my part I was not using os.readlink correctly.
> Since links can be relative to their location I think it would make sense
> to provide an os.path.resolve helper that automatically returns the
> absolute path:
>
> def resolve(filename):
> try:
> target = os.readlink(filename)
> except OSError as e:
> if e.errno == errno.EINVAL:
> return abspath(filename)
> raise
> return normpath(join(dirname(filename), target))
>
> The above implementation also does not fail if an entity exists but is not
> a link and just returns the absolute path of the given filename in that
> case.

It's expensive (not to mention racy) to do this correctly, when any
component of the pathname (not just the component after the last slash)
might be a symlink. For example:

mkdir -p foo1/foo2
touch bar
ln -s ../../bar foo1/foo2/symlink
ln -s foo1/foo2 foo

Now try to resolve "foo/symlink" using your function. It produces
"../bar", which doesn't exist.

Why not just work with the pathname you're given and let the kernel
worry about resolving it?

-Phil
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ncoghlan at gmail

Jun 21, 2012, 7:12 AM

Post #11 of 12 (503 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On Thu, Jun 21, 2012 at 11:16 PM, Antoine Pitrou <solipsis [at] pitrou> wrote:
> On Thu, 21 Jun 2012 15:04:17 +0200
> Christian Heimes <lists [at] cheimes> wrote:
>>
>> How about adding keyword support to OSError and derive the strerror from
>> errno if the second argument is not given?
>
> That's not the original behaviour:
>
> Python 3.2.2+ (3.2:9ef20fbd340f, Oct 15 2011, 21:22:07)
> [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> e = OSError(5)
>>>> e.errno
>>>> e.strerror
>>>> str(e)
> '5'
>
>
> I don't mind making this particular compatibility-breaking change,
> though.

+1 from me. Existing code that just passes errno will now get strerror
set automatically, and existing code *can't* just be passing the errno
and filename, since OSError doesn't yet support keyword arguments.

Cheers,
Nick.

--
Nick Coghlan�� |�� ncoghlan [at] gmail�� |�� Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Jun 21, 2012, 7:48 AM

Post #12 of 12 (503 views)
Permalink

Re: Add os.path.resolve to simplify the use of os.readlink [In reply to]

On Thu, 21 Jun 2012 10:23:25 -0000
"Armin Ronacher" <armin.ronacher [at] active-4> wrote:
> Due to an user error on my part I was not using os.readlink correctly.
> Since links can be relative to their location I think it would make sense
> to provide an os.path.resolve helper that automatically returns the
> absolute path:
>
> def resolve(filename):
> try:
> target = os.readlink(filename)
> except OSError as e:
> if e.errno == errno.EINVAL:
> return abspath(filename)
> raise
> return normpath(join(dirname(filename), target))

Note that abspath() is buggy in the face of symlinks, for example it
will happily collapse /etc/foo/../bar into /etc/bar, even
though /etc/foo might be a link to /usr/lib/foo

The only safe way to collapse ".." elements is to resolve symlinks.

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads