Mailing List Archive: Python: Bugs

[issue15206] uuid module falls back to unsuitable RNG

Index | Next | Previous | View Threaded

report at bugs

Jun 27, 2012, 7:01 AM

Post #1 of 16 (148 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG

New submission from Christian Heimes <lists [at] cheimes>:

The uuid module uses Mersenne Twister from the random module as last fallback. However a MT isn't suitable for cryptographic purposes. The module should first try to use os.urandom() and then perhaps use its own instance of random.Random, similar to uuid_generate_* [1]

The problem doesn't apply to most modern platforms as the uuid module uses either libuuid or the Windows API with ctypes. Therefore I consider the real world severity as low. It may not require a backport to Python 2.x.

[1] http://linux.die.net/man/3/uuid_generate

----------
components: Library (Lib)
messages: 164157
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: uuid module falls back to unsuitable RNG
type: security
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 8:56 AM

Post #2 of 16 (138 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Changes by Arfrever Frehtes Taifersar Arahesis <Arfrever.FTA [at] GMail>:

----------
nosy: +Arfrever

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 11:10 AM

Post #3 of 16 (143 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Christian Heimes <lists [at] cheimes> added the comment:

Further analysis:

* uuid1() uses random.randrange() if the system doesn't provide uuid_generate_time

* uuid1() also falls back to random.randrange() in getnode()'s _random_getnode() if no hardware address can be acquired.

* uuid4() is fine as it only falls back to random.randrange() when os.urandom() fails.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 11:29 AM

Post #4 of 16 (143 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

Can you elaborate why it is unsuitable? None of the uuid functions claim any cryptographic properties, so even if MT was unsuitable for cryptographic purposes, this wouldn't rule it out for generating uuids.

----------
nosy: +loewis

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 2:11 PM

Post #5 of 16 (135 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Christian Heimes <lists [at] cheimes> added the comment:

IMHO it's all about managing expectations. As libuuid is using a crypto RNG before it falls back to a less suitable RNG. We should follow this example. I couldn't find any information about the implementation details of Window's UuidCreate().

I agree that we can disagree on my reasoning. However the usage of random.random() and random.randint() in uuid is flawed for a second reason. The default instance random._inst doesn't compensate for fork(). After fork() the two processes share the same random state and thus will create the same uuids. For example tempfile._RandomNameSequence re-creates the RNG when it notices a different PID.

----------
keywords: +patch
Added file: http://bugs.python.org/file26190/issue15206.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 3:22 PM

Post #6 of 16 (135 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Raymond Hettinger <raymond.hettinger [at] gmail> added the comment:

Are uuid's promised to be cryptographically secure?

----------
assignee: -> rhettinger
nosy: +rhettinger

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 3:44 PM

Post #7 of 16 (134 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Christian Heimes <lists [at] cheimes> added the comment:

Not, not by definition. However an uuid generator shall geenerate uuid in a way that make collisions highly improbable. IMHO this verdict implies that an uuid generator should use the cryptographic RNG if available.

The behavior after fork() is clearly a bug as it will generate lots of collisions on systems that fall back to random.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 4:25 PM

Post #8 of 16 (135 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

> However a MT isn't suitable for cryptographic purposes.
> The module should first try to use os.urandom() and
> then perhaps use its own instance of random.Random,
> similar to uuid_generate_* [1]

os.urandom() is not suitable for cryptographic purposes :-) Python 3.3 has also ssl.RAND_bytes() which is better than os.urandom(), but it's not possible (easy?) to build a custom random.Random class with an arbitrary RNG (like os.urandom or ssl.RAND_bytes).

It would be nice to provide an API to choose the best RNG depending on a set of requirements. I wrote the Hasard library which implements such idea: the library provides "profiles" and chooses the best RNG for a profile. Profiles:
- fast
- secure nonblocking
- secure blocking
- hardware

See the doc directory the Hasard project for details:
https://bitbucket.org/haypo/hasard/
https://bitbucket.org/haypo/hasard/src/82d13450c552/doc/profile_list.rst

See also the issue #12858 for another user of a better RNG.

I'm quite sure that all these RNG issues are a good candidate for a PEP because RNG is complex problem, there are different use cases, various implements, and a lot of common issue (in RNG implementations). Handling fork or not is an important question, which impact performances, for example.

See also the issue #12754: "Add alternative random number generators".

----------
nosy: +haypo

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 4:29 PM

Post #9 of 16 (135 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

>From the /dev/urandom Linux man page:

If you are unsure about whether you should use /dev/random or
/dev/urandom, then probably you want to use the latter. As a general
rule, /dev/urandom should be used for everything except long-lived
GPG/SSL/SSH keys.

If a seed file is saved across reboots as recommended below (all major
Linux distributions have done this since 2000 at least), the output is
cryptographically secure against attackers without local root access as
soon as it is reloaded in the boot sequence, and perfectly adequate for
network encryption session keys.

So, yes, /dev/urandom is suitable for most cryptographic purposes (except long-lived private keys).

----------
nosy: +pitrou

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 4:38 PM

Post #10 of 16 (138 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Christian Heimes <lists [at] cheimes> added the comment:

Antoine beat me to it and he is totally right.

Please don't derail this bug report. I agree with your analysis that the RNG core of random.Random subclass can't be replaced easily and that more implementations for different purposes would be great. You should stick the analysis into a different ticket or write a PEP. This ticket is the wrong place. I'll support you if you keep the ticket on course. ;)

Let's concentrate on the topic at hand and discuss if

a) my patch handles the fork() issue correctly
b) if it's a good idea to try SystemRandom first
c) a backport to 2.6, 2.7, 3.1 and 3.2 is required
and perhaps
d) if I should open another ticket to work on a general solution for the RNG + fork() issue.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 10:54 PM

Post #11 of 16 (136 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

> a) my patch handles the fork() issue correctly

If the system has urandom, yes.

> b) if it's a good idea to try SystemRandom first

Certainly.

> c) a backport to 2.6, 2.7, 3.1 and 3.2 is required
> and perhaps

Cannot backport to 2.6 and 3.1; it's not a security issue.

> d) if I should open another ticket to work on a general solution for
> the RNG + fork() issue.

I'm not quite sure what a solution could be, or wether there is
an issue in the first place, so -0.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 10:55 PM

Post #12 of 16 (135 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Changes by Martin v. Löwis <martin [at] v>:

----------
versions: -Python 2.6, Python 3.1

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 28, 2012, 2:05 AM

Post #13 of 16 (134 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Christian Heimes <lists [at] cheimes> added the comment:

Am 28.06.2012 07:54, schrieb Martin v. Löwis:
>
> Martin v. Löwis <martin [at] v> added the comment:
>
>> a) my patch handles the fork() issue correctly
>
> If the system has urandom, yes.

That's the easy and trivial case. It also handles fork() by storing the
PID and comparing it to os.getpid() whenever the RNG is acquired.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 28, 2012, 2:27 AM

Post #14 of 16 (134 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

+ except Exception:

I don't think that's a good idea. You should list the specific exceptions here (NotImplementedError, OSError perhaps?).

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 1, 2012, 8:42 AM

Post #15 of 16 (134 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Christian Heimes <lists [at] cheimes> added the comment:

The rest of the module uses bar excepts. I could change the signature if you insist.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 1, 2012, 11:40 AM

Post #16 of 16 (134 views)
Permalink

[issue15206] uuid module falls back to unsuitable RNG [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

> The rest of the module uses bar excepts.

It was probably written in prehistoric times :)
The other excepts can be converted later, if the module gets other changes. I don't think it is a deliberate style choice (it would be particularly distasteful :-)).

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15206>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads