Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue3001] RLock's are SLOW

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Nov 5, 2009, 4:57 PM

Post #1 of 12 (97 views)
Permalink
[issue3001] RLock's are SLOW

Changes by Antoine Pitrou <pitrou[at]free.fr>:


----------
assignee: -> pitrou
components: -Interpreter Core
stage: patch review -> needs patch
versions: +Python 3.2 -Python 2.7, Python 3.1

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 6, 2009, 5:07 PM

Post #2 of 12 (86 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Antoine Pitrou <pitrou[at]free.fr> added the comment:

Here is a new patch against py3k. It passes all tests and is generally
10-15x faster than the pure Python version.

----------
Added file: http://bugs.python.org/file15280/rlock2.patch

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 6, 2009, 5:13 PM

Post #3 of 12 (86 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Changes by Antoine Pitrou <pitrou[at]free.fr>:


----------
stage: needs patch -> patch review

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 6, 2009, 11:48 PM

Post #4 of 12 (86 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Gregory P. Smith <greg[at]krypto.org> added the comment:

Reviewers: ,

http://codereview.appspot.com/150055/diff/1/4
File Modules/_threadmodule.c (right):

http://codereview.appspot.com/150055/diff/1/4#newcode221
Modules/_threadmodule.c:221: return PyBool_FromLong((long) r);
This explicit (long) cast is unnecessary.

http://codereview.appspot.com/150055/diff/1/4#newcode246
Modules/_threadmodule.c:246: PyThread_release_lock(self->rlock_lock);
reset self->rlock_owner to 0 before releasing the lock.

Description:
code review for http://bugs.python.org/issue3001

Please review this at http://codereview.appspot.com/150055

Affected files:
M Lib/test/test_threading.py
M Lib/threading.py
M Modules/_threadmodule.c

----------

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 7, 2009, 12:57 AM

Post #5 of 12 (86 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

STINNER Victor <victor.stinner[at]haypocalc.com> added the comment:

rlock_acquire_doc: "(...) return None once the lock is acquired".
That's wrong, acquire() always return a boolean (True or False).

rlock_release(): Is the assert(self->rlock_count > 0); really required?
You're checking its value few lines before.

rlock_acquire_restore(): raise an error if the lock acquire failed,
whereas rlock_acquire() doesn't. Why not raising an error in
rlock_acquire() (especially if blocking is True)? Or if the error can
never occur, remove the error checking in rlock_acquire_restore().

rlock_acquire_restore(): (maybe) set owner to 0 if count is 0.

rlock_release_save(): may also reset self->rlock_owner to 0, as
rlock_release().

rlock_new(): why not reusing type instead of Py_TYPE(self) in
"Py_TYPE(self)->tp_free(self)"?

__exit__: rlock_release() is defined as __exit__() with METH_VARARGS,
but it has no argument. Can it be a problem?

Anything, thanks for the faster RLock!

If your patch is commited, you can reconsider #3618 (possible deadlock
in python IO implementation).

----------

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 7, 2009, 7:06 AM

Post #6 of 12 (85 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Antoine Pitrou <pitrou[at]free.fr> added the comment:

Thanks for the review. I will make the suggested modifications.

http://codereview.appspot.com/150055/diff/1/4
File Modules/_threadmodule.c (right):

http://codereview.appspot.com/150055/diff/1/4#newcode221
Modules/_threadmodule.c:221: return PyBool_FromLong((long) r);
On 2009/11/07 07:48:05, gregory.p.smith wrote:
> This explicit (long) cast is unnecessary.

Right.

http://codereview.appspot.com/150055/diff/1/4#newcode246
Modules/_threadmodule.c:246: PyThread_release_lock(self->rlock_lock);
On 2009/11/07 07:48:05, gregory.p.smith wrote:
> reset self->rlock_owner to 0 before releasing the lock.

We always check rlock_count before rlock_owner anyway but, yes, I could
reset rlock_owner out of safety.

http://codereview.appspot.com/150055

----------

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 7, 2009, 7:17 AM

Post #7 of 12 (85 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Antoine Pitrou <pitrou[at]free.fr> added the comment:

> rlock_acquire_doc: "(...) return None once the lock is acquired".
> That's wrong, acquire() always return a boolean (True or False).

You're right, I should fix that docstring.

> rlock_release(): Is the assert(self->rlock_count > 0); really required?
> You're checking its value few lines before.

Right again :) I forgot this was unsigned.

> rlock_acquire_restore(): raise an error if the lock acquire failed,
> whereas rlock_acquire() doesn't. Why not raising an error in
> rlock_acquire() (especially if blocking is True)?

For rlock_acquire(), I mimicked what the Python code (_PyRLock.acquire)
does: if locking fails, it returns False instead. It is part of the API.

(and I agree this is not necessarily right, because failing to lock if
blocking is True would probably indicate a low-level system error, but
the purpose of the patch is not to change the API)

But you're right that the Python version of rlock_acquire_restore()
doesn't check the return code either, so I may remove this check from
the C code, although the rest of the method clearly assumes the lock has
been taken.

> rlock_acquire_restore(): (maybe) set owner to 0 if count is 0.
>
> rlock_release_save(): may also reset self->rlock_owner to 0, as
> rlock_release().

As I said to Gregory, the current code doesn't rely on rlock_owner to be
reset but, yes, we could still add it out of safety.

> rlock_new(): why not reusing type instead of Py_TYPE(self) in
> "Py_TYPE(self)->tp_free(self)"?

Good point.

> __exit__: rlock_release() is defined as __exit__() with METH_VARARGS,
> but it has no argument. Can it be a problem?

I just mimicked the corresponding declarations in the non-recursive lock
object. Apparently it's safe on all architectures Python has been
running on for years, although I agree it looks strange.

> If your patch is commited, you can reconsider #3618 (possible deadlock
> in python IO implementation).

Indeed.

Thanks !

----------

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 7, 2009, 9:47 AM

Post #8 of 12 (84 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Antoine Pitrou <pitrou[at]free.fr> added the comment:

Here is an updated patch. I addressed all review comments, except the
one about acquire_restore() checking the return result of acquire(),
because I think it's really a weakness in the Python implementation.

----------
Added file: http://bugs.python.org/file15284/rlock3.patch

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 7, 2009, 11:44 AM

Post #9 of 12 (84 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Gregory P. Smith <greg[at]krypto.org> added the comment:

Can you make the C implementation's repr() show something similar to the
Python implementation?

----------

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 7, 2009, 12:12 PM

Post #10 of 12 (84 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Antoine Pitrou <pitrou[at]free.fr> added the comment:

Yes, here is a new patch adding tp_repr.

----------
Added file: http://bugs.python.org/file15285/rlock4.patch

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 8, 2009, 6:10 AM

Post #11 of 12 (82 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

STINNER Victor <victor.stinner[at]haypocalc.com> added the comment:

rlock4.patch looks correct and pass test_threading.py tests.

----------

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Nov 10, 2009, 10:52 AM

Post #12 of 12 (64 views)
Permalink
[issue3001] RLock's are SLOW [In reply to]

Antoine Pitrou <pitrou[at]free.fr> added the comment:

I've committed the latest patch in r76189. Thanks for the reviews, everyone.

----------
resolution: -> fixed
status: open -> closed

_______________________________________
Python tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue3001>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.