Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue14339] Optimizing bin, oct and hex

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Mar 31, 2012, 3:54 PM

Post #1 of 21 (214 views)
Permalink
[issue14339] Optimizing bin, oct and hex

Antoine Pitrou <pitrou [at] free> added the comment:

While optimizing stuff is nice, I'm not sure there's a use case here. Does it significantly speed up a real-world workload you're having?

----------
nosy: +pitrou

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 1, 2012, 12:10 AM

Post #2 of 21 (203 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Serhiy Storchaka <storchaka [at] gmail> added the comment:

No, I do not think that any application will significantly speeded up.

In fact, it is not optimization, and getting rid of the apparent pessimization. In addition to increasing the speed, memory fragmentation is reduced.

The patch has a minimum size, the new code is not larger and not more complex than the old one, it is similar to code already used in this file, there are small nonconditional advantages. I see no reason not to accept the changes.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 13, 2012, 9:28 PM

Post #3 of 21 (199 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Changes by Ezio Melotti <ezio.melotti [at] gmail>:


----------
nosy: +ezio.melotti

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 15, 2012, 11:48 AM

Post #4 of 21 (191 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

A few comments:

(1) The patch appears to assume that a Unicode string created with PyUnicode_New(size, 127) will have 'kind' PyUnicode_1BYTE_KIND. While this might be true in the current implementation, I don't know whether this is guaranteed in general. Martin, any comments on this?

(2) The patch doesn't compile with '--with-pydebug': there's a reference to the (now) undefined variable 'buffer' in one of the asserts.

(3) The overflow check looks as though it needs to be reworked.

----------
nosy: +loewis

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 15, 2012, 1:32 PM

Post #5 of 21 (191 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Serhiy Storchaka <storchaka [at] gmail> added the comment:

> (1) The patch appears to assume that a Unicode string created with PyUnicode_New(size, 127) will have 'kind' PyUnicode_1BYTE_KIND. While this might be true in the current implementation, I don't know whether this is guaranteed in general. Martin, any comments on this?

The same assumption is used above in long_to_decimal_string(). I added
the same assert and changed maxchar to 'f'.

> (2) The patch doesn't compile with '--with-pydebug': there's a reference to the (now) undefined variable 'buffer' in one of the asserts.

Fixed.

> (3) The overflow check looks as though it needs to be reworked.

I was left an old constraint (it is enough), but it really can be
weakened (in fact, it can be so weakened in the current code).

Thank you for the comments and the found error.

----------
Added file: http://bugs.python.org/file25227/long_to_binary_base_2.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
Attachments: long_to_binary_base_2.patch (3.00 KB)


report at bugs

Apr 16, 2012, 12:14 AM

Post #6 of 21 (193 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

Thanks for the updates.


> The same assumption is used above in long_to_decimal_string(). I added
> the same assert and changed maxchar to 'f'.

I think 'x' would be more appropriate. :-)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 16, 2012, 2:02 AM

Post #7 of 21 (190 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Serhiy Storchaka <storchaka [at] gmail> added the comment:

> I think 'x' would be more appropriate. :-)

Oops. You are right, of cause.

----------
Added file: http://bugs.python.org/file25234/long_to_binary_base_3.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
Attachments: long_to_binary_base_3.patch (3.00 KB)


report at bugs

Apr 16, 2012, 6:25 AM

Post #8 of 21 (192 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

> (1) The patch appears to assume that a Unicode string created with
> PyUnicode_New(size, 127) will have 'kind' PyUnicode_1BYTE_KIND.
> While this might be true in the current implementation, I don't know
> whether this is guaranteed in general. Martin, any comments on
> this?

There is a guarantee that the shortest form must always be used.
So as long as the Unicode representation doesn't fundamentally change
again, this property is indeed guaranteed.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 16, 2012, 6:38 AM

Post #9 of 21 (196 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

> There is a guarantee that the shortest form must always be used.

Okay, sounds good. Thanks.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 17, 2012, 4:16 AM

Post #10 of 21 (189 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Changes by Mark Dickinson <dickinsm [at] gmail>:


----------
assignee: -> mark.dickinson

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 20, 2012, 1:21 PM

Post #11 of 21 (191 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset dcd3344b6d5b by Mark Dickinson in branch 'default':
Issue #14339: Improve speed of bin, oct and hex builtins. Patch by Serhiy Storchaka (with minor modifications).
http://hg.python.org/cpython/rev/dcd3344b6d5b

----------
nosy: +python-dev

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 20, 2012, 1:23 PM

Post #12 of 21 (188 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

Thanks for the patch. I made some minor changes, notably moving the overflow check closer to where it's needed, moving some comments around, and removing a (possibly inappropriate) PyErr_NoMemory call.

----------
resolution: -> fixed
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 20, 2012, 1:41 PM

Post #13 of 21 (189 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

This should have waited until Serhiy submits a contributor form. Serhiy, can you please do this soon? Else I'll have to revert the change.

----------
status: closed -> open

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 20, 2012, 1:49 PM

Post #14 of 21 (187 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

Aargh. Sorry, yes. Serhiy, can you do this? The form can be found at:

http://www.python.org/psf/contrib/contrib-form-python/

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 20, 2012, 1:52 PM

Post #15 of 21 (190 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

Whoops; that looks like a slightly older version of the form. I think the correct one is:

http://www.python.org/psf/contrib/contrib-form/

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 23, 2012, 2:00 PM

Post #16 of 21 (178 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Serhiy Storchaka <storchaka [at] gmail> added the comment:

I can only do this tomorrow.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 24, 2012, 3:06 PM

Post #17 of 21 (172 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

2.100 - v = PyUnicode_DecodeASCII(p, &buffer[sz] - p, NULL);
...
2.104 + assert(p == PyUnicode_1BYTE_DATA(v));
2.105 return v;
2.106 }

You may call assert(_PyUnicode_CheckConsistency(v, 1)) to ensure that the newly created string is "consistent" (see the function for the details).

----------
nosy: +haypo

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 26, 2012, 11:33 PM

Post #18 of 21 (173 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

Serhiy: what's the status of your contributor form? Note that you can also fax it, or scan it and send it by email.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 27, 2012, 4:19 AM

Post #19 of 21 (172 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

> You may call assert(_PyUnicode_CheckConsistency(v, 1)) to ensure
> that the newly created string is "consistent" (see the function
> for the details).

Done in the following changeset:

changeset: 76560:3bdcf0cab164
parent: 76558:5fea362b92fc
user: Victor Stinner <victor.stinner [at] gmail>
date: Thu Apr 26 00:37:21 2012 +0200
files: Objects/longobject.c
description:
long_to_decimal_string() and _PyLong_Format() check the consistency of newly
created strings using _PyUnicode_CheckConsistency() in debug mode

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 27, 2012, 7:52 AM

Post #20 of 21 (172 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Serhiy Storchaka <storchaka [at] gmail> added the comment:

I sent the signed form. Martin, sorry for the delay. Mark, sorry, that
involuntarily let you down.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 27, 2012, 2:51 PM

Post #21 of 21 (171 views)
Permalink
[issue14339] Optimizing bin, oct and hex [In reply to]

Mark Dickinson <dickinsm [at] gmail> added the comment:

Serhiy: thanks for submitting the form! No need to apologise: it was my bad for making the commit prematurely.

And now that I see that all-important asterisk next to your name, I'll reclose the issue. :-)

----------
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.