Mailing List Archive: Python: Bugs

[issue2804] Integer right shift raises OverflowError when second operand is large

Index | Next | Previous | View Threaded

report at bugs

May 9, 2008, 11:17 AM

Post #1 of 11 (269 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large

New submission from Mark Dickinson <dickinsm[at]gmail.com>:

In Python 2.6a3:

>>> 1 >> (2**31) # unexpected exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int
>>> 1 >> (2**31-1) # works as expected
0L

It might make more sense for the first expression to return 0 instead.
Similarly, perhaps 0 << (2**31) should also return 0.

----------
components: Interpreter Core
messages: 66484
nosy: marketdickinson
severity: normal
status: open
title: Integer right shift raises OverflowError when second operand is large
type: behavior
versions: Python 2.6, Python 3.0

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 1:38 PM

Post #2 of 11 (265 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Alexander Belopolsky <belopolsky[at]users.sourceforge.net> added the comment:

Attached patch makes a >> b return 0 for b > maxsize.

----------
keywords: +patch
nosy: +belopolsky
Added file: http://bugs.python.org/file10234/issue2804.diff

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 2:01 PM

Post #3 of 11 (265 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Alexander Belopolsky <belopolsky[at]users.sourceforge.net> added the comment:

On the second thought, it is theoretically possible to have a long int a
for which a >> maxsize is not zero because that would require only
maxsize/15 digits. I am not sure it is worth the trouble to account for
that.

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 2:15 PM

Post #4 of 11 (265 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Mark Dickinson <dickinsm[at]gmail.com> added the comment:

> On the second thought, it is theoretically possible to have a long int
a
> for which a >> maxsize is not zero

This occurred to me too; somehow this hardly seems worth worrying about-
--there are already other problems in longobject.c when integers get
very large, and I don't think one more is going to hurt. Perhaps a
warning somewhere in the documentation would be appropriate.

(It seems to me that if anyone really cares, the *right* thing to do is
to limit all integers to be at most maxsize bits long.)

Thanks for the patch! I'll take a look.

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 4:08 PM

Post #5 of 11 (263 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Martin v. Löwis <martin[at]v.loewis.de> added the comment:

I'm -1 on this patch (or any other fixing the perceived problem).
Special cases aren't special enough to break the rules.

If you ever see this error in a real application, you have deeper
problems than the exception.

----------
nosy: +loewis

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 4:37 PM

Post #6 of 11 (260 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Mark Dickinson <dickinsm[at]gmail.com> added the comment:

I'm not sure I understand. What rule would be broken by this patch?

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 5:13 PM

Post #7 of 11 (260 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Martin v. Löwis <martin[at]v.loewis.de> added the comment:

> I'm not sure I understand. What rule would be broken by this patch?

The (implicit) rule that there is a range limitation on the shift width.
The limitation would continue to exist, just not for selected left-hand
parameters. This is fairly arbitrary.

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 5:27 PM

Post #8 of 11 (260 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Alexander Belopolsky <belopolsky[at]users.sourceforge.net> added the comment:

On Fri, May 9, 2008 at 7:08 PM, Martin v. Löwis <report[at]bugs.python.org> wrote:
..
> I'm -1 on this patch (or any other fixing the perceived problem).
> Special cases aren't special enough to break the rules.
>
> If you ever see this error in a real application, you have deeper
> problems than the exception.

That was my first reaction as well, but then I thought that it was
easy to fix because the answer is guaranteed to be 0 for ridiculously
large right shifts. However, since the patch is not theoretically
correct, I would only be +0 on applying it.

I also note that the current requirement is that shift fits long
rather than Py_ssize_t, which may lead to a >> sys.maxsize being
invalid on platforms where sizeof(long) < sizeof(Py_ssize_t). This
may be a problem because sys.maxsize is likely to be used as a
placeholder for an arbitrary large number.

This said, I sill don't feel strongly one way or another.

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 5:49 PM

Post #9 of 11 (260 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Mark Dickinson <dickinsm[at]gmail.com> added the comment:

It's the current behaviour that seems arbitrary to me: a >> b is well-
defined, and efficiently computable, for any integer a and nonnegative
integer b; it seems odd to have an unnecessary and (from a user's
viewpoint) arbitrary limit on the size of b. It's simple to have the
operation always succeed, and this seems to me like both the least
surprising, and the most correct, thing to do.

On the other hand, I can't really think of any real-world uses for large
shifts: I'm guessing that in applications a shift of more than 63 or so
must be rare. So it probably doesn't matter that much one way or
another.

Changing the limit on the shift to sys.maxsize rather than LONG_MAX
would seem to make some sense, though.

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 9, 2008, 11:51 PM

Post #10 of 11 (254 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Martin v. Löwis <martin[at]v.loewis.de> added the comment:

I still don't think the improvement in observable behavior is worth the
cost of additional code (and as Alexander observes, the patch in
file10234 is incorrect in boundary cases).

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

May 10, 2008, 5:35 AM

Post #11 of 11 (244 views)
Permalink

[issue2804] Integer right shift raises OverflowError when second operand is large [In reply to]

Mark Dickinson <dickinsm[at]gmail.com> added the comment:

Fair enough. Closing as won't fix.

----------
resolution: -> wont fix
status: open -> closed

__________________________________
Tracker <report[at]bugs.python.org>
<http://bugs.python.org/issue2804>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com