Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

slice subscripts for sequences and mappings

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


eliben at gmail

Mar 3, 2012, 12:36 AM

Post #1 of 18 (955 views)
Permalink
slice subscripts for sequences and mappings

Hello,

I find a strange discrepancy in Python with regards to slice
subscripting of objects, at the C API level. I mean things like
obj[start:end:step].

I'd expect slice subscripts to be part of the sequence interface, and
yet they are not. In fact, they are part of the mapping interface. For
example, the list object has its slice get/set methods assigned to a
PyMappingMethods struct. So does a bytes object, and pretty much every
other object that wants to support subscripts.

This doesn't align well with the documentation, in at least two places.

1) The library documentation
(http://docs.python.org/dev/library/stdtypes.html) in 4.8 says:

††† "Mappings are mutable objects. There is currently only one
standard mapping type, the dictionary"

Why then does a list implement the mapping interface? Moreover, why
does bytes, an immutable object, implement the mapping interface?

2) The same documentation page in 4.6 says, in the operation table:

s[i:j] slice of s from i to j
s[i:j:k] slice of s from i to j with step k

But in the implementation, the slice subscripts are part of the
mapping, not the sequence inteface.

The PySequenceMethods structure does have fields for slice accessors,
but their naming (was_sq_slice, was_sq_ass_slice) suggests they're
just deprecated placeholders.

This also doesn't align well with logic, since mappings like dict have
no real meaning for slice subscripts. These logically belong to a
sequence. Moreover, it takes subscripts for single a single numeric
index away from subscripts for a slice into a different protocol (the
former in sequence, the latter in mapping).

I realize I must be missing some piece of the history here and not
suggesting to change anything. I do think that the documentation,
especially in the area of the type object that defines the sequence
and mapping protocols, could be clarified to express what is expected
of a new type that wants to act as a sequence. In particular, it
should be said explicitly that such a type must implement the mapping
protocol if it wants slice subscripting.

If this makes any sense at all, I will open an issue.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


eliben at gmail

Mar 3, 2012, 12:50 AM

Post #2 of 18 (933 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

> This doesn't align well with the documentation, in at least two places.
>
<snip>

Another place is in http://docs.python.org/dev/reference/datamodel.html:

"
object.__getitem__(self, key)

Called to implement evaluation of self[key]. For sequence types,
the accepted keys should be integers and slice objects. [...]
"

Once again, at the C API level this isn't accurate since only integer
keys are handled by the sequence protocol, leaving slice keys to the
mapping protocol.

The datamodel doc should stay as it is, because it's correct for
Python-written classes. But the relevant C API sections really need
some clarification.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


stefan_ml at behnel

Mar 3, 2012, 1:24 AM

Post #3 of 18 (935 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

Eli Bendersky, 03.03.2012 09:36:
> I find a strange discrepancy in Python with regards to slice
> subscripting of objects, at the C API level. I mean things like
> obj[start:end:step].
>
> I'd expect slice subscripts to be part of the sequence interface, and
> yet they are not. In fact, they are part of the mapping interface. For
> example, the list object has its slice get/set methods assigned to a
> PyMappingMethods struct. So does a bytes object, and pretty much every
> other object that wants to support subscripts.
>
> This doesn't align well with the documentation, in at least two places.
>
> 1) The library documentation
> (http://docs.python.org/dev/library/stdtypes.html) in 4.8 says:
>
> "Mappings are mutable objects. There is currently only one
> standard mapping type, the dictionary"
>
> Why then does a list implement the mapping interface? Moreover, why
> does bytes, an immutable object, implement the mapping interface?

I think that's (partly?) for historical reasons. Originally, there were the
slicing functions as part of the sequence interface. They took a start and
an end index of the slice. Then, extended slicing was added to the
language, and that used a slice object, which didn't fit into the sequence
slicing interface. So the interface was unified using the existing mapping
getitem interface, and the sequence slicing functions were eventually
deprecated and removed in Py3.

Stefan

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


eliben at gmail

Mar 3, 2012, 4:07 AM

Post #4 of 18 (939 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, Mar 3, 2012 at 11:24, Stefan Behnel <stefan_ml [at] behnel> wrote:
> Eli Bendersky, 03.03.2012 09:36:
>> I find a strange discrepancy in Python with regards to slice
>> subscripting of objects, at the C API level. I mean things like
>> obj[start:end:step].
>>
>> I'd expect slice subscripts to be part of the sequence interface, and
>> yet they are not. In fact, they are part of the mapping interface. For
>> example, the list object has its slice get/set methods assigned to a
>> PyMappingMethods struct. So does a bytes object, and pretty much every
>> other object that wants to support subscripts.
>>
>> This doesn't align well with the documentation, in at least two places.
>>
>> 1) The library documentation
>> (http://docs.python.org/dev/library/stdtypes.html) in 4.8 says:
>>
>> † † "Mappings are mutable objects. There is currently only one
>> standard mapping type, the dictionary"
>>
>> Why then does a list implement the mapping interface? Moreover, why
>> does bytes, an immutable object, implement the mapping interface?
>
> I think that's (partly?) for historical reasons. Originally, there were the
> slicing functions as part of the sequence interface. They took a start and
> an end index of the slice. Then, extended slicing was added to the
> language, and that used a slice object, which didn't fit into the sequence
> slicing interface. So the interface was unified using the existing mapping
> getitem interface, and the sequence slicing functions were eventually
> deprecated and removed in Py3.

This make sense. Not that now there's also duplication in almost all
objects because the mapping protocol essentially supersedes the
sequence protocol for accessing elements. I.e. sq_item and sq_ass_item
are no longer needed if an object implements the mapping protocol,
because the mapping interface has precedence, and mp_subscript &
mp_ass_subscript are called instead, respectively. Because of that,
the first thing they do is check whether the index is a simple number
and do the work of their sequence protocol cousins. This duplicates
code in almost all objects that need to support __getitem__.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


solipsis at pitrou

Mar 3, 2012, 4:20 AM

Post #5 of 18 (926 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

Hi,

> I'd expect slice subscripts to be part of the sequence interface, and
> yet they are not. In fact, they are part of the mapping interface. For
> example, the list object has its slice get/set methods assigned to a
> PyMappingMethods struct. So does a bytes object, and pretty much every
> other object that wants to support subscripts.

It comes from:
http://hg.python.org/cpython/rev/245224d1b8c9
http://bugs.python.org/issue400998

Written by Michael Hudson and reviewed by Guido.
I wonder why this patch chose to add mapping protocol support to tuples
and lists, rather than add a tp_ slot for extended slicing.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


eliben at gmail

Mar 3, 2012, 4:41 AM

Post #6 of 18 (925 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

>> I'd expect slice subscripts to be part of the sequence interface, and
>> yet they are not. In fact, they are part of the mapping interface. For
>> example, the list object has its slice get/set methods assigned to a
>> PyMappingMethods struct. So does a bytes object, and pretty much every
>> other object that wants to support subscripts.
>
> It comes from:
> http://hg.python.org/cpython/rev/245224d1b8c9
> http://bugs.python.org/issue400998
>
> Written by Michael Hudson and reviewed by Guido.
> I wonder why this patch chose to add mapping protocol support to tuples
> and lists, rather than add a tp_ slot for extended slicing.
>

Why a separate tp_ slot for extended slicing? ISTM slicing pertains to
sequences, similarly to other numeric indices. If you look at
PySequenceMethods it has these (apparently no longer used fields):

void *was_sq_slice;
void *was_sq_ass_slice;

These were "simple" slices (pairs of numbers). I suppose if any change
is considered, these fields can be re-incarnated to accept PyObject*
slices similarly to the current mp_subscript and mp_ass_subscript.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


solipsis at pitrou

Mar 3, 2012, 4:48 AM

Post #7 of 18 (924 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

Le samedi 03 mars 2012 à 14:41 +0200, Eli Bendersky a écrit :
> >> I'd expect slice subscripts to be part of the sequence interface, and
> >> yet they are not. In fact, they are part of the mapping interface. For
> >> example, the list object has its slice get/set methods assigned to a
> >> PyMappingMethods struct. So does a bytes object, and pretty much every
> >> other object that wants to support subscripts.
> >
> > It comes from:
> > http://hg.python.org/cpython/rev/245224d1b8c9
> > http://bugs.python.org/issue400998
> >
> > Written by Michael Hudson and reviewed by Guido.
> > I wonder why this patch chose to add mapping protocol support to tuples
> > and lists, rather than add a tp_ slot for extended slicing.
> >
>
> Why a separate tp_ slot for extended slicing? ISTM slicing pertains to
> sequences, similarly to other numeric indices. If you look at
> PySequenceMethods it has these (apparently no longer used fields):

Yes, I meant sq_ slot, my bad.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Mar 3, 2012, 9:58 AM

Post #8 of 18 (922 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, Mar 3, 2012 at 4:20 AM, Antoine Pitrou <solipsis [at] pitrou> wrote:
>> I'd expect slice subscripts to be part of the sequence interface, and
>> yet they are not. In fact, they are part of the mapping interface. For
>> example, the list object has its slice get/set methods assigned to a
>> PyMappingMethods struct. So does a bytes object, and pretty much every
>> other object that wants to support subscripts.
>
> It comes from:
> http://hg.python.org/cpython/rev/245224d1b8c9
> http://bugs.python.org/issue400998
>
> Written by Michael Hudson and reviewed by Guido.
> I wonder why this patch chose to add mapping protocol support to tuples
> and lists, rather than add a tp_ slot for extended slicing.

That's long ago... IIRC it was for binary compatibility -- I didn't
want to add an extra slot to the sq struct because it would require
recompilation of 3rd party extensions. At the time that was an
important concern.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


eliben at gmail

Mar 3, 2012, 10:18 AM

Post #9 of 18 (914 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, Mar 3, 2012 at 19:58, Guido van Rossum <guido [at] python> wrote:
> On Sat, Mar 3, 2012 at 4:20 AM, Antoine Pitrou <solipsis [at] pitrou> wrote:
>>> I'd expect slice subscripts to be part of the sequence interface, and
>>> yet they are not. In fact, they are part of the mapping interface. For
>>> example, the list object has its slice get/set methods assigned to a
>>> PyMappingMethods struct. So does a bytes object, and pretty much every
>>> other object that wants to support subscripts.
>>
>> It comes from:
>> http://hg.python.org/cpython/rev/245224d1b8c9
>> http://bugs.python.org/issue400998
>>
>> Written by Michael Hudson and reviewed by Guido.
>> I wonder why this patch chose to add mapping protocol support to tuples
>> and lists, rather than add a tp_ slot for extended slicing.
>
> That's long ago... IIRC it was for binary compatibility -- I didn't
> want to add an extra slot to the sq struct because it would require
> recompilation of 3rd party extensions. At the time that was an
> important concern.
>

Perhaps the situation can be fixed now without binary compatibility
concerns. PySequenceMethods is:

typedef struct {
lenfunc sq_length;
binaryfunc sq_concat;
ssizeargfunc sq_repeat;
ssizeargfunc sq_item;
void *was_sq_slice;
ssizeobjargproc sq_ass_item;
void *was_sq_ass_slice;
objobjproc sq_contains;

binaryfunc sq_inplace_concat;
ssizeargfunc sq_inplace_repeat;
} PySequenceMethods;

The slots "was_sq_slice" and "was_sq_ass_slice" aren't used any
longer. These can be re-incarnated to accept a slice object, and
sequence objects can be rewritten to use them instead of implementing
the mapping protocol (is there any reason listobject implements the
mapping protocol, other than to gain the ability to use slices for
__getitem__?). Existing 3rd party extensions don't *need* to be
recompiled or changed, however. They *can* be, if their authors are
interested, of course.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


thomas at python

Mar 3, 2012, 12:59 PM

Post #10 of 18 (920 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, Mar 3, 2012 at 10:18, Eli Bendersky <eliben [at] gmail> wrote:

> On Sat, Mar 3, 2012 at 19:58, Guido van Rossum <guido [at] python> wrote:
> > On Sat, Mar 3, 2012 at 4:20 AM, Antoine Pitrou <solipsis [at] pitrou>
> wrote:
> >>> I'd expect slice subscripts to be part of the sequence interface, and
> >>> yet they are not. In fact, they are part of the mapping interface. For
> >>> example, the list object has its slice get/set methods assigned to a
> >>> PyMappingMethods struct. So does a bytes object, and pretty much every
> >>> other object that wants to support subscripts.
> >>
> >> It comes from:
> >> http://hg.python.org/cpython/rev/245224d1b8c9
> >> http://bugs.python.org/issue400998
> >>
> >> Written by Michael Hudson and reviewed by Guido.
> >> I wonder why this patch chose to add mapping protocol support to tuples
> >> and lists, rather than add a tp_ slot for extended slicing.
> >
> > That's long ago... IIRC it was for binary compatibility -- I didn't
> > want to add an extra slot to the sq struct because it would require
> > recompilation of 3rd party extensions. At the time that was an
> > important concern.
> >
>
> Perhaps the situation can be fixed now without binary compatibility
> concerns. PySequenceMethods is:
>
> typedef struct {
> lenfunc sq_length;
> binaryfunc sq_concat;
> ssizeargfunc sq_repeat;
> ssizeargfunc sq_item;
> void *was_sq_slice;
> ssizeobjargproc sq_ass_item;
> void *was_sq_ass_slice;
> objobjproc sq_contains;
>
> binaryfunc sq_inplace_concat;
> ssizeargfunc sq_inplace_repeat;
> } PySequenceMethods;
>
> The slots "was_sq_slice" and "was_sq_ass_slice" aren't used any
> longer. These can be re-incarnated to accept a slice object, and
> sequence objects can be rewritten to use them instead of implementing
> the mapping protocol (is there any reason listobject implements the
> mapping protocol, other than to gain the ability to use slices for
> __getitem__?). Existing 3rd party extensions don't *need* to be
> recompiled or changed, however. They *can* be, if their authors are
> interested, of course.


Why even have separate tp_as_sequence and tp_as_mapping anymore? That
particular distinction never existed for Python types, so why should it
exist for C types at all? I forget if there was ever a real point to it,
but all it seems to do now is create confusion, what with many sequence
types implementing both, and PyMapping_Check() and PySequence_Check() doing
seemingly random things to come up with somewhat sensible answers. Do note
that the dict type actually implements tp_as_sequence (in order to support
containtment tests) and that PySequence_Check() has to explicitly return 0
for dicts -- which means that it will give the "wrong" answer for another
type that behaves exactly like dicts.

Getting rid of the misleading distinction seems like a much better idea
than trying to re-conflate some of the issues.

--
Thomas Wouters <thomas [at] python>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!


solipsis at pitrou

Mar 3, 2012, 1:02 PM

Post #11 of 18 (918 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, 3 Mar 2012 12:59:13 -0800
Thomas Wouters <thomas [at] python> wrote:
>
> Why even have separate tp_as_sequence and tp_as_mapping anymore? That
> particular distinction never existed for Python types, so why should it
> exist for C types at all? I forget if there was ever a real point to it,
> but all it seems to do now is create confusion, what with many sequence
> types implementing both, and PyMapping_Check() and PySequence_Check() doing
> seemingly random things to come up with somewhat sensible answers.

Ironically, most of the confusion stems from sequence types
implementing the mapping protocol for extended slicing.

> Do note
> that the dict type actually implements tp_as_sequence (in order to support
> containtment tests) and that PySequence_Check() has to explicitly return 0
> for dicts -- which means that it will give the "wrong" answer for another
> type that behaves exactly like dicts.

It seems to be a leftover:

int
PySequence_Check(PyObject *s)
{
if (PyDict_Check(s))
return 0;
return s != NULL && s->ob_type->tp_as_sequence &&
s->ob_type->tp_as_sequence->sq_item != NULL;
}

Dict objects have a NULL sq_item so even removing the explicit check
would still return the right answer.

> Getting rid of the misleading distinction seems like a much better idea
> than trying to re-conflate some of the issues.

This proposal sounds rather backwards, given that we now have separate
Mapping and Sequence ABCs.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


stefan_ml at behnel

Mar 3, 2012, 1:12 PM

Post #12 of 18 (917 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

Thomas Wouters, 03.03.2012 21:59:
> Why even have separate tp_as_sequence and tp_as_mapping anymore? That
> particular distinction never existed for Python types, so why should it
> exist for C types at all? I forget if there was ever a real point to it,
> but all it seems to do now is create confusion, what with many sequence
> types implementing both, and PyMapping_Check() and PySequence_Check() doing
> seemingly random things to come up with somewhat sensible answers. Do note
> that the dict type actually implements tp_as_sequence (in order to support
> containtment tests) and that PySequence_Check() has to explicitly return 0
> for dicts -- which means that it will give the "wrong" answer for another
> type that behaves exactly like dicts.
>
> Getting rid of the misleading distinction seems like a much better idea
> than trying to re-conflate some of the issues.

We're too far away from the release of Python 4 to change something with
that kind of impact, though.

Stefan

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


thomas at python

Mar 3, 2012, 6:20 PM

Post #13 of 18 (909 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, Mar 3, 2012 at 13:02, Antoine Pitrou <solipsis [at] pitrou> wrote:

> On Sat, 3 Mar 2012 12:59:13 -0800
> Thomas Wouters <thomas [at] python> wrote:
> >
> > Why even have separate tp_as_sequence and tp_as_mapping anymore? That
> > particular distinction never existed for Python types, so why should it
> > exist for C types at all? I forget if there was ever a real point to it,
> > but all it seems to do now is create confusion, what with many sequence
> > types implementing both, and PyMapping_Check() and PySequence_Check()
> doing
> > seemingly random things to come up with somewhat sensible answers.
>
> Ironically, most of the confusion stems from sequence types
> implementing the mapping protocol for extended slicing.
>
> > Do note
> > that the dict type actually implements tp_as_sequence (in order to
> support
> > containtment tests) and that PySequence_Check() has to explicitly return
> 0
> > for dicts -- which means that it will give the "wrong" answer for another
> > type that behaves exactly like dicts.
>
> It seems to be a leftover:
>
> int
> PySequence_Check(PyObject *s)
> {
> if (PyDict_Check(s))
> return 0;
> return s != NULL && s->ob_type->tp_as_sequence &&
> s->ob_type->tp_as_sequence->sq_item != NULL;
> }
>
> Dict objects have a NULL sq_item so even removing the explicit check
> would still return the right answer.
>
> > Getting rid of the misleading distinction seems like a much better idea
> > than trying to re-conflate some of the issues.
>
> This proposal sounds rather backwards, given that we now have separate
> Mapping and Sequence ABCs.
>

I'm not sure how the ABCs, which are abstract declarations of semantics,
tie into this specific implementation detail. ABCs work just as well for
Python types as for C types, and Python types don't have this distinction.
The distinction in C types has been *practically* useless for years, so why
should it stay? What is the actual benefit here?

--
Thomas Wouters <thomas [at] python>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!


thomas at python

Mar 3, 2012, 6:24 PM

Post #14 of 18 (910 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, Mar 3, 2012 at 13:12, Stefan Behnel <stefan_ml [at] behnel> wrote:

> Thomas Wouters, 03.03.2012 21:59:
> > Why even have separate tp_as_sequence and tp_as_mapping anymore? That
> > particular distinction never existed for Python types, so why should it
> > exist for C types at all? I forget if there was ever a real point to it,
> > but all it seems to do now is create confusion, what with many sequence
> > types implementing both, and PyMapping_Check() and PySequence_Check()
> doing
> > seemingly random things to come up with somewhat sensible answers. Do
> note
> > that the dict type actually implements tp_as_sequence (in order to
> support
> > containtment tests) and that PySequence_Check() has to explicitly return
> 0
> > for dicts -- which means that it will give the "wrong" answer for another
> > type that behaves exactly like dicts.
> >
> > Getting rid of the misleading distinction seems like a much better idea
> > than trying to re-conflate some of the issues.
>
> We're too far away from the release of Python 4 to change something with
> that kind of impact, though.


It's not hard to do this in a backward-compatible way. Either grow one of
the tp_as_* to include everything a 'unified' tp_as_everything struct would
need, or add a new tp_as_everything slot in the type struct. Then add a
tp_flag to indicate that the type has this new layout/slot and guard all
uses of the new slots with a check for that flag. If the type doesn't have
the new layout or doesn't have it or the slots in it set, the code can fall
back to the old try-one-and-then-the-other behaviour of dealing with
tp_as_sequence and tp_as_mapping.

(Let's not forget about tp_as_sequence.sq_concat, tp_as_number.nb_add,
tp_as_sequence.sq_repeat and tp_as_number.nb_mul either.)

--
Thomas Wouters <thomas [at] python>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!


eliben at gmail

Mar 3, 2012, 7:37 PM

Post #15 of 18 (916 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

>> Thomas Wouters, 03.03.2012 21:59:
>> > Why even have separate tp_as_sequence and tp_as_mapping anymore? That
>> > particular distinction never existed for Python types, so why should it
>> > exist for C types at all? I forget if there was ever a real point to it,
>> > but all it seems to do now is create confusion, what with many sequence
>> > types implementing both, and PyMapping_Check() and PySequence_Check()
>> > doing
>> > seemingly random things to come up with somewhat sensible answers. Do
>> > note
>> > that the dict type actually implements tp_as_sequence (in order to
>> > support
>> > containtment tests) and that PySequence_Check() has to explicitly return
>> > 0
>> > for dicts -- which means that it will give the "wrong" answer for
>> > another
>> > type that behaves exactly like dicts.
>> >
>> > Getting rid of the misleading distinction seems like a much better idea
>> > than trying to re-conflate some of the issues.
>>
>> We're too far away from the release of Python 4 to change something with
>> that kind of impact, though.
>
>
> It's not hard to do this in a backward-compatible way. Either grow one of
> the tp_as_* to include everything a 'unified' tp_as_everything struct would
> need, or add a new tp_as_everything slot in the type struct. Then add a
> tp_flag to indicate that the type has this new layout/slot and guard all
> uses of the new slots with a check for that flag. If the type doesn't have
> the new layout or doesn't have it or the slots in it set, the code can fall
> back to the old try-one-and-then-the-other behaviour of dealing with
> tp_as_sequence and tp_as_mapping.
>
> (Let's not forget about tp_as_sequence.sq_concat, tp_as_number.nb_add,
> tp_as_sequence.sq_repeat and tp_as_number.nb_mul either.)
>

There's nothing to unify, really, since PyMappingMethods is just a
subset of PySequenceMethods:

typedef struct {
lenfunc mp_length;
binaryfunc mp_subscript;
objobjargproc mp_ass_subscript;
} PyMappingMethods;

with the small difference that in PySequenceMethods sq_item and
sq_ass_item just accept numeric indices. However, if PySequenceMethods
has the was_sq_sclies and was_sq_ass_slice fields are reinstated to
accept a generic PyObject, PyMappingMethods will be a true subset.

If we look at the code, this becomes even clearer: in a full grep on
the Python 3.3 source, there is no object that defines tp_as_mapping
but does not also define tp_as_sequence, except Modules/_sqlite/row.c
[.I'm not familiar enough with the _sqlite module, but there's a chance
it would make sense for the Row to be a sequence too].

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Mar 3, 2012, 7:59 PM

Post #16 of 18 (915 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sun, Mar 4, 2012 at 12:24 PM, Thomas Wouters <thomas [at] python> wrote:
> (Let's not forget about tp_as_sequence.sq_concat, tp_as_number.nb_add,
> tp_as_sequence.sq_repeat and tp_as_number.nb_mul either.)

Indeed, let's not forget about those, which are a compatibility
problem in and of themselves: http://bugs.python.org/issue11477

At most, the tp_mapping and tp_as_sequence overlap should be an FAQ
entry in the devguide that says "yes, the implementation of this is
weird. It's like that for historical reasons, and fixing it is a long
way down the priority list for changes"

Cheers,
Nick.

--
Nick Coghlan†† |†† ncoghlan [at] gmail†† |†† Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


solipsis at pitrou

Mar 4, 2012, 5:56 AM

Post #17 of 18 (900 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

On Sat, 3 Mar 2012 18:20:22 -0800
Thomas Wouters <thomas [at] python> wrote:
>
> I'm not sure how the ABCs, which are abstract declarations of semantics,
> tie into this specific implementation detail. ABCs work just as well for
> Python types as for C types, and Python types don't have this distinction.
> The distinction in C types has been *practically* useless for years, so why
> should it stay? What is the actual benefit here?

For one, it's certainly easier to implement an extension type if your
getitem function receives a Py_ssize_t directly, rather than a PyObject.
(it can be more efficient too)

Regards

Antoine.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


greg.ewing at canterbury

Mar 4, 2012, 1:14 PM

Post #18 of 18 (901 views)
Permalink
Re: slice subscripts for sequences and mappings [In reply to]

Thomas Wouters wrote:

> Why even have separate tp_as_sequence and tp_as_mapping anymore? That
> particular distinction never existed for Python types, so why should it
> exist for C types at all? I forget if there was ever a real point to it,

I imagine the original motivation was to provide a fast path
for types that take ints as indexes. Also, it dates from the
very beginnings of Python, before it had user defined classes.
At that time the archetypal sequence (list) and the archetypal
mapping (dict) were very distinct -- I don't think dicts
supported 'in' then, so there was no overlap. It looks like a
case of "it seemed like a good idea at the time".

The distinction broke down fairly soon after, but it's so
embedded in the extension module API that it's been very hard
to get rid of.

--
Greg
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.