Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

PEP 362: 4th edition

 

 

First page Previous page 1 2 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded


yselivanov.ml at gmail

Jun 18, 2012, 7:37 AM

Post #26 of 44 (1002 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

Jim,

On 2012-06-18, at 3:08 AM, Jim Jewett wrote:
> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
>> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett <jimjjewett [at] gmail> wrote:
>
>>> Instead of defining a BoundArguments class, just return
>>> a copy of the Signature, with "value" attributes added to
>>> the Parameters.
>
>> No, the "BoundArguments" class is designed to be easy to
>> feed to a function call as f(*args, **kwds)
>
> Why does that take a full class, as opposed to a method returning a
> tuple and a dict?

Read this thread, please: http://mail.python.org/pipermail/python-dev/2012-June/120000.html
And also take a look at the check types example in the pep.

In short - it's easy to work with 'BoundArguments.arguments' list,
but it is not enough for invocation, thus 'BoundArguments.args' &
'.kwargs' properties.

>>> Use subclasses to distinguish the parameter kind.
>
>> Please, no, using subclasses when there is no behavioural
>> change is annoying.
>
> A **kwargs argument is very different from an ordinary parameter. Its
> name doesn't matter (and therefore should not be considered in
> __eq__),

The importance of its name depends hugely on the use context. In some
it may be very important.

> it can only appear once per signature, and the possible
> location of its appearance is different.

I'll fix the __eq__ to ignore positions of **kwargs & keyword-only
parameters.

> It is formatted differently
> (which I would prefer to do in the Parameter, rather than in
> Signature).

I think we'll remove the 'Signature.format()' from the PEP, leaving just
'Signature.__str__'. And just for '__str__' I don't think it's a bad
idea to let Signature format its parameters.

> It also holds very different data, and must be treated
> specially by several Signature methods, particularly when either
> validating or binding. (It is bound to a Mapping, rather than to a
> single value, so you have to keep it around longer and use a different
> "bind method".)

And it is treated specially, along with the *args.

>>>> A Signature object has the following public attributes and methods:
>
> The more I try to work with it, the more I want direct references to
> the two special arguments (*args, **kwargs) if they exist. FWIW, the
> current bind logic to find them -- particularly kwargs -- seems
> contorted, compared to self.kwargsparameter.

Well, 'self.kwargsparameter' will break 'self.parameters' collection,
unless you want one parameter to be in two places. I've already had
three or four implementations of this PEP, with the first couple having
**kwargs parameter stored separately, but keeping all parameters in one
collection turned out to be the most elegant solution.

In fact, the check types example (in the PEP) is currently shorter and
easier to read with 'Signature.parameters' than with dedicated property
for '**kwargs' parameter.

And if after all you need direct references to *args or **kwargs - write
a little helper, which finds them in 'Signature.parameters'.

>>> I'm not sure
>>> if positional parameters should also check position, or if that
>>> can be left to the Signature.
>
>> Positional parameters don't know their relative position, so it *has*
>> to be left to the signature.
>
> But perhaps they *should* know their relative position.

I disagree here. That will just complicate things.

> Also,
> positional_only, *args, and **kwargs should be able to remove name
> from the list of compared attributes.

I still believe in the most contexts the name of a parameter matters
(even if it's **kwargs). Besides, how can we make __eq__ to be
configurable? Let's make it do the most explicit and simple logic,
and those who need a custom one will implement such for themselves.

Thank you,
-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


pje at telecommunity

Jun 18, 2012, 10:35 AM

Post #27 of 44 (997 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Fri, Jun 15, 2012 at 5:03 PM, R. David Murray <rdmurray [at] bitdance>wrote:

> On Fri, 15 Jun 2012 22:48:42 +0200, Victor Stinner <
> victor.stinner [at] gmail> wrote:
> > > 1. Should we keep 'Parameter.implemented' or not. *Please vote*
>
> -1 to implemented.
>
> > I still disagree with the deepcopy. I read somewhere that Python
> > developers are consenting adult. If someone really want to modify a
> > Signature, it would be nice to provide a simple method to copy it. But
> > I don't see why it should be copied *by default*. I expect that
> > modifying a signature is more rare than just reading a signature.
>
> The issue isn't "consenting adults", the issue is consistency.
> Without the deepcopy, sometimes what you get back from the
> inspect function is freely modifiable and sometimes it is not.
> That inconsistency is a bad thing.
>

Then just copy the signature itself; as currently written, this is going to
copy the annotation objects, which could produce weird side-effects from
introspection. Using deepcopy seems like overkill when all that's needed
is a new Signature instance with a fresh OrderedDict.

Or, better yet: make signature and parameter objects immutable (along with
the OrderedDict) and the whole problem of modification and copying goes
away altogether. Or is there some reason not mentioned in the PEP why
mutability is necessary? (The PEP provides no rationale at present for
making any part of a signature mutable)


yselivanov.ml at gmail

Jun 18, 2012, 11:09 AM

Post #28 of 44 (1006 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 2012-06-18, at 1:35 PM, PJ Eby wrote:

> On Fri, Jun 15, 2012 at 5:03 PM, R. David Murray <rdmurray [at] bitdance> wrote:
> On Fri, 15 Jun 2012 22:48:42 +0200, Victor Stinner <victor.stinner [at] gmail> wrote:
> > > 1. Should we keep 'Parameter.implemented' or not. *Please vote*
>
> -1 to implemented.
>
> > I still disagree with the deepcopy. I read somewhere that Python
> > developers are consenting adult. If someone really want to modify a
> > Signature, it would be nice to provide a simple method to copy it. But
> > I don't see why it should be copied *by default*. I expect that
> > modifying a signature is more rare than just reading a signature.
>
> The issue isn't "consenting adults", the issue is consistency.
> Without the deepcopy, sometimes what you get back from the
> inspect function is freely modifiable and sometimes it is not.
> That inconsistency is a bad thing.
>
> Then just copy the signature itself; as currently written, this is going to copy the annotation objects, which could produce weird side-effects from introspection. Using deepcopy seems like overkill when all that's needed is a new Signature instance with a fresh OrderedDict.

That's an excerpt from Signature.__deepcopy__:

cls = type(self)
sig = cls.__new__(cls)
sig.parameters = OrderedDict((name, param.__copy__()) \
for name, param in self.parameters.items())

And Parameter.__copy__:

cls = type(self)
copy = cls.__new__(cls)
copy.__dict__.update(self.__dict__)
return copy

So we don't recursively deepcopy parameters in Signature.__deepcopy__
(I hope that we don't violate the deepcopy meaning here)

> Or, better yet: make signature and parameter objects immutable (along with the OrderedDict) and the whole problem of modification and copying goes away altogether. Or is there some reason not mentioned in the PEP why mutability is necessary? (The PEP provides no rationale at present for making any part of a signature mutable)

The rationale is that sometimes you need to modify signatures.
For instance, in decorators.

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Jun 18, 2012, 1:25 PM

Post #29 of 44 (999 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Mon, Jun 18, 2012 at 11:09 AM, Yury Selivanov
<yselivanov.ml [at] gmail> wrote:
> The rationale is that sometimes you need to modify signatures.
> For instance, in decorators.

A decorator should make a modified copy, not modify it in place (since
the signature of the decorated function does not change, and you have
no guarantee that that function is no longer accessible.)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


jimjjewett at gmail

Jun 18, 2012, 2:06 PM

Post #30 of 44 (993 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Mon, Jun 18, 2012 at 10:37 AM, Yury Selivanov
<yselivanov.ml [at] gmail> wrote:
> Jim,
>
> On 2012-06-18, at 3:08 AM, Jim Jewett wrote:
>> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
>>> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett <jimjjewett [at] gmail> wrote:

>>>>    Instead of defining a BoundArguments class, just return
>>>>   a copy  of the Signature, with "value" attributes added to
>>>>   the Parameters.

>>> No, the "BoundArguments" class is designed to be easy to
>>> feed to a function call as f(*args, **kwds)

>> Why does that take a full class, as opposed to a method returning a
>> tuple and a dict?

> Read this thread, please: http://mail.python.org/pipermail/python-dev/2012-June/120000.html

I reread that. I still don't see why it needs to be an instance of a
specific independent class, as opposed to a Signature method that
returns a (tuple of) a tuple and a dict.

((arg1, arg2, arg3...), {key1: val2, key2: val2})


>>>>    Use subclasses to distinguish the parameter kind.

>>> Please, no, using subclasses when there is no behavioural
>>> change is annoying.

[Examples of how the "kinds" of parameters are qualitatively different.]

>> A **kwargs argument is very different from an ordinary parameter.  Its
>> name doesn't matter (and therefore should not be considered in
>> __eq__),

> The importance of its name depends hugely on the use context.  In some
> it may be very important.

The name of kwargs can only be for documentation purposes. Like an
annotation or a docstring, it won't affect the success of an attempted
call. Annotations are kept because (often) their entire purpose is to
document the signature. But docstrings are being dropped, because
they often serve other purposes. I've had far more use for docstrings
than for the names of positional-only parameters. (In fact, knowing
the name of a positional-only parameter has sometimes been an
attractive nuisance.)

> And it is treated specially, along with the *args.

Right -- but this was in response to Nick's claim that the
distinctions should not be represented as a subclass, because the
behavior wasn't different.

I consider different __eq__ implementations or formatting concers to
be sufficient on their own; I also consider different possible use
locations and counts, different used-by-the-system attributes (name),
or different value types (object vs collection) to be sufficiently
behavioral.

>>>>> A Signature object has the following public attributes and methods:

>> The more I try to work with it, the more I want direct references to
>> the two special arguments (*args, **kwargs) if they exist.  FWIW, the
>> current bind logic to find them -- particularly kwargs -- seems
>> contorted, compared to self.kwargsparameter.

> Well, 'self.kwargsparameter'  will break 'self.parameters' collection,
> unless you want one parameter to be in two places.

Correct; it should be redundant. Signature.kwargsparameter should be
the same object that occurs as the nth element of
Signature.parameters.values(). It is just more convenient to retrieve
the parameter directly than it is to iterate through a collection
inspecting each element for the value of a specific attribute.


> In fact, the check types example (in the PEP) is currently shorter and
> easier to read with 'Signature.parameters' than with dedicated property
> for '**kwargs' parameter.

Agreed; the short-cuts *args and **kwargs are only useful because they
are special; they aren't needed when you're doing the same thing to
all parameters regardless of type.

> And if after all you need direct references to *args or **kwargs - write
> a little helper, which finds them in 'Signature.parameters'.

Looking at http://bugs.python.org/review/15008/diff/5143/Lib/inspect.py
you already need one in _bind; it is just that saving the info when
you pass it isn't too bad if you're already iterating through the
whole collection anyhow.

>>  Also,
>> positional_only, *args, and **kwargs should be able to remove name
>> from the list of compared attributes.

> I still believe in the most contexts the name of a parameter matters
> (even if it's **kwargs).  Besides, how can we make __eq__ to be
> configurable?

__eq__ can can an _eq_fields attribute to see which other attributes
matter -- but it makes more sense for that to be (sub-) class
property.

-jJ
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Jun 18, 2012, 3:54 PM

Post #31 of 44 (989 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

Jim,

On 2012-06-18, at 5:06 PM, Jim Jewett wrote:
> On Mon, Jun 18, 2012 at 10:37 AM, Yury Selivanov
> <yselivanov.ml [at] gmail> wrote:
>> Jim,
>>
>> On 2012-06-18, at 3:08 AM, Jim Jewett wrote:
>>> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
>>>> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett <jimjjewett [at] gmail> wrote:
>
>>>>> Instead of defining a BoundArguments class, just return
>>>>> a copy of the Signature, with "value" attributes added to
>>>>> the Parameters.
>
>>>> No, the "BoundArguments" class is designed to be easy to
>>>> feed to a function call as f(*args, **kwds)
>
>>> Why does that take a full class, as opposed to a method returning a
>>> tuple and a dict?
>
>> Read this thread, please: http://mail.python.org/pipermail/python-dev/2012-June/120000.html
>
> I reread that. I still don't see why it needs to be an instance of a
> specific independent class, as opposed to a Signature method that
> returns a (tuple of) a tuple and a dict.
>
> ((arg1, arg2, arg3...), {key1: val2, key2: val2})

I'll try to explain with the code:

def foo(a, *args, b, **kwargs): pass
sig = signature(foo)

* Case one. We have BoundArguments:

ba = sig.bind(1, 2, 3, b=123, c='foo', d='bar')

Now, in 'ba.arguments':

{'a': 1, 'args': (2, 3), 'b': 123, 'kwargs': {'c': 'foo', 'd': 'bar'}}

It's easy to work with 'ba.arguments', just traverse it as follows:

for arg_name, arg_value in ba.arguments:
param = sig.parameters[arg_name]

So you have argument name and value, and the corresponding parameter.

* Case two. We return tuple and dict:

ba = sig.bind(1, 2, 3, b=123, c='foo', d='bar')
((1, 2, 3), {'b': 123, 'c': 'foo', 'd': 'bar'})

Now, how are you going to work with that? How will you map those
values to the corresponding parameters? The whole point of having
'Signature.bind()' is to provide you that mapping.

In the link I gave you, I also explained why we need 'BoundArguments.args'
and 'BoundArguments.kwargs'.

>>>>> Use subclasses to distinguish the parameter kind.
>
>>>> Please, no, using subclasses when there is no behavioural
>>>> change is annoying.
>
> [Examples of how the "kinds" of parameters are qualitatively different.]
>
>>> A **kwargs argument is very different from an ordinary parameter. Its
>>> name doesn't matter (and therefore should not be considered in
>>> __eq__),
>
>> The importance of its name depends hugely on the use context. In some
>> it may be very important.
>
> The name of kwargs can only be for documentation purposes.

Not really. I've seen once the code where types of acceptable values
were encoded parameter names (after '__', like 'kwargs__int').
This is a rather extreme example, but it illustrates that names
may be important.

> Like an
> annotation or a docstring, it won't affect the success of an attempted
> call.

It very well may affect it. For instance, I use annotations to
specify arguments types in RPC dispatch. The call will not be dispatched
in case of a wrong type. So 'foo(a:int)' in my context does not
equal to 'foo(a:str)'.

>> And it is treated specially, along with the *args.
>
> Right -- but this was in response to Nick's claim that the
> distinctions should not be represented as a subclass, because the
> behavior wasn't different.

Yes, it's behaviour of the outer code (Signature) that is different, not
the Parameter instances. It's Signature who makes a decision of how to map
parameters, not parameters themselves.

> I consider different __eq__ implementations or formatting concers to
> be sufficient on their own; I also consider different possible use
> locations and counts, different used-by-the-system attributes (name),
> or different value types (object vs collection) to be sufficiently
> behavioral.
>
>>>>>> A Signature object has the following public attributes and methods:
>
>>> The more I try to work with it, the more I want direct references to
>>> the two special arguments (*args, **kwargs) if they exist. FWIW, the
>>> current bind logic to find them -- particularly kwargs -- seems
>>> contorted, compared to self.kwargsparameter.
>
>> Well, 'self.kwargsparameter' will break 'self.parameters' collection,
>> unless you want one parameter to be in two places.
>
> Correct; it should be redundant. Signature.kwargsparameter should be
> the same object that occurs as the nth element of
> Signature.parameters.values().

It will be always the last parameter (if specified at all)

> It is just more convenient to retrieve
> the parameter directly than it is to iterate through a collection
> inspecting each element for the value of a specific attribute.

Again, depends on the use case. Can you show a realistic use case,
that needs that? (The use case should be also common and widespread,
i.e. it should worth it to uglify Signature class structure)

>> In fact, the check types example (in the PEP) is currently shorter and
>> easier to read with 'Signature.parameters' than with dedicated property
>> for '**kwargs' parameter.
>
> Agreed; the short-cuts *args and **kwargs are only useful because they
> are special; they aren't needed when you're doing the same thing to
> all parameters regardless of type.
>
>> And if after all you need direct references to *args or **kwargs - write
>> a little helper, which finds them in 'Signature.parameters'.
>
> Looking at http://bugs.python.org/review/15008/diff/5143/Lib/inspect.py
> you already need one in _bind; it is just that saving the info when
> you pass it isn't too bad if you're already iterating through the
> whole collection anyhow.

Having a 'Signature.kwargs_param' would save just 2 lines of code
(1736:'kwargs_param = None', and 1745:'kwargs_param = param') out of 120.
And that's BTW is a pretty complex piece of code.

Again, it all depends on the use-case.

>>> Also,
>>> positional_only, *args, and **kwargs should be able to remove name
>>> from the list of compared attributes.
>
>> I still believe in the most contexts the name of a parameter matters
>> (even if it's **kwargs). Besides, how can we make __eq__ to be
>> configurable?
>
> __eq__ can can an _eq_fields attribute to see which other attributes
> matter -- but it makes more sense for that to be (sub-) class
> property.

Well we can make __eq__ customizable, but I'm afraid it will just
complicate things too much. It should take you 10-20 lines of code to
implement any comparison algorithm you want - why try to predict all
of them and put in the stdlib?

Thank you,
-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Jun 18, 2012, 4:10 PM

Post #32 of 44 (999 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 2012-06-18, at 4:25 PM, Guido van Rossum wrote:

> On Mon, Jun 18, 2012 at 11:09 AM, Yury Selivanov
> <yselivanov.ml [at] gmail> wrote:
>> The rationale is that sometimes you need to modify signatures.
>> For instance, in decorators.
>
> A decorator should make a modified copy, not modify it in place (since
> the signature of the decorated function does not change, and you have
> no guarantee that that function is no longer accessible.)



It seems that we have the following options for 'signature(obj)':

1. If 'obj' has a '__signature__' attribute - return a copy of it,
if not - create a new one.

2. If 'obj' has a '__signature__' attribute - return it,
if not - create a new one.

3. Same as '2', but Signature is also immutable.


The first option is the one currently implemented. Its advantage
is consistency - we always have a Signature we can safely modify.

The second option has a design flaw - sometimes the result Signature
is safe to modify, sometimes not, you never know.

The third option is hard to work with.
Instead of:

sig = signature(wrapper)
sig.parameters.popitem(last=False)
decorator.__signature__ = sig

We will have (because Signature is immutable):

sig = signature(wrapper)
params = OrderedDict(sig.parameters.items())
params.popitem(last=False)

attrs = {'parameters': params}
try:
ra = sig.return_annotation
except AttributeError:
pass
else:
attrs['return_annotation'] = ra

decorator.__signature__ = Signature.from_attrs(**attrs)

It looks like a total overkill (unless we can come up with a nicer
API).

So it was decided to go with the first option, as it has the least
complications. Plus, the copying itself should be fast, as
Signatures contain little information.

What do you think?

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ethan at stoneleaf

Jun 18, 2012, 6:17 PM

Post #33 of 44 (986 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

Yury Selivanov wrote:
> On 2012-06-18, at 4:25 PM, Guido van Rossum wrote:
>
>> On Mon, Jun 18, 2012 at 11:09 AM, Yury Selivanov
>> <yselivanov.ml [at] gmail> wrote:
>>> The rationale is that sometimes you need to modify signatures.
>>> For instance, in decorators.
>> A decorator should make a modified copy, not modify it in place (since
>> the signature of the decorated function does not change, and you have
>> no guarantee that that function is no longer accessible.)
>
>
>
> It seems that we have the following options for 'signature(obj)':
>
> 1. If 'obj' has a '__signature__' attribute - return a copy of it,
> if not - create a new one.
>
> 2. If 'obj' has a '__signature__' attribute - return it,
> if not - create a new one.
>
> 3. Same as '2', but Signature is also immutable.
>
>
> The first option is the one currently implemented. Its advantage
> is consistency - we always have a Signature we can safely modify.
>
> The second option has a design flaw - sometimes the result Signature
> is safe to modify, sometimes not, you never know.
>
> The third option is hard to work with.
> Instead of:
>
> sig = signature(wrapper)
> sig.parameters.popitem(last=False)
> decorator.__signature__ = sig
>
> We will have (because Signature is immutable):
>
> sig = signature(wrapper)
> params = OrderedDict(sig.parameters.items())
> params.popitem(last=False)
>
> attrs = {'parameters': params}
> try:
> ra = sig.return_annotation
> except AttributeError:
> pass
> else:
> attrs['return_annotation'] = ra
>
> decorator.__signature__ = Signature.from_attrs(**attrs)
>
> It looks like a total overkill (unless we can come up with a nicer
> API).
>
> So it was decided to go with the first option, as it has the least
> complications. Plus, the copying itself should be fast, as
> Signatures contain little information.
>
> What do you think?

Option 1 makes sense to me -- we already know we'll have cases where we
want to modify a given signature, so why make it hard on ourselves?

~Ethan~
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Jun 18, 2012, 6:22 PM

Post #34 of 44 (1000 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Mon, Jun 18, 2012 at 5:08 PM, Jim Jewett <jimjjewett [at] gmail> wrote:
> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
>> No. This is the full set of binding behaviours. "self" is just an
>> ordinary "POSITIONAL_OR_KEYWORD" argument (or POSITIONAL_ONLY, in some
>> builtin cases).
>
> Or no longer a "parameter" at all, once the method is bound.  Except
> it sort of still is.  Same for the space parameter in PyPy.  I don't
> expect the stdlib implementation to support them initially, but I
> don't want it to get in the way, either.  A supposedly closed set gets
> in the way.

It's not supposedly closed, it *is* closed: Python doesn't support any
other ways of binding arguments to parameters. Now, you can have
additional state on a callable that gets used by that callable (such
as __closure__ and __globals__ on a function, or __self__ on a method,
or arbitrary state on an object that implements __call__) but that
extra state is not part of the call *signature*, and thus should not
be exposed on the result of inspect.getsignature(). Remember, this
object is not meant to be a representation of the full state of a
callable, it's solely about providing a consistent introspection
mechanism that allows arbitrary callables to define how they will bind
arguments to parameters.

That's why I keep pointing out that there will always need to be a
higher level object that brings in other related information, such as
the docstring, the name of the callable, etc. This is not an API that
describes *everything* that is worth knowing about an arbitrary
callable, nor is it intended to be.

I believe you have raised a legitimate question regarding whether or
not there is sufficient variation in behaviour amongst the parameter
kinds for it to be worth using a subclassing approach to override
__str__ and __eq__, compared to just implementing a few "kind" checks
in the base implementation.

However, given that the possible binding behaviours (positional only,
keyword-or-positional, excess positional, keyword-only, excess
keywords) *is* a closed set, I think a subclass based solution is
overkill and adds excessive complexity to the public API.

Cheers,
Nick.

P.S. A more complete response to the question of what constitutes
"suitable complexity" for this API.

Consider the following implementation sketch for a kind based
implementation that pushes as much behaviour as is reasonable into the
Parameter object (including the definition of equality and decisions
on how the parameter should be displayed):

_sentinel = object()
class Parameter:
def __init__(self, name, kind, default=_sentinel,
annotation=_sentinel):
if not name:
raise ValueError("All parameters must be named for
introspection purposes (even positional-only parameters)")
self.name = name
if kind not in Parameter.KINDS:
raise ValueError("Unrecognised parameter binding type
{}".format(kind))
self.kind = kind
if default is not _sentinel:
if kind.startswith("VAR"):
raise ValueError("Cannot specify default value
for {} parameter".format(kind))
self.default = default
if annotation is not _sentinel:
self.annotation = annotation

def _get_key(self):
default = getattr(self, "default", _sentinel)
annotation = getattr(self, "annotation", _sentinel)
if self.kind in (Parameter.KEYWORD_OR_POSITIONAL,
Parameter.KEYWORD_ONLY):
# The name is considered significant for parameters
that can be specified
# as keyword arguments
return (self.name, self.kind, default, annotation)
# Otherwise, we don't really care about the name
return (self.kind, default, annotation)

def __eq__(self, other):
if not isinstance(other, Parameter):
return NotImplemented
return self._get_key() == other._get_key()

def __str__(self):
kind = self.kind
components = []
if kind == Parameter.VAR_POSITIONAL:
components += ["*"]
elif kind == Parameter.VAR_KEYWORD:
components += ["**"]
if kind == Parameter.POSITIONAL_ONLY:
components += ["<", self.name, ">"]
else:
components += [self.name]
try:
default = self.default
except AttributeError:
pass
else:
components += ["=", repr(default)]
try:
annotation = self.annotation
except AttributeError:
pass
else:
components += [":", repr(annotation)]
return "".join(components)

The points of variation:
- VAR_POSITIONAL and VAR_KEYWORD do not permit a "default" attribute
- VAR_POSITIONAL adds a "*" before the name when printed out
- VAR_KEYWORD adds a "**" before the name when printed out
- POSITIONAL_ONLY adds "<>" around the name when printed out
- all three of those ignore "name" for comparisons

Now, suppose we dispense with the kind attribute and use subclassing instead:

_sentinel = object()
class _ParameterBase:
"""Common behaviour for all parameter types"""
def __init__(self, name, default=_sentinel, annotation=_sentinel):
if not name:
raise ValueError("All parameters must be named for
introspection purposes")
self.name = name
self.kind = kind
if default is not _sentinel:
self.default = default
if annotation is not _sentinel:
self.annotation = annotation

def _get_key(self):
default = getattr(self, "default", _sentinel)
annotation = getattr(self, "annotation", _sentinel)
return (self.name, default, annotation)

def __eq__(self, other):
if not isinstance(other, type(self)):
return NotImplemented
return self._get_key() == other._get_key()

def __str__(self):
components = [self.name]
try:
default = self.default
except AttributeError:
pass
else:
components += ["=", repr(default)]
try:
annotation = self.annotation
except AttributeError:
pass
else:
components += [":", repr(annotation)]
return "".join(components)

class Parameter(_ParameterBase):
"""Representation of a normal Python function parameter"""

class KeywordOnlyParameter(_ParameterBase):
"""Representation of a keyword-only Python function parameter"""

class PositionalOnlyParameter(_ParameterBase):
"""Representation of a positional-only parameter"""
def __init__(self, index, name=None,
default=_sentinel, annotation=_sentinel):
if name is not None:
display_name = "<{}:{}>".format(index, name)
else:
display_name = "<{}>".format(index)
super().__init__(display_name, default, annotation)

def _get_key(self):
default = getattr(self, "default", _sentinel)
annotation = getattr(self, "annotation", _sentinel)
return (default, annotation)

class _VarParameterBase(_ParameterBase):
"""Common behaviour for variable argument parameters"""

class VarPositionalParameter(_VarParameterBase):
"""Representation of a parameter for excess positional arguments"""
def __str__(self):
return "*" + super().__str__()

class VarKeywordParameter(_VarParameterBase):
"""Representation of a parameter for excess keyword arguments"""
def __str__(self):
return "**" + super().__str__()

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Jun 18, 2012, 6:29 PM

Post #35 of 44 (986 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Tue, Jun 19, 2012 at 7:06 AM, Jim Jewett <jimjjewett [at] gmail> wrote:
> Correct; it should be redundant.  Signature.kwargsparameter should be
> the same object that occurs as the nth element of
> Signature.parameters.values().  It is just more convenient to retrieve
> the parameter directly than it is to iterate through a collection
> inspecting each element for the value of a specific attribute.

I suspect in 3.4 we will add the following additional convenience properties:

Signature.positional -> list[Parameter]
List of POSITIONAL_ONLY and KEYWORD_OR_POSITIONAL parameters
Signature.var_positional -> None or Parameter
Reference to the VAR_POSITIONAL parameter, if any
Signature.keyword -> dict{name:Parameter}
Mapping of all KEYWORD_ONLY and KEYWORD_OR_POSITIONAL parameters
Signature.var_keyword -> None or Parameter
Reference to the VAR_KEYWORD parameter, if any

However, I don't think we should add such convenience properties
*right now*. One step at a time.

> __eq__ can can an _eq_fields attribute to see which other attributes
> matter -- but it makes more sense for that to be (sub-) class
> property.

Only if you accept the premise that there are other possible parameter
binding behaviours beyond the five already defined.
Hypergeneralisation is a great way to make an API far more complex
than it needs to be.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Jun 18, 2012, 6:33 PM

Post #36 of 44 (994 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 2012-06-18, at 9:29 PM, Nick Coghlan wrote:

> On Tue, Jun 19, 2012 at 7:06 AM, Jim Jewett <jimjjewett [at] gmail> wrote:
>> Correct; it should be redundant. Signature.kwargsparameter should be
>> the same object that occurs as the nth element of
>> Signature.parameters.values(). It is just more convenient to retrieve
>> the parameter directly than it is to iterate through a collection
>> inspecting each element for the value of a specific attribute.
>
> I suspect in 3.4 we will add the following additional convenience properties:
>
> Signature.positional -> list[Parameter]
> List of POSITIONAL_ONLY and KEYWORD_OR_POSITIONAL parameters
> Signature.var_positional -> None or Parameter
> Reference to the VAR_POSITIONAL parameter, if any
> Signature.keyword -> dict{name:Parameter}
> Mapping of all KEYWORD_ONLY and KEYWORD_OR_POSITIONAL parameters
> Signature.var_keyword -> None or Parameter
> Reference to the VAR_KEYWORD parameter, if any

Maybe. But I'd suggest to avoid the intersection of 'Signature.positional'
and 'Signature.keyword'. Better to have 'Signature.keywordonly'.

> However, I don't think we should add such convenience properties
> *right now*. One step at a time.

+1. 'Signature.parameters' seems to be enough right now.

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Jun 18, 2012, 6:36 PM

Post #37 of 44 (996 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Tue, Jun 19, 2012 at 4:09 AM, Yury Selivanov <yselivanov.ml [at] gmail> wrote:
> On 2012-06-18, at 1:35 PM, PJ Eby wrote:
>> Then just copy the signature itself; as currently written, this is going to copy the annotation objects, which could produce weird side-effects from introspection.  Using deepcopy seems like overkill when all that's needed is a new Signature instance with a fresh OrderedDict.
>
> That's an excerpt from Signature.__deepcopy__:
>
>     cls = type(self)
>     sig = cls.__new__(cls)
>     sig.parameters = OrderedDict((name, param.__copy__()) \
>                           for name, param in self.parameters.items())
>
> And Parameter.__copy__:
>
>        cls = type(self)
>        copy = cls.__new__(cls)
>        copy.__dict__.update(self.__dict__)
>        return copy
>
> So we don't recursively deepcopy parameters in Signature.__deepcopy__
> (I hope that we don't violate the deepcopy meaning here)

In my opinion, It's better to redefine what you mean by a shallow copy
(making it a bit deeper than just the direct attributes) rather than
making a so-called deep copy shallower.

So keep the current copying semantics for Signature objects (i.e.
creating new copies of the Parameter objects as well), but call it a
shallow copy rather than a deep copy. Make it clear in the
documentation that any defaults and annotations are still shared with
the underlying callable.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steve at pearwood

Jun 18, 2012, 6:50 PM

Post #38 of 44 (996 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Mon, Jun 18, 2012 at 07:10:33PM -0400, Yury Selivanov wrote:

> It seems that we have the following options for 'signature(obj)':
>
> 1. If 'obj' has a '__signature__' attribute - return a copy of it,
> if not - create a new one.
>
> 2. If 'obj' has a '__signature__' attribute - return it,
> if not - create a new one.
>
> 3. Same as '2', but Signature is also immutable.

There's a slight ambiguity there. Do you mean, create a __signature__
attribute, or just create a new Signature instance?

I presume you mean the later, a Signature instance, and not cache it in
__signature__.



--
Steven

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Jun 18, 2012, 6:57 PM

Post #39 of 44 (986 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 2012-06-18, at 9:50 PM, Steven D'Aprano wrote:

> On Mon, Jun 18, 2012 at 07:10:33PM -0400, Yury Selivanov wrote:
>
>> It seems that we have the following options for 'signature(obj)':
>>
>> 1. If 'obj' has a '__signature__' attribute - return a copy of it,
>> if not - create a new one.
>>
>> 2. If 'obj' has a '__signature__' attribute - return it,
>> if not - create a new one.
>>
>> 3. Same as '2', but Signature is also immutable.
>
> There's a slight ambiguity there. Do you mean, create a __signature__
> attribute, or just create a new Signature instance?
>
> I presume you mean the later, a Signature instance, and not cache it in
> __signature__.


Right. No implicit caching to __signature__ by the signature() function.

So, more verbose:

1. If 'obj' has a '__signature__' attribute - return a copy of its value,
if not - create a new Signature and return it.

2. If 'obj' has a '__signature__' attribute - return a copy of its value,
if not - create a new Signature and return it.

3. Same as '2', but Signature is also immutable.

Thanks!
-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Jun 18, 2012, 7:00 PM

Post #40 of 44 (985 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 2012-06-18, at 9:36 PM, Nick Coghlan wrote:

> On Tue, Jun 19, 2012 at 4:09 AM, Yury Selivanov <yselivanov.ml [at] gmail> wrote:
>> On 2012-06-18, at 1:35 PM, PJ Eby wrote:
>>> Then just copy the signature itself; as currently written, this is going to copy the annotation objects, which could produce weird side-effects from introspection. Using deepcopy seems like overkill when all that's needed is a new Signature instance with a fresh OrderedDict.
>>
>> That's an excerpt from Signature.__deepcopy__:
>>
>> cls = type(self)
>> sig = cls.__new__(cls)
>> sig.parameters = OrderedDict((name, param.__copy__()) \
>> for name, param in self.parameters.items())
>>
>> And Parameter.__copy__:
>>
>> cls = type(self)
>> copy = cls.__new__(cls)
>> copy.__dict__.update(self.__dict__)
>> return copy
>>
>> So we don't recursively deepcopy parameters in Signature.__deepcopy__
>> (I hope that we don't violate the deepcopy meaning here)
>
> In my opinion, It's better to redefine what you mean by a shallow copy
> (making it a bit deeper than just the direct attributes) rather than
> making a so-called deep copy shallower.

Agree. That's the only thing about the implementation that I really didn't
like - deepcopy that's not exactly deep.

> So keep the current copying semantics for Signature objects (i.e.
> creating new copies of the Parameter objects as well), but call it a
> shallow copy rather than a deep copy. Make it clear in the
> documentation that any defaults and annotations are still shared with
> the underlying callable.

So, 'Signature.__deepcopy__()' -> 'Signature.shallow_copy()'? Or make
it private - 'Signature._shallow_copy()'?

-
Yury

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steve at pearwood

Jun 18, 2012, 7:02 PM

Post #41 of 44 (1000 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Mon, Jun 18, 2012 at 02:09:17PM -0400, Yury Selivanov wrote:

> That's an excerpt from Signature.__deepcopy__:
>
> cls = type(self)
> sig = cls.__new__(cls)
> sig.parameters = OrderedDict((name, param.__copy__()) \
> for name, param in self.parameters.items())
>
> And Parameter.__copy__:
>
> cls = type(self)
> copy = cls.__new__(cls)
> copy.__dict__.update(self.__dict__)
> return copy
>
> So we don't recursively deepcopy parameters in Signature.__deepcopy__
> (I hope that we don't violate the deepcopy meaning here)


I think you are. I would describe the above as a shallow copy, not a
deep copy. I expect a deep copy to go *all the way down*, as deep as
possible.

Even if it makes no practical difference, I think it will be less
confusing to just describe it as a "copy" rather than a deep copy,
unless you recursively copy everything all the way down.


--
Steven

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Jun 18, 2012, 7:06 PM

Post #42 of 44 (987 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On Tue, Jun 19, 2012 at 12:00 PM, Yury Selivanov
<yselivanov.ml [at] gmail> wrote:
> On 2012-06-18, at 9:36 PM, Nick Coghlan wrote:
>> So keep the current copying semantics for Signature objects (i.e.
>> creating new copies of the Parameter objects as well), but call it a
>> shallow copy rather than a deep copy. Make it clear in the
>> documentation that any defaults and annotations are still shared with
>> the underlying callable.
>
> So, 'Signature.__deepcopy__()' -> 'Signature.shallow_copy()'?  Or make
> it private - 'Signature._shallow_copy()'?

I'd just call it Signature.__copy__ :)

You're not doing anything unusual here, just declaring that the list
of parameters is a part of the Signature object's state, and thus even
a shallow copy shouldn't share the parameter objects.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


yselivanov.ml at gmail

Jun 18, 2012, 7:35 PM

Post #43 of 44 (995 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 2012-06-18, at 10:06 PM, Nick Coghlan wrote:

> On Tue, Jun 19, 2012 at 12:00 PM, Yury Selivanov
> <yselivanov.ml [at] gmail> wrote:
>> On 2012-06-18, at 9:36 PM, Nick Coghlan wrote:
>>> So keep the current copying semantics for Signature objects (i.e.
>>> creating new copies of the Parameter objects as well), but call it a
>>> shallow copy rather than a deep copy. Make it clear in the
>>> documentation that any defaults and annotations are still shared with
>>> the underlying callable.
>>
>> So, 'Signature.__deepcopy__()' -> 'Signature.shallow_copy()'? Or make
>> it private - 'Signature._shallow_copy()'?
>
> I'd just call it Signature.__copy__ :)
>
> You're not doing anything unusual here, just declaring that the list
> of parameters is a part of the Signature object's state, and thus even
> a shallow copy shouldn't share the parameter objects.

OK, done ;)

BTW, http://bugs.python.org/issue15008 has the latest implementation
attached (if anybody wants to play with it)

-
Yury
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


larry at hastings

Jun 19, 2012, 2:43 AM

Post #44 of 44 (974 views)
Permalink
Re: PEP 362: 4th edition [In reply to]

On 06/18/2012 07:35 PM, Yury Selivanov wrote:
> BTW, http://bugs.python.org/issue15008 has the latest implementation
> attached (if anybody wants to play with it)

I've also posted the latest minor tweaks to the PEP, on behalf of Yury.
The new version is already live:

http://www.python.org/dev/peps/pep-0362


Cheers,


//arry/

First page Previous page 1 2 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.