Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Language reference updated for metaclasses

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


ncoghlan at gmail

May 20, 2012, 1:38 AM

Post #1 of 12 (174 views)
Permalink
Language reference updated for metaclasses

When writing the docs for types.new_class(), I discovered that the
description of the class creation process in the language reference
was not only hard to follow, it was actually *incorrect* when it came
to describing the algorithm for determining the correct metaclass.

I rewrote the offending section of the language reference to both
describe the correct algorithm, and hopefully also to be easier to
read. Once people have had a chance to review the changes in the 3.3
docs, I'll backport the update to 3.2.

Previous docs: http://docs.python.org/py3k/reference/datamodel.html#customizing-class-creation
Updated docs: http://docs.python.org/dev/reference/datamodel.html#customizing-class-creation

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


urban.dani+py at gmail

May 20, 2012, 1:56 AM

Post #2 of 12 (169 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

I think there is a small mistake in section "3.3.3.4. Creating the
class object":
"After the class object is created, any class decorators included in
the *function* definition are invoked ..."
That probaly should be "class definition".


Daniel
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


pje at telecommunity

Jun 4, 2012, 2:52 PM

Post #3 of 12 (135 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Sun, May 20, 2012 at 4:38 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:

> When writing the docs for types.new_class(), I discovered that the
> description of the class creation process in the language reference
> was not only hard to follow, it was actually *incorrect* when it came
> to describing the algorithm for determining the correct metaclass.
>
> I rewrote the offending section of the language reference to both
> describe the correct algorithm, and hopefully also to be easier to
> read. Once people have had a chance to review the changes in the 3.3
> docs, I'll backport the update to 3.2.
>
> Previous docs:
> http://docs.python.org/py3k/reference/datamodel.html#customizing-class-creation
> Updated docs:
> http://docs.python.org/dev/reference/datamodel.html#customizing-class-creation
>

This is only sort-of-related, but while reviewing the above, the bit about
__class__ caught my eye and brought this question to mind: how do class
decorators interact with __class__? Specifically, what happens (or more to
the point, is *supposed* to happen and documented as such) if a class
decorator returns a different class object?

PEP 3135 doesn't address this, AFAICT. It refers only to "the class", but
doesn't say whether this is the class-as-returned-by-decorator or original
defined class. (ISTM that it should be the decorated class, since
otherwise this would be different behavior compared to code that explicitly
named the class.)

(Oh, and the rewrite looked good!)


ncoghlan at gmail

Jun 4, 2012, 3:15 PM

Post #4 of 12 (138 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

It's actually the pre-decoration class, since the cell is initialised
before the class is passed to the first decorator. I agree it's a little
weird, but I did try to describe it accurately in the new docs.

--
Sent from my phone, thus the relative brevity :)
On Jun 5, 2012 7:52 AM, "PJ Eby" <pje [at] telecommunity> wrote:

> On Sun, May 20, 2012 at 4:38 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
>
>> When writing the docs for types.new_class(), I discovered that the
>> description of the class creation process in the language reference
>> was not only hard to follow, it was actually *incorrect* when it came
>> to describing the algorithm for determining the correct metaclass.
>>
>> I rewrote the offending section of the language reference to both
>> describe the correct algorithm, and hopefully also to be easier to
>> read. Once people have had a chance to review the changes in the 3.3
>> docs, I'll backport the update to 3.2.
>>
>> Previous docs:
>> http://docs.python.org/py3k/reference/datamodel.html#customizing-class-creation
>> Updated docs:
>> http://docs.python.org/dev/reference/datamodel.html#customizing-class-creation
>>
>
> This is only sort-of-related, but while reviewing the above, the bit about
> __class__ caught my eye and brought this question to mind: how do class
> decorators interact with __class__? Specifically, what happens (or more to
> the point, is *supposed* to happen and documented as such) if a class
> decorator returns a different class object?
>
> PEP 3135 doesn't address this, AFAICT. It refers only to "the class", but
> doesn't say whether this is the class-as-returned-by-decorator or original
> defined class. (ISTM that it should be the decorated class, since
> otherwise this would be different behavior compared to code that explicitly
> named the class.)
>
> (Oh, and the rewrite looked good!)
>
>
>


pje at telecommunity

Jun 4, 2012, 3:58 PM

Post #5 of 12 (135 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Mon, Jun 4, 2012 at 6:15 PM, Nick Coghlan <ncoghlan [at] gmail> wrote:

> It's actually the pre-decoration class, since the cell is initialised
> before the class is passed to the first decorator. I agree it's a little
> weird, but I did try to describe it accurately in the new docs.
>
I see that now; it might be helpful to explicitly call that out.

This is adding to my list of Python 3 metaclass gripes, though. In Python
2, I have in-the-body-of-a-class decorators implemented using metaclasses,
that will no longer work because of PEP 3115... and if I switch to using
class decorators instead, then they won't work because of PEP 3135. :-(

Meanwhile, mixing metaclasses is more difficult than ever, due to
__prepare__, and none of these flaws can be worked around officially,
because __build_class__ is an "implementation detail". I *really* want to
like Python 3, but am still hoping against hope for the restoration of
hooks that __metaclass__ allowed, or some alternative mechanism that would
serve the same use cases.

Specifically, my main use case is method-level decorators and attribute
descriptors that need to interact with a user-defined class, *without*
requiring that user-defined class to either 1) redundantly decorate the
class or 2) inherit from some specific base or inject a specific
metaclass. I only use __metaclass__ in 2.x for this because it's the only
way for code executed in a class body to gain access to the class at
creation time.

The reason for wanting this to be transparent is that 1) if you forget the
redundant class-decorator, mixin, or metaclass, stuff will silently not
work, and 2) mixing bases or metaclasses has much higher coupling to the
library providing the decorators or descriptors, and greatly increases the
likelihood of mixing metaclasses.

And at the moment, the only workaround I can come up with that *doesn't*
involve replacing __build_class__ is abusing the system trace hook; ISTM
that replacing __build_class__ is the better of those two options.

At this point, with the additions of types.new_class(), ISTM that every
Python implementation will have to *have* a __build_class__ function or its
equivalent; all that remains is the question of whether they allow
*replacing* it.


eliben at gmail

Jun 4, 2012, 8:24 PM

Post #6 of 12 (128 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Sun, May 20, 2012 at 10:38 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
> When writing the docs for types.new_class(), I discovered that the
> description of the class creation process in the language reference
> was not only hard to follow, it was actually *incorrect* when it came
> to describing the algorithm for determining the correct metaclass.
>
> I rewrote the offending section of the language reference to both
> describe the correct algorithm, and hopefully also to be easier to
> read. Once people have had a chance to review the changes in the 3.3
> docs, I'll backport the update to 3.2.
>
> Previous docs: http://docs.python.org/py3k/reference/datamodel.html#customizing-class-creation
> Updated docs: http://docs.python.org/dev/reference/datamodel.html#customizing-class-creation
>

"if an explicit metaclass is given and it is not an instance of
type(), then it is used directly as the metaclass"

Could you elaborate on this point? Would it perhaps be clearer to say
"if an explicit metaclass is given and it is not a class"?

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Jun 5, 2012, 12:18 AM

Post #7 of 12 (128 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Tue, Jun 5, 2012 at 1:24 PM, Eli Bendersky <eliben [at] gmail> wrote:
>
> "if an explicit metaclass is given and it is not an instance of
> type(), then it is used directly as the metaclass"
>
> Could you elaborate on this point? Would it perhaps be clearer to say
> "if an explicit metaclass is given and it is not a class"?

Unfortunately, the term "a class" is slightly ambiguous. "cls is a
class" can mean either "isinstance(cls, type)" or it can be shorthand
for "cls is a user-defined class (i.e. not a builtin type)".

I'll likely rephrase that section of the docs based on the current
discussion with PJE, though.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


eliben at gmail

Jun 5, 2012, 12:20 AM

Post #8 of 12 (128 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Tue, Jun 5, 2012 at 10:18 AM, Nick Coghlan <ncoghlan [at] gmail> wrote:
> On Tue, Jun 5, 2012 at 1:24 PM, Eli Bendersky <eliben [at] gmail> wrote:
>>
>> "if an explicit metaclass is given and it is not an instance of
>> type(), then it is used directly as the metaclass"
>>
>> Could you elaborate on this point? Would it perhaps be clearer to say
>> "if an explicit metaclass is given and it is not a class"?
>
> Unfortunately, the term "a class" is slightly ambiguous. "cls is a
> class" can mean either "isinstance(cls, type)" or it can be shorthand
> for "cls is a user-defined class (i.e. not a builtin type)".
>

Yes, confusing it is
(http://eli.thegreenplace.net/2012/03/30/python-objects-types-classes-and-instances-a-glossary/)

Still, instance of type()" is a bit too cryptic for mere mortals, IMHO.

Eli
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


steve at pearwood

Jun 5, 2012, 12:58 AM

Post #9 of 12 (130 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Tue, Jun 05, 2012 at 10:20:58AM +0300, Eli Bendersky wrote:

> Still, instance of type()" is a bit too cryptic for mere mortals, IMHO.

I think that if somebody finds "instance of type" too cryptic, they
won't have any chance at all to understand metaclasses.

Personally, I think there is a lot confusing about metaclasses, but the
idea that classes are instances (objects) is not one of them.



--
Steven
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


mark at hotpy

Jun 5, 2012, 1:34 AM

Post #10 of 12 (128 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

Steven D'Aprano wrote:
> On Tue, Jun 05, 2012 at 10:20:58AM +0300, Eli Bendersky wrote:
>
>> Still, instance of type()" is a bit too cryptic for mere mortals, IMHO.
>
> I think that if somebody finds "instance of type" too cryptic, they
> won't have any chance at all to understand metaclasses.
>
> Personally, I think there is a lot confusing about metaclasses, but the
> idea that classes are instances (objects) is not one of them.
>

One thing that *is* confusing is that the metaclass parameter in class
creation is not the metaclass (class of the class), but the class
factory. For example:

def silly(*args):
print(*args)
return int

class C(metaclass=silly):
def m(self): pass

C () {'m': <function C.m at 0xb721fe14>, '__qualname__': 'C',
'__module__': '__main__'}

print(C)
int

In this example the metaclass (ie the class of C) is type (C is int),
even though the declared metaclass is 'silly'.

I assume it is too late to change the name of the 'metaclass' keyword to
'factory', but we could use that terminology in the docs.

Cheers,
Mark
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fuzzyman at voidspace

Jun 5, 2012, 1:55 AM

Post #11 of 12 (124 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On 5 Jun 2012, at 09:34, Mark Shannon wrote:

> Steven D'Aprano wrote:
>> On Tue, Jun 05, 2012 at 10:20:58AM +0300, Eli Bendersky wrote:
>>> Still, instance of type()" is a bit too cryptic for mere mortals, IMHO.
>> I think that if somebody finds "instance of type" too cryptic, they won't have any chance at all to understand metaclasses.
>> Personally, I think there is a lot confusing about metaclasses, but the idea that classes are instances (objects) is not one of them.
>
> One thing that *is* confusing is that the metaclass parameter in class creation is not the metaclass (class of the class), but the class factory. For example:
>
> def silly(*args):
> print(*args)
> return int
>
> class C(metaclass=silly):
> def m(self): pass
>
> C () {'m': <function C.m at 0xb721fe14>, '__qualname__': 'C', '__module__': '__main__'}
>
> print(C)
> int
>
> In this example the metaclass (ie the class of C) is type (C is int),
> even though the declared metaclass is 'silly'.
>
> I assume it is too late to change the name of the 'metaclass' keyword to 'factory', but we could use that terminology in the docs.


Well, the same was always true in Python 2 as well - __metaclass__ could be a function that was identically "silly". The real "metaclass" (type of the class) is whatever you use to construct the class.

Michael

>
> Cheers,
> Mark
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing
http://www.sqlite.org/different.html





_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ncoghlan at gmail

Jun 5, 2012, 3:20 AM

Post #12 of 12 (127 views)
Permalink
Re: Language reference updated for metaclasses [In reply to]

On Tue, Jun 5, 2012 at 6:34 PM, Mark Shannon <mark [at] hotpy> wrote:
> In this example the metaclass (ie the class of C) is type (C is int),
> even though the declared metaclass is 'silly'.
>
> I assume it is too late to change the name of the 'metaclass' keyword to
> 'factory', but we could use that terminology in the docs.

"factory" is also wrong (since a more derived metaclass from a base
class may be used instead). "metaclass_hint" or "requested_metaclass"
would be more accurate names - as in Python 2, the value provided in
the class definition is only one input to the algorithm that
determines the metaclass (which is now correctly described in the
language reference), rather than a simple factory function or direct
specification of __class__.

That slightly blurry terminology isn't new in Python 3 though, it's
been around for pretty much as long as Python has supported
metaclasses.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.