Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Zope: Dev

zope.catalog in ZTK

 

 

Zope dev RSS feed   Index | Next | Previous | View Threaded


alga at pov

Sep 27, 2009, 4:16 AM

Post #1 of 8 (891 views)
Permalink
zope.catalog in ZTK

Hi all,

zope.catalog is a crucial part of ZTK, as it adds the equivalent of
RDBMS SELECT queries to our ODB based apps. However in my opinion its
default setup is suboptimal as a generic library in ZTK, because it is
hardwired up for ZMI "point and click" operation. As soon as you
create and register an ICatalog utility, every ObjectAddedEvent and
ObjectModifiedEvent will trigger stuffing the object into intids, and
each intid registration event will trigger an attempt to index an
object. When you add an index to the catalog, it gets reindexed
automatically. Because these behaviours are implemented by event
handlers, there is no way for an app to override or disable them.

There is an "opt out" mechanism for keeping objects out of the catalog
(INoAutoIndex, INoAutoReindex marker interfaces), but curiously the
latter does not inherit from the former,so your object can be indexed
on modification even if you made it provide INoAutoIndex.

Automatically stuffing all newly added objects into the int ids
utility and then further into the catalog is bad, because, first, it
cripples any alternative uses of the int ids utility, turns it into
purely a catalog accessory, and second, places some restrictions on
containers and contained objects due to the persistent oid based key
reference implementation.

About a year ago zope.app.catalog as been moved to zope.catalog. I
think during this move there was a unique opportunity to leave these
event handlers behind in zope.app.catalog, so that the "no .app"
version is free of these forced choices. But perhaps it's not too
late to do in now? After all, no stable version of Zope 3 / ZTK
codebase has been released after this move? People relying on
automatic indexing would have to register a couple of event handlers
in their configurations (or include zope.app.catalog), but the rest of
us would be able to choose the policy of how and when the objects are
registered with IntIds and indexed in the catalog.

Albertas
--
http://www.pov.lt/ -- Zope and Python consulting
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


chrism at plope

Sep 27, 2009, 10:18 AM

Post #2 of 8 (841 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

We teased out all this ZMI- and event-handling related behavior in
repoze.catalog. r.catalog consumes no events, and disuses the IIntIds utility.
It's otherwise a lot like zope.catalog. It might be a good starting point if
you want more control.

FWIW, we use this code heavily in several production apps, it's not a "throw it
over the wall" sort of things. It has good test coverage.

http://pypi.python.org/pypi/repoze.catalog/0.7.1


Albertas Agejevas wrote:
> Hi all,
>
> zope.catalog is a crucial part of ZTK, as it adds the equivalent of
> RDBMS SELECT queries to our ODB based apps. However in my opinion its
> default setup is suboptimal as a generic library in ZTK, because it is
> hardwired up for ZMI "point and click" operation. As soon as you
> create and register an ICatalog utility, every ObjectAddedEvent and
> ObjectModifiedEvent will trigger stuffing the object into intids, and
> each intid registration event will trigger an attempt to index an
> object. When you add an index to the catalog, it gets reindexed
> automatically. Because these behaviours are implemented by event
> handlers, there is no way for an app to override or disable them.
>
> There is an "opt out" mechanism for keeping objects out of the catalog
> (INoAutoIndex, INoAutoReindex marker interfaces), but curiously the
> latter does not inherit from the former,so your object can be indexed
> on modification even if you made it provide INoAutoIndex.
>
> Automatically stuffing all newly added objects into the int ids
> utility and then further into the catalog is bad, because, first, it
> cripples any alternative uses of the int ids utility, turns it into
> purely a catalog accessory, and second, places some restrictions on
> containers and contained objects due to the persistent oid based key
> reference implementation.
>
> About a year ago zope.app.catalog as been moved to zope.catalog. I
> think during this move there was a unique opportunity to leave these
> event handlers behind in zope.app.catalog, so that the "no .app"
> version is free of these forced choices. But perhaps it's not too
> late to do in now? After all, no stable version of Zope 3 / ZTK
> codebase has been released after this move? People relying on
> automatic indexing would have to register a couple of event handlers
> in their configurations (or include zope.app.catalog), but the rest of
> us would be able to choose the policy of how and when the objects are
> registered with IntIds and indexed in the catalog.
>
> Albertas

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


faassen at startifact

Sep 29, 2009, 4:09 AM

Post #3 of 8 (832 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

Hi there,

Albertas Agejevas wrote:
[snip]
> About a year ago zope.app.catalog as been moved to zope.catalog. I
> think during this move there was a unique opportunity to leave these
> event handlers behind in zope.app.catalog, so that the "no .app"
> version is free of these forced choices.

Doing this would have needed a migration strategy more complicated than
"Oh, I'm just going to lose the ZMI when I switch" and people's
applications somehow would have to gain registrations manually. I guess
a damn good changelog would've helped somewhat..

In general, I think it's not too bad we lose the ZMI in a step
separately from other policies. Changing too much in one step could be
more disruption than the community can manage.

> But perhaps it's not too
> late to do in now? After all, no stable version of Zope 3 / ZTK
> codebase has been released after this move? People relying on
> automatic indexing would have to register a couple of event handlers
> in their configurations (or include zope.app.catalog), but the rest of
> us would be able to choose the policy of how and when the objects are
> registered with IntIds and indexed in the catalog.

Would you propose the same treatment for zope.intid as well?

Would this involve removing the configure.zcml in these packages, or
what is your proposal exactly?

Again, it needs damn good documentation if we were to go ahead with
this, including details on how to register event handlers to accomplish
the previous behavior. Both the changelog and the documentation that
appears on pypi would need to be updated with instructions about what is
going on. This change would also definitely be worth a feature release.

+1 if documentation is good.

Regards,

Martijn

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


hanno at hannosch

Sep 29, 2009, 4:33 AM

Post #4 of 8 (841 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

On Sun, Sep 27, 2009 at 1:16 PM, Albertas Agejevas <alga [at] pov> wrote:
> About a year ago zope.app.catalog as been moved to zope.catalog.  I
> think during this move there was a unique opportunity to leave these
> event handlers behind in zope.app.catalog, so that the "no .app"
> version is free of these forced choices.  But perhaps it's not too
> late to do in now?  After all, no stable version of Zope 3 / ZTK
> codebase has been released after this move?  People relying on
> automatic indexing would have to register a couple of event handlers
> in their configurations (or include zope.app.catalog), but the rest of
> us would be able to choose the policy of how and when the objects are
> registered with IntIds and indexed in the catalog.

Is there any reason you don't just ignore the configure.zcml in that package?

Nobody forces you to load the zcml file. The whole point of having
these policies in zcml is to be able to ignore or override them
easily.

Hanno
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


alga at pov

Sep 29, 2009, 5:11 AM

Post #5 of 8 (837 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

On Tue, Sep 29, 2009 at 01:09:08PM +0200, Martijn Faassen wrote:
> > About a year ago zope.app.catalog as been moved to zope.catalog. I
> > think during this move there was a unique opportunity to leave these
> > event handlers behind in zope.app.catalog, so that the "no .app"
> > version is free of these forced choices.
>
> Doing this would have needed a migration strategy more complicated than
> "Oh, I'm just going to lose the ZMI when I switch" and people's
> applications somehow would have to gain registrations manually. I guess
> a damn good changelog would've helped somewhat..

Correct me if I'm wrong, but zope.app.catalog still remains with the
ZMI bits and re-imports of the zope.catalog functionality. So apps
depending on zope.app.catalog are not affected at all. I'm saying
that some of the event handlers (or their ZCML registrations) could
have stayed there as well.

> > But perhaps it's not too
> > late to do in now? After all, no stable version of Zope 3 / ZTK
> > codebase has been released after this move? People relying on
> > automatic indexing would have to register a couple of event handlers
> > in their configurations (or include zope.app.catalog), but the rest of
> > us would be able to choose the policy of how and when the objects are
> > registered with IntIds and indexed in the catalog.
>
> Would you propose the same treatment for zope.intid as well?

I don't know, the subscribers in zope.intid don't bother me too much.

> Would this involve removing the configure.zcml in these packages, or
> what is your proposal exactly?

In essence, I would like to see zope/catalog/subscribers.zcml removed
from the default zope/catalog/configure.zcml.

> Again, it needs damn good documentation if we were to go ahead with
> this, including details on how to register event handlers to accomplish
> the previous behavior. Both the changelog and the documentation that
> appears on pypi would need to be updated with instructions about what is
> going on. This change would also definitely be worth a feature release.
>
> +1 if documentation is good.

:D

Albertas
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


alga at pov

Sep 29, 2009, 5:22 AM

Post #6 of 8 (837 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

On Tue, Sep 29, 2009 at 01:33:21PM +0200, Hanno Schlichting wrote:
> Is there any reason you don't just ignore the configure.zcml in that package?
>
> Nobody forces you to load the zcml file. The whole point of having
> these policies in zcml is to be able to ignore or override them
> easily.

Good point, thanks! I haven't looked at the configure.zcml for
zope.catalog previously. In fact the config for zope.catalog is split
into several files, so it's very easy to leave out the subscribers but
include the rest.

Albertas
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


faassen at startifact

Sep 29, 2009, 7:08 AM

Post #7 of 8 (839 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

Albertas Agejevas wrote:
> On Tue, Sep 29, 2009 at 01:09:08PM +0200, Martijn Faassen wrote:
>>> About a year ago zope.app.catalog as been moved to zope.catalog. I
>>> think during this move there was a unique opportunity to leave these
>>> event handlers behind in zope.app.catalog, so that the "no .app"
>>> version is free of these forced choices.
>> Doing this would have needed a migration strategy more complicated than
>> "Oh, I'm just going to lose the ZMI when I switch" and people's
>> applications somehow would have to gain registrations manually. I guess
>> a damn good changelog would've helped somewhat..
>
> Correct me if I'm wrong, but zope.app.catalog still remains with the
> ZMI bits and re-imports of the zope.catalog functionality. So apps
> depending on zope.app.catalog are not affected at all. I'm saying
> that some of the event handlers (or their ZCML registrations) could
> have stayed there as well.

We hope to deprecate zope.app.catalog entirely eventually, so we should
support people as much as possible in moving over to zope.catalog. Just
removing the ZMI stuff is a relatively straightforward deprecation step.
Removing configuration essential to the functioning of many applications
is less so, which is why I'm saying it's not bad we do this in two steps.

>> Would this involve removing the configure.zcml in these packages, or
>> what is your proposal exactly?
>
> In essence, I would like to see zope/catalog/subscribers.zcml removed
> from the default zope/catalog/configure.zcml.

Ah, yeah, that would make documenting things easier too: just include
zope/catalog/subscribers.zcml if you them.

Regards,

Martijn

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


tseaver at palladion

Sep 30, 2009, 7:26 AM

Post #8 of 8 (800 views)
Permalink
Re: zope.catalog in ZTK [In reply to]

Albertas Agejevas wrote:

> People relying on
> automatic indexing would have to register a couple of event handlers
> in their configurations (or include zope.app.catalog), but the rest of
> us would be able to choose the policy of how and when the objects are
> registered with IntIds and indexed in the catalog.

+1 from me: reusable library code shouldn't be imposing policy on the
application.


Tres.
--
===================================================================
Tres Seaver +1 540-429-0999 tseaver [at] palladion
Palladion Software "Excellence by Design" http://palladion.com

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
https://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )

Zope dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.