Mailing List Archive: Zope: CMF

What is the status of GS wiping catalog indexes on catalog.xml import?

Index | Next | Previous | View Threaded

matt.halstead at gmail

Feb 25, 2008, 11:45 PM

Post #1 of 25 (1066 views)
Permalink

What is the status of GS wiping catalog indexes on catalog.xml import?

The bug/feature referenced here https://bugs.launchpad.net/zope-cmf/+bug/161682
relates to the wiping of indexes in the ZCatalog when a GS step for
the catalog is run.

I was wondering if there is a targeted release for this fix(feature).
It has a huge impact on site migrations that require reloading of
profiles to update configuration states.

cheers
Matt
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

jens at dataflake

Feb 25, 2008, 11:49 PM

Post #2 of 25 (1040 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

On Feb 26, 2008, at 08:45 , greenman wrote:

> The bug/feature referenced here https://bugs.launchpad.net/zope-cmf/+bug/161682
> relates to the wiping of indexes in the ZCatalog when a GS step for
> the catalog is run.
>
> I was wondering if there is a targeted release for this fix(feature).
> It has a huge impact on site migrations that require reloading of
> profiles to update configuration states.

I'm afraid there is no specific release target for this feature
request. As you can see from the bug history a set of patches were
proposed which only led to more questions, and the discussion has
stopped there for the moment.

jens

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

wichert at wiggy

Feb 26, 2008, 2:16 AM

Post #3 of 25 (1040 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Previously greenman wrote:
> The bug/feature referenced here https://bugs.launchpad.net/zope-cmf/+bug/161682
> relates to the wiping of indexes in the ZCatalog when a GS step for
> the catalog is run.
>
> I was wondering if there is a targeted release for this fix(feature).
> It has a huge impact on site migrations that require reloading of
> profiles to update configuration states.

As documented in that issue until the catalog index API is changed and
all indexes have been updated to support the new API that bug is
impossible to fix. So far there do not seem to be any volunteers to do
that.

Wichert.

--
Wichert Akkerman <wichert[at]wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

matt.halstead at gmail

Feb 27, 2008, 1:19 PM

Post #4 of 25 (1032 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

I was wondering if there was a simpler solution. The proposal is
oriented towards testing current state to the proposed new state for
individual indexes. This avoids the need for the profile declarations
to assume a present state of the catalog. But looking at other parts
of GS - e.g. viewlet managers where you can ask for a hidden flag to
be removed, and object node handlers where you can ask for a specific
object to be removed with remove="True", then it seems that GS is
quite entrenched in creating state transitions that are based on
actions for an assumed current state. I guess this is the only way to
manage extension profiles which modify parts of state and do not
reflect the entire state of the system and which are applied linearly
over perhaps a set of profiles.

So, for the catalog.xml importer, why can't the trigger for reindexing
an index be a flag on the catalog index declaration itself? Is it
really generic setups role to determine if changes to an index
invalidate the values it already holds? If you were to change
properties of an index through code and not GS, then it would be up to
you whether you reindexed all your objects or not.

Perhaps the catalog should have an event listener for index invalidate
events and accumulate these until code or human causes all invalidated
indexes to be reindexed. That way, some catalog API level calls that
change certain properties of an index that require reindexing could
trigger their own invalidate events without some external code (such
as GS) trying to guess whether or not it needs to force a re-index.

On Feb 26, 11:16�pm, Wichert Akkerman <wich...@wiggy.net> wrote:
> Previously greenman wrote:
> > The bug/feature referenced herehttps://bugs.launchpad.net/zope-cmf/+bug/161682
> > relates to the wiping of indexes in the ZCatalog when a GS step for
> > the catalog is run.
>
> > I was wondering if there is a targeted release for this fix(feature).
> > It has a huge impact on site migrations that require reloading of
> > profiles to update configuration states.
>
> As documented in that issue until the catalog index API is changed and
> all indexes have been updated to support the new API that bug is
> impossible to fix. So far there do not seem to be any volunteers to do
> that.
>
> Wichert.
>
> --
> Wichert Akkerman <wich...@wiggy.net> � �It is simple to make things.http://www.wiggy.net/� � � � � � � � � It is hard to make things simple.
> _______________________________________________
> Zope-CMF maillist �- �Zope-...@lists.zope.orghttp://mail.zope.org/mailman/listinfo/zope-cmf
>
> Seehttp://collector.zope.org/CMFfor bug reports and feature requests
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

m.van.rees at zestsoftware

Feb 27, 2008, 1:59 PM

Post #5 of 25 (1033 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

greenman, on 2008-02-27:
> So, for the catalog.xml importer, why can't the trigger for reindexing
> an index be a flag on the catalog index declaration itself? Is it
> really generic setups role to determine if changes to an index
> invalidate the values it already holds? If you were to change
> properties of an index through code and not GS, then it would be up to
> you whether you reindexed all your objects or not.

The problem is that GenericSetup does not know if your current index
is of the same type and has the same settings/properties as the index
that you specify in catalog.xml. Apparently it is hard/impossible to
reliably compare the existing and the wanted index. So GenericSetup
has no choice but to remove the existing index and make a new one.

--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

matt.halstead at gmail

Feb 27, 2008, 4:03 PM

Post #6 of 25 (1027 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

On Feb 28, 10:59�am, Maurits van Rees <m.van.r...@zestsoftware.nl>
wrote:
> greenman, on 2008-02-27:
>
> > So, for the catalog.xml importer, why can't the trigger for reindexing
> > an index be a flag on the catalog index declaration itself? Is it
> > really generic setups role to determine if changes to an index
> > invalidate the values it already holds? If you were to change
> > properties of an index through code and not GS, then it would be up to
> > you whether you reindexed all your objects or not.
>
> The problem is that GenericSetup does not know if your current index
> is of the same type and has the same settings/properties as the index
> that you specify in catalog.xml. �Apparently it is hard/impossible to
> reliably compare the existing and the wanted index. �So GenericSetup
> has no choice but to remove the existing index and make a new one.
>

Can't that be wrapped in a try/except? The most common case is that
the index will have the attributes that GS is trying to set. As for
type, then again, given the prevalence of actions in GS that modify an
assumed current state - then a flag 'recreate' would allow for a type
change to be pushed through.

> --
> Maurits van Rees |http://maurits.vanrees.org/
> � � � � � � Work |http://zestsoftware.nl/
> "This is your day, don't let them take it away." [Barlow Girl]
>
> _______________________________________________
> Zope-CMF maillist �- �Zope-...@lists.zope.orghttp://mail.zope.org/mailman/listinfo/zope-cmf
>
> Seehttp://collector.zope.org/CMFfor bug reports and feature requests
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

lists at zopyx

Feb 27, 2008, 10:13 PM

Post #7 of 25 (1027 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 27. Februar 2008 21:59:58 +0000 Maurits van Rees
<m.van.rees[at]zestsoftware.nl> wrote:

> greenman, on 2008-02-27:
>> So, for the catalog.xml importer, why can't the trigger for reindexing
>> an index be a flag on the catalog index declaration itself? Is it
>> really generic setups role to determine if changes to an index
>> invalidate the values it already holds? If you were to change
>> properties of an index through code and not GS, then it would be up to
>> you whether you reindexed all your objects or not.
>
> The problem is that GenericSetup does not know if your current index
> is of the same type and has the same settings/properties as the index
> that you specify in catalog.xml. Apparently it is hard/impossible to
> reliably compare the existing and the wanted index. So GenericSetup
> has no choice but to remove the existing index and make a new one.
>
>

How about the following idea:

- within the Zope core we define an _optional_ interface for indexes -
something like:

class IIndexConfiguration(Interface):

def getConfiguration():
""" Returns a dict with index specific configuration
parameters.
"""

- on the CMF/GS side we could register adapter for each index type
we know (basically the Zope 2 core indexes, ExtendedPathIndex,
TextIndexNG 3) and retrieve the related information

- the related GS asks each index for its configuration and takes
appropriate action based on the comparison of the values from the
profile and the existing index.

Adding the interface to Zope 2.11 or backporting it to Zope 2.10
would not be a problem. Since the Zope 2.11 branch is offically closed for
new features, the index specific implementations of IIndexConfiguration
should be implemented outside the Zope core but we might move the
implementation into the Zope core with Zope 2.12. Sounds reasonable?

Andreas

y.2008 at wcm-solutions

Feb 28, 2008, 12:38 AM

Post #8 of 25 (1027 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Hi!

Andreas Jung wrote:
> How about the following idea:
>
> - within the Zope core we define an _optional_ interface for indexes -
> something like:
>
> class IIndexConfiguration(Interface):
>
> def getConfiguration():
> """ Returns a dict with index specific configuration
> parameters.
> """
>
> - on the CMF/GS side we could register adapter for each index type
> we know (basically the Zope 2 core indexes, ExtendedPathIndex,
> TextIndexNG 3) and retrieve the related information
>
> - the related GS asks each index for its configuration and takes
> appropriate action based on the comparison of the values from the
> profile and the existing index.
>
> Adding the interface to Zope 2.11 or backporting it to Zope 2.10
> would not be a problem. Since the Zope 2.11 branch is offically closed
> for new features, the index specific implementations of
> IIndexConfiguration
> should be implemented outside the Zope core but we might move the
> implementation into the Zope core with Zope 2.12. Sounds reasonable?

Yes. textindexng already has getSettings(), I used it for the
export/import code of TextIndexNG3:

http://textindexng.svn.sourceforge.net/viewvc/textindexng/TextIndexNG3/trunk/exportimport.py?revision=1833&view=markup

That code compares the settings and makes sure TextIndexNG3 is only
modified and cleared if necessary. We could use similar code for all
other indexes if they would provide a method like that.

I'd prefer a IConfigurableIndex interface that also has a set method.

Cheers,

Yuppie

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

dieter at handshake

Feb 28, 2008, 11:35 AM

Post #9 of 25 (1023 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Andreas Jung wrote at 2008-2-28 07:13 +0100:
>--On 27. Februar 2008 21:59:58 +0000 Maurits van Rees
><m.van.rees[at]zestsoftware.nl> wrote:
>
>> greenman, on 2008-02-27:
>>> So, for the catalog.xml importer, why can't the trigger for reindexing
>>> an index be a flag on the catalog index declaration itself? Is it
>>> really generic setups role to determine if changes to an index
>>> invalidate the values it already holds? If you were to change
>>> properties of an index through code and not GS, then it would be up to
>>> you whether you reindexed all your objects or not.
>>
>> The problem is that GenericSetup does not know if your current index
>> is of the same type and has the same settings/properties as the index
>> that you specify in catalog.xml. Apparently it is hard/impossible to
>> reliably compare the existing and the wanted index. So GenericSetup
>> has no choice but to remove the existing index and make a new one.
>>
>>
>
>How about the following idea:
>
> - within the Zope core we define an _optional_ interface for indexes -
> something like:
>
> class IIndexConfiguration(Interface):
>
> def getConfiguration():
> """ Returns a dict with index specific configuration
> parameters.
> """
>
> - on the CMF/GS side we could register adapter for each index type
> we know (basically the Zope 2 core indexes, ExtendedPathIndex,
> TextIndexNG 3) and retrieve the related information

I do not understand why something like this should be necessary.

When the export handler is able to extract all relevant configuration
parameters for an index, why should the import handler
not be able to check the configuration parameters in a profile
against an existing index and determine that it needs to do nothing?

--
Dieter
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

lists at zopyx

Feb 28, 2008, 9:43 PM

Post #10 of 25 (1017 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 28. Februar 2008 20:35:09 +0100 Dieter Maurer <dieter[at]handshake.de>
wrote:

> Andreas Jung wrote at 2008-2-28 07:13 +0100:
>> --On 27. Februar 2008 21:59:58 +0000 Maurits van Rees
>> <m.van.rees[at]zestsoftware.nl> wrote:
>>
>>> greenman, on 2008-02-27:
>>>> So, for the catalog.xml importer, why can't the trigger for reindexing
>>>> an index be a flag on the catalog index declaration itself? Is it
>>>> really generic setups role to determine if changes to an index
>>>> invalidate the values it already holds? If you were to change
>>>> properties of an index through code and not GS, then it would be up to
>>>> you whether you reindexed all your objects or not.
>>>
>>> The problem is that GenericSetup does not know if your current index
>>> is of the same type and has the same settings/properties as the index
>>> that you specify in catalog.xml. Apparently it is hard/impossible to
>>> reliably compare the existing and the wanted index. So GenericSetup
>>> has no choice but to remove the existing index and make a new one.
>>>
>>>
>>
>> How about the following idea:
>>
>> - within the Zope core we define an _optional_ interface for indexes -
>> something like:
>>
>> class IIndexConfiguration(Interface):
>>
>> def getConfiguration():
>> """ Returns a dict with index specific configuration
>> parameters.
>> """
>>
>> - on the CMF/GS side we could register adapter for each index type
>> we know (basically the Zope 2 core indexes, ExtendedPathIndex,
>> TextIndexNG 3) and retrieve the related information
>
> I do not understand why something like this should be necessary.
>
> When the export handler is able to extract all relevant configuration
> parameters for an index, why should the import handler
> not be able to check the configuration parameters in a profile
> against an existing index and determine that it needs to do nothing?

Huh? Because we're looking for a clean solution and not for a hack!
Because this solution is extensible. An index can provide the introspection
directly or another application could implement the functionality as needed
through adaption. Having something hardcoded for each index type within the
setuphandlers is a hacker solution. And the setuphandler should ideally not
know about index internals.

Andreas

optilude at gmx

Feb 29, 2008, 12:39 AM

Post #11 of 25 (1020 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Andreas Jung wrote:
>
> --On 28. Februar 2008 20:35:09 +0100 Dieter Maurer <dieter[at]handshake.de>
> wrote:
>
>> Andreas Jung wrote at 2008-2-28 07:13 +0100:
>>> --On 27. Februar 2008 21:59:58 +0000 Maurits van Rees
>>> <m.van.rees[at]zestsoftware.nl> wrote:
>>>
>>>> greenman, on 2008-02-27:
>>>>> So, for the catalog.xml importer, why can't the trigger for reindexing
>>>>> an index be a flag on the catalog index declaration itself? Is it
>>>>> really generic setups role to determine if changes to an index
>>>>> invalidate the values it already holds? If you were to change
>>>>> properties of an index through code and not GS, then it would be up to
>>>>> you whether you reindexed all your objects or not.
>>>> The problem is that GenericSetup does not know if your current index
>>>> is of the same type and has the same settings/properties as the index
>>>> that you specify in catalog.xml. Apparently it is hard/impossible to
>>>> reliably compare the existing and the wanted index. So GenericSetup
>>>> has no choice but to remove the existing index and make a new one.
>>>>
>>>>
>>> How about the following idea:
>>>
>>> - within the Zope core we define an _optional_ interface for indexes -
>>> something like:
>>>
>>> class IIndexConfiguration(Interface):
>>>
>>> def getConfiguration():
>>> """ Returns a dict with index specific configuration
>>> parameters.
>>> """
>>>
>>> - on the CMF/GS side we could register adapter for each index type
>>> we know (basically the Zope 2 core indexes, ExtendedPathIndex,
>>> TextIndexNG 3) and retrieve the related information
>> I do not understand why something like this should be necessary.
>>
>> When the export handler is able to extract all relevant configuration
>> parameters for an index, why should the import handler
>> not be able to check the configuration parameters in a profile
>> against an existing index and determine that it needs to do nothing?
>
> Huh? Because we're looking for a clean solution and not for a hack!
> Because this solution is extensible. An index can provide the introspection
> directly or another application could implement the functionality as needed
> through adaption. Having something hardcoded for each index type within the
> setuphandlers is a hacker solution. And the setuphandler should ideally not
> know about index internals.

Respectfully, I'd say that if we can have a hacky solution today that
doesn't wipe indexes on re-install, then that's very valuable. The set
of standard index types is well known and doesn't change much.

Of course, we should also provide a way to get an interface or something
else describing the configuration for introspection purposes. Waiting
one or two Zope versions for that to get a non-purging GS import handler
when there's a works-90%-of-the-time solution (falling back on current
behaviour) would be a shame though.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

lists at zopyx

Feb 29, 2008, 1:00 AM

Post #12 of 25 (1019 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 29. Februar 2008 08:39:05 +0000 Martin Aspeli <optilude[at]gmx.net> wrote:

>
> Of course, we should also provide a way to get an interface or something
> else describing the configuration for introspection purposes. Waiting one
> or two Zope versions for that to get a non-purging GS import handler when
> there's a works-90%-of-the-time solution (falling back on current
> behaviour) would be a shame though.
>

You don't have to wait for for new Zope versions. Defining the interface
officially in Zope 2.10, 2.11 and trunk will raise no problems. I have no
problems with putting this into the official release branches - or?

Andreas

jens at dataflake

Feb 29, 2008, 2:17 AM

Post #13 of 25 (1019 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

On Feb 29, 2008, at 10:00 , Andreas Jung wrote:

>
>
> --On 29. Februar 2008 08:39:05 +0000 Martin Aspeli
> <optilude[at]gmx.net> wrote:
>
>
>>
>> Of course, we should also provide a way to get an interface or
>> something
>> else describing the configuration for introspection purposes.
>> Waiting one
>> or two Zope versions for that to get a non-purging GS import
>> handler when
>> there's a works-90%-of-the-time solution (falling back on current
>> behaviour) would be a shame though.
>>
>
> You don't have to wait for for new Zope versions. Defining the
> interface
> officially in Zope 2.10, 2.11 and trunk will raise no problems. I
> have no problems with putting this into the official release
> branches - or?

My personal opinion: I'd rather see the interface-based solution in a
few weeks or a couple months (the next Zope release) than the, umh,
less-than-professional solution that will stick around forever. As
such solutions have a tendency to do. "It works now" absolves everyone
from the task to come back later and improve the solution, so no one
does.

jens

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

wichert at wiggy

Feb 29, 2008, 4:09 AM

Post #14 of 25 (1017 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Previously Jens Vagelpohl wrote:
>
> On Feb 29, 2008, at 10:00 , Andreas Jung wrote:
>
> >
> >
> >--On 29. Februar 2008 08:39:05 +0000 Martin Aspeli
> ><optilude[at]gmx.net> wrote:
> >
> >
> >>
> >>Of course, we should also provide a way to get an interface or
> >>something
> >>else describing the configuration for introspection purposes.
> >>Waiting one
> >>or two Zope versions for that to get a non-purging GS import
> >>handler when
> >>there's a works-90%-of-the-time solution (falling back on current
> >>behaviour) would be a shame though.
> >>
> >
> >You don't have to wait for for new Zope versions. Defining the
> >interface
> >officially in Zope 2.10, 2.11 and trunk will raise no problems. I
> >have no problems with putting this into the official release
> >branches - or?
>
> My personal opinion: I'd rather see the interface-based solution in a
> few weeks or a couple months (the next Zope release) than the, umh,
> less-than-professional solution that will stick around forever. As
> such solutions have a tendency to do. "It works now" absolves everyone
> from the task to come back later and improve the solution, so no one
> does.

Amen to that.

Wichert.

--
Wichert Akkerman <wichert[at]wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

wichert at wiggy

Feb 29, 2008, 4:09 AM

Post #15 of 25 (1015 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

lists at zopyx

Feb 29, 2008, 4:17 AM

Post #16 of 25 (1017 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 29. Februar 2008 13:09:01 +0100 Wichert Akkerman <wichert[at]wiggy.net>
wrote:

>>
>> My personal opinion: I'd rather see the interface-based solution in a
>> few weeks or a couple months (the next Zope release) than the, umh,
>> less-than-professional solution that will stick around forever. As
>> such solutions have a tendency to do. "It works now" absolves everyone
>> from the task to come back later and improve the solution, so no one
>> does.
>
>

Sorry, I can't follow...what is the outcome?

I volunteer to add the interface to the Zope 2.10-2.11 branches and trunk
right now. This would be good enough for you for writing the related
adapter. The related code can be moved already into the Zope core on the
trunk (but not for any of the release branches).

ANdreas

lists at zopyx

Feb 29, 2008, 4:17 AM

Post #17 of 25 (1012 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

lists at zopyx

Feb 29, 2008, 4:52 AM

Post #18 of 25 (1017 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 28. Februar 2008 09:38:31 +0100 yuppie <y.2008[at]wcm-solutions.de> wrote:

> Hi!
>
>
> Andreas Jung wrote:
>> How about the following idea:
>>
>> - within the Zope core we define an _optional_ interface for indexes -
>> something like:
>>
>> class IIndexConfiguration(Interface):
>>
>> def getConfiguration():
>> """ Returns a dict with index specific configuration
>> parameters.
>> """
>>
>> - on the CMF/GS side we could register adapter for each index type
>> we know (basically the Zope 2 core indexes, ExtendedPathIndex,
>> TextIndexNG 3) and retrieve the related information
>>
>> - the related GS asks each index for its configuration and takes
>> appropriate action based on the comparison of the values from the
>> profile and the existing index.
>>
>> Adding the interface to Zope 2.11 or backporting it to Zope 2.10
>> would not be a problem. Since the Zope 2.11 branch is offically closed
>> for new features, the index specific implementations of
>> IIndexConfiguration
>> should be implemented outside the Zope core but we might move the
>> implementation into the Zope core with Zope 2.12. Sounds reasonable?
>
> Yes. textindexng already has getSettings(), I used it for the
> export/import code of TextIndexNG3:
>
> http://textindexng.svn.sourceforge.net/viewvc/textindexng/TextIndexNG3/tr
> unk/exportimport.py?revision=1833&view=markup
>
> That code compares the settings and makes sure TextIndexNG3 is only
> modified and cleared if necessary. We could use similar code for all
> other indexes if they would provide a method like that.
>
> I'd prefer a IConfigurableIndex interface that also has a set method.

I added the IIndexConfiguration to the Zope trunk. I don't think that a set
method is a good idea. Removing and re-adding is possibly the cleanest
solution. Some indexes might perform some configurations within their
constructor. Calling clear() would not be enough for getting their
configuration right.

Andreas

jens at dataflake

Feb 29, 2008, 5:07 AM

Post #19 of 25 (1016 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

On Feb 29, 2008, at 13:17 , Andreas Jung wrote:

>
>
> --On 29. Februar 2008 13:09:01 +0100 Wichert Akkerman <wichert[at]wiggy.net
> > wrote:
>
>>>
>>> My personal opinion: I'd rather see the interface-based solution
>>> in a
>>> few weeks or a couple months (the next Zope release) than the, umh,
>>> less-than-professional solution that will stick around forever. As
>>> such solutions have a tendency to do. "It works now" absolves
>>> everyone
>>> from the task to come back later and improve the solution, so no one
>>> does.
>>
>>
>
> Sorry, I can't follow...what is the outcome?
>
> I volunteer to add the interface to the Zope 2.10-2.11 branches and
> trunk
> right now. This would be good enough for you for writing the related
> adapter. The related code can be moved already into the Zope core on
> the trunk (but not for any of the release branches).

I misunderstood one thing here. You only talked about the interface,
but I kept thinking implementation as well :-) So my desired outcome
was implementation plus interface, so that everything is ready to be
used with the next release. With the interface alone we only help
those indices that are not part of Zope itself, since the Zope core
indices apparently won't be able to have a working implementation
until Zope 2.12 comes around..?

jens

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

lists at zopyx

Feb 29, 2008, 6:38 AM

Post #20 of 25 (1006 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 29. Februar 2008 14:07:57 +0100 Jens Vagelpohl <jens[at]dataflake.org>
wrote:

>
> On Feb 29, 2008, at 13:17 , Andreas Jung wrote:
>
>>
>>
>> --On 29. Februar 2008 13:09:01 +0100 Wichert Akkerman <wichert[at]wiggy.net
>> > wrote:
>>
>>>>
>>>> My personal opinion: I'd rather see the interface-based solution
>>>> in a
>>>> few weeks or a couple months (the next Zope release) than the, umh,
>>>> less-than-professional solution that will stick around forever. As
>>>> such solutions have a tendency to do. "It works now" absolves
>>>> everyone
>>>> from the task to come back later and improve the solution, so no one
>>>> does.
>>>
>>>
>>
>> Sorry, I can't follow...what is the outcome?
>>
>> I volunteer to add the interface to the Zope 2.10-2.11 branches and
>> trunk
>> right now. This would be good enough for you for writing the related
>> adapter. The related code can be moved already into the Zope core on
>> the trunk (but not for any of the release branches).
>
> I misunderstood one thing here. You only talked about the interface, but
> I kept thinking implementation as well :-)

> So my desired outcome was
> implementation plus interface, so that everything is ready to be used
> with the next release.

I gave you the interface and you'll put the implementations as adapters
for all indexes you need into GS.

> With the interface alone we only help those
> indices that are not part of Zope itself, since the Zope core indices
> apparently won't be able to have a working implementation until Zope 2.12
> comes around..?

Sure - through adaptation with the GS core. You can of course depend at
some point that the core indices implement the behavior on their own. But
adapter approach allows you to deal with the GS problem right now and you
don't have to wait until Zope 2.12.

Andreas

y.2008 at wcm-solutions

Feb 29, 2008, 7:45 AM

Post #21 of 25 (1011 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Andreas Jung wrote:
> --On 28. Februar 2008 09:38:31 +0100 yuppie
> <y.2008-E2EsyBC0hj3+aS/vkh9bjw[at]public.gmane.org> wrote:
>> I'd prefer a IConfigurableIndex interface that also has a set method.
>
> I added the IIndexConfiguration to the Zope trunk. I don't think that a
> set method is a good idea. Removing and re-adding is possibly the
> cleanest solution. Some indexes might perform some configurations within
> their constructor. Calling clear() would not be enough for getting their
> configuration right.

All the export/import adapters shipped with GenericSetup and the adapter
shipped with TextIndexNG3 *modify* the indexes and call clear(). AFAICT
this works fine. And with an official set method it would no longer be a
hack.

Switching to the remove-and-re-add pattern would not be easy.

Cheers, Yuppie

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

lists at zopyx

Feb 29, 2008, 7:52 AM

Post #22 of 25 (1010 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

--On 29. Februar 2008 16:45:19 +0100 yuppie <y.2008[at]wcm-solutions.de> wrote:

> Andreas Jung wrote:
>> --On 28. Februar 2008 09:38:31 +0100 yuppie
>> <y.2008-E2EsyBC0hj3+aS/vkh9bjw[at]public.gmane.org> wrote:
>>> I'd prefer a IConfigurableIndex interface that also has a set method.
>>
>> I added the IIndexConfiguration to the Zope trunk. I don't think that a
>> set method is a good idea. Removing and re-adding is possibly the
>> cleanest solution. Some indexes might perform some configurations within
>> their constructor. Calling clear() would not be enough for getting their
>> configuration right.
>
> All the export/import adapters shipped with GenericSetup and the adapter
> shipped with TextIndexNG3 *modify* the indexes and call clear(). AFAICT
> this works fine. And with an official set method it would no longer be a
> hack.
>
> Switching to the remove-and-re-add pattern would not be easy.
>
>

So just go ahead and define the interface as you need it :-)

Andreas

dieter at handshake

Feb 29, 2008, 10:20 AM

Post #23 of 25 (1011 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Andreas Jung wrote at 2008-2-29 06:43 +0100:
> ...
>> When the export handler is able to extract all relevant configuration
>> parameters for an index, why should the import handler
>> not be able to check the configuration parameters in a profile
>> against an existing index and determine that it needs to do nothing?
>
>Huh? Because we're looking for a clean solution and not for a hack!

Why is it a hack when the import handler uses as much detail about
an object as the export handler does?

As I understood Wichert, we can currently export indexes reliably but
we cannot import it without removal and recreation of the index.
I cannot understand this.

--
Dieter
_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

optilude at gmx

Feb 29, 2008, 12:08 PM

Post #24 of 25 (1004 views)
Permalink

Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

Jens Vagelpohl wrote:
> On Feb 29, 2008, at 10:00 , Andreas Jung wrote:
>
>>
>> --On 29. Februar 2008 08:39:05 +0000 Martin Aspeli
>> <optilude[at]gmx.net> wrote:
>>
>>
>>> Of course, we should also provide a way to get an interface or
>>> something
>>> else describing the configuration for introspection purposes.
>>> Waiting one
>>> or two Zope versions for that to get a non-purging GS import
>>> handler when
>>> there's a works-90%-of-the-time solution (falling back on current
>>> behaviour) would be a shame though.
>>>
>> You don't have to wait for for new Zope versions. Defining the
>> interface
>> officially in Zope 2.10, 2.11 and trunk will raise no problems. I
>> have no problems with putting this into the official release
>> branches - or?
>
> My personal opinion: I'd rather see the interface-based solution in a
> few weeks or a couple months (the next Zope release) than the, umh,
> less-than-professional solution that will stick around forever. As
> such solutions have a tendency to do. "It works now" absolves everyone
> from the task to come back later and improve the solution, so no one
> does.

I agree in principle, however this has been left unresolved for far too
long. I'm glad things are happening now, though!

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

jens at dataflake

Feb 29, 2008, 3:22 PM

Post #25 of 25 (998 views)
Permalink

Re: Re: What is the status of GS wiping catalog indexes on catalog.xml import? [In reply to]

On Feb 29, 2008, at 21:08 , Martin Aspeli wrote:

> Jens Vagelpohl wrote:
>> On Feb 29, 2008, at 10:00 , Andreas Jung wrote:
>>> You don't have to wait for for new Zope versions. Defining the
>>> interface
>>> officially in Zope 2.10, 2.11 and trunk will raise no problems. I
>>> have no problems with putting this into the official release
>>> branches - or?
>> My personal opinion: I'd rather see the interface-based solution in
>> a few weeks or a couple months (the next Zope release) than the,
>> umh, less-than-professional solution that will stick around
>> forever. As such solutions have a tendency to do. "It works now"
>> absolves everyone from the task to come back later and improve the
>> solution, so no one does.
>
> I agree in principle, however this has been left unresolved for far
> too long. I'm glad things are happening now, though!

I agree that something had been left unresolved, but it's a bit odd if
an issue languishes in the collector for a long time and then all of a
sudden several people come along and say it's a big deal. It can't be
such a big deal if there were no complaints in a long time.

Anyway, it appears Andreas' interface additions, even though they may
not be the final solution, pointed us there. Someone can amend that
interface to everyones' linking and we have it in the next Zope
releases. After that, it's a matter of adding adapter code to GS.

jens

_______________________________________________
Zope-CMF maillist - Zope-CMF[at]lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com