toolkit at magma
Sep 22, 2004, 10:53 AM
Post #3 of 3
(1187 views)
Permalink
|
From: "Chris Withers" <chris [at] simplistix>
> Jonathan Hobbs wrote:
>
> > I have a ZCatalog that has grown to about 6.3Gb in size and as a result
no
> > longer fits in memory - resulting in a severe degradation in performance
(a
> > search that used to take less than 2 cpu seconds, now takes 5-14 cpu
> > seconds). Is there any way to 'extract' a single index from the
ZCatalog (a
> > ZCTextIndex) and hold it in memory to improve search retrieval
performance ?
> >
> > Also, because we are using the OkapiIndex relevance ranking process, we
will
> > need to hold the Lexicon in memory. Has anyone tried/accomplished this?
> >
> > Hoping there is a solution...
>
> Try ZODB mount points, moutn the index and the lexicon in a seperate
> storage...
Thanks for the idea Chris!
I can't see a way to take a single catalog and put the indexes into separate
storages (DBTab wants a folder as a mount point, but ZCatalog index creation
does not allow for this), however your suggestion has given me the idea of
breaking the catalog into several pieces (one index + lexicon in each piece)
and putting each 'sub-catalog' into its own storage on a temporary folder.
I would then using a script/external method to combine the results from each
catalog; sounds ugly but at this point I am willing to try anything!
Thanks again,
Jonathan
_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
|
