Mailing List Archive: kinosearch: discuss

KinoSearch incremental updates

Index | Next | Previous | View Threaded

converter42 at gmail

Jun 11, 2008, 10:19 PM

Post #1 of 2 (732 views)
Permalink

KinoSearch incremental updates

KinoSearch version 0.20_05

I have some code that is building an inverted index across a large group of
small files and as far as I can tell it is rebuilding the entire index
rather than doing an incremental update.
The documentation suggests that &KinoSearch::InvIndexer::new takes a create
param that when passed with the value zero will change this behavior and
permit me to delete documents from the index and add update the index only
for new or changed documents, but the inclusion of this parameter causes an
exception for invalid parameter. Example:

my $idxer = KinoSearch::InvIndexer->new(
create => 0,
invindex => $invindex,
lock_factory => $lock_factory,
);

causes the exception:

Error in function build_args_hash at ../c_src/xs/XSBind.c:50: Invalid
parameter: 'create'

Are incremental index updates supported in general, and in my particular
version?

Thanks

marvin at rectangular

Jun 12, 2008, 10:14 AM

Post #2 of 2 (682 views)
Permalink

Re: KinoSearch incremental updates [In reply to]

On Jun 11, 2008, at 10:19 PM, David P.C. Wollmann wrote:

> KinoSearch version 0.20_05

> The documentation suggests that &KinoSearch::InvIndexer::new takes a
> create param

> causes the exception:
>
> Error in function build_args_hash at ../c_src/xs/XSBind.c:50:
> Invalid parameter: 'create'

The 'create' parameter to InvIndexer's constructor has been removed in
0.20_xx, and because KinoSearch validates all parameters, you get an
exception rather than silent failure, exactly as intended.

> Are incremental index updates supported in general, and in my
> particular version?

Yes, they're supported in both the stable branch (0.1x) and the dev
branch (0.20_xx), but the API has changed. In the dev branch, you do
something like this if you want to update an index...

my $invindexer = KinoSearch::InvIndexer->new(
invindex => MySchema->open('/path/to/invindex'),
);

... as opposed to this, which starts fresh:

my $invindexer = KinoSearch::InvIndexer->new(
invindex => MySchema->clobber('/path/to/invindex'),
);

See <http://www.rectangular.com/kinosearch/docs/devel/KinoSearch/InvIndexer.html
>.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

_______________________________________________
KinoSearch mailing list
KinoSearch [at] rectangular
http://www.rectangular.com/mailman/listinfo/kinosearch

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads