Mailing List Archive: Lucene: Java-User

Can I add new field values to a existing lucene index ?

Index | Next | Previous | View Threaded

anupamb82 at gmail

Mar 28, 2012, 5:16 AM

Post #1 of 4 (531 views)
Permalink

Can I add new field values to a existing lucene index ?

Does Lucene API allows to add new field values to a existing doc.

For example,
Initially

Doc 1 FieldName Value id 1 schoolname xyz zipcode pqr

After update can I keep the existing field values and add 2 more fields.

Doc 1 FieldName Value id 1 schoolname xyz zipcode pqr *telephone* *
112233* *grade* *10*

lucene at mikemccandless

Mar 28, 2012, 5:31 AM

Post #2 of 4 (524 views)
Permalink

Re: Can I add new field values to a existing lucene index ? [In reply to]

Alas, no, not yet. This is an oft-requested feature, but challenging to build.

That said, there is a possible start towards making something possible in 4.0:

https://issues.apache.org/jira/browse/LUCENE-3837

Mike McCandless

http://blog.mikemccandless.com

On Wed, Mar 28, 2012 at 8:16 AM, Anupam Bhattacharya
<anupamb82 [at] gmail> wrote:
> Does Lucene API allows to add new field values to a existing doc.
>
> For example,
> Initially
>
> Doc 1 FieldName Value id 1 schoolname xyz zipcode pqr
>
>
> After update can I keep the existing field values and add 2 more fields.
>
> Doc 1 FieldName Value id 1 schoolname xyz zipcode pqr *telephone* *
> 112233* *grade* *10*

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

timeck at gmail

Mar 28, 2012, 11:30 AM

Post #3 of 4 (511 views)
Permalink

RE: Can I add new field values to a existing lucene index ? [In reply to]

Excuse my ignorance of lucene internals, but is the problem any easier if
the requirement is just to allow the addition/removal of stored only fields
(as opposed to indexed)?

I suspect this wasn't intent of the original question, but a document can
certainly be deleted and re-added to the index with a new set of fields.

-----Original Message-----
From: Michael McCandless [mailto:lucene [at] mikemccandless]
Sent: Wednesday, March 28, 2012 5:31 AM
To: java-user [at] lucene
Subject: Re: Can I add new field values to a existing lucene index ?

Alas, no, not yet. This is an oft-requested feature, but challenging to
build.

That said, there is a possible start towards making something possible in
4.0:

https://issues.apache.org/jira/browse/LUCENE-3837

Mike McCandless

http://blog.mikemccandless.com

On Wed, Mar 28, 2012 at 8:16 AM, Anupam Bhattacharya
<anupamb82 [at] gmail> wrote:
> Does Lucene API allows to add new field values to a existing doc.
>
> For example,
> Initially
>
> Doc 1 FieldName Value id 1 schoolname xyz zipcode pqr
>
>
> After update can I keep the existing field values and add 2 more fields.
>
> Doc 1 FieldName Value id 1 schoolname xyz zipcode pqr *telephone*
*
> 112233* *grade* *10*

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

lucene at mikemccandless

Mar 29, 2012, 5:44 AM

Post #4 of 4 (510 views)
Permalink

Re: Can I add new field values to a existing lucene index ? [In reply to]

On Wed, Mar 28, 2012 at 2:30 PM, Tim Eck <timeck [at] gmail> wrote:
> Excuse my ignorance of lucene internals, but is the problem any easier if
> the requirement is just to allow the addition/removal of stored only fields
> (as opposed to indexed)?

It would substantially simplify the problem... but even this
simplification is quite a bit of work.

Lucene's segments are write-once, so when some number of documents in
a given segment are updated, we'd need to write new files against that
segment (we call this a "stacked segment"), sort of like a patch over
the original segment.

But Lucene's stored fields writer/reader impls are set up to handle
this sort of "patching", so some work must be done to correlate docIDs
in the patched segment against docIDs in the base segment...

> I suspect this wasn't intent of the original question, but a document can
> certainly be deleted and re-added to the index with a new set of fields.

Ahh, right, I forgot to mention that: fully removing the original
document, and then fully adding the new document (with all original
fields, plus the new fields) is always possible... just not "ideal" if
your update is tiny and your original fields are huge.

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads