Mailing List Archive: Lucene: General

Synchronization and merging indexes

Index | Next | Previous | View Threaded

logan at electricstorm

Dec 20, 2008, 4:12 PM

Post #1 of 3 (1030 views)
Permalink

Synchronization and merging indexes

Hi. I'm currently using Lucene.Net as the backing store for a client Windows
app and it's working great, however I'm now looking at making this an
occasionally-connected remote-synchronized store.

In other words, I want to use one of the free online storage APIs out there
that my users can subscribe to and provide login credentials, and use it to
back up entire copies of the index (we're talking relatively small indexes
here).

The scenario should allow for multiple clients to be simultaneously
modifying their local copies of the index, and therefore I will need to
merge the indexes to allow for multiple sources of change.

My question is first of all if anyone has any experience with this, just for
some advice, but in particular I'm concerned with the merging process--does
merging two indexes simply concatenate all documents in each, even if they
are identical, or is there some kind of logic performed to union duplicates?
If not, how should I go about doing that manually in an efficient way?

I'm not terribly worried about conflicts or collisions--in the worst case I
can simply duplicate the document, but I don't want duplicate copies of
documents created when there is no conflict.

Thanks for any advice.

Logan
--
View this message in context: http://www.nabble.com/Synchronization-and-merging-indexes-tp21110690p21110690.html
Sent from the Lucene - General mailing list archive at Nabble.com.

otis_gospodnetic at yahoo

Dec 21, 2008, 7:15 PM

Post #2 of 3 (948 views)
Permalink

Re: Synchronization and merging indexes [In reply to]

Logan,

My guess is you'll get more help if you post your question to the Lucene.Net mailing list (and whose address I don't recall off the top of my head).

Otis --
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

----- Original Message ----
> From: chaiguy1337 <logan [at] electricstorm>
> To: general [at] lucene
> Sent: Saturday, December 20, 2008 7:12:24 PM
> Subject: Synchronization and merging indexes
>
>
> Hi. I'm currently using Lucene.Net as the backing store for a client Windows
> app and it's working great, however I'm now looking at making this an
> occasionally-connected remote-synchronized store.
>
> In other words, I want to use one of the free online storage APIs out there
> that my users can subscribe to and provide login credentials, and use it to
> back up entire copies of the index (we're talking relatively small indexes
> here).
>
> The scenario should allow for multiple clients to be simultaneously
> modifying their local copies of the index, and therefore I will need to
> merge the indexes to allow for multiple sources of change.
>
> My question is first of all if anyone has any experience with this, just for
> some advice, but in particular I'm concerned with the merging process--does
> merging two indexes simply concatenate all documents in each, even if they
> are identical, or is there some kind of logic performed to union duplicates?
> If not, how should I go about doing that manually in an efficient way?
>
> I'm not terribly worried about conflicts or collisions--in the worst case I
> can simply duplicate the document, but I don't want duplicate copies of
> documents created when there is no conflict.
>
> Thanks for any advice.
>
> Logan
> --
> View this message in context:
> http://www.nabble.com/Synchronization-and-merging-indexes-tp21110690p21110690.html
> Sent from the Lucene - General mailing list archive at Nabble.com.

erik at ehatchersolutions

Dec 22, 2008, 2:08 AM

Post #3 of 3 (947 views)
Permalink

Re: Synchronization and merging indexes [In reply to]

lucene-net-user [at] lucene

More info here: <http://incubator.apache.org/lucene.net/>

Erik

On Dec 21, 2008, at 10:15 PM, Otis Gospodnetic wrote:

> Logan,
>
> My guess is you'll get more help if you post your question to the
> Lucene.Net mailing list (and whose address I don't recall off the
> top of my head).
>
>
> Otis --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> ----- Original Message ----
>> From: chaiguy1337 <logan [at] electricstorm>
>> To: general [at] lucene
>> Sent: Saturday, December 20, 2008 7:12:24 PM
>> Subject: Synchronization and merging indexes
>>
>>
>> Hi. I'm currently using Lucene.Net as the backing store for a
>> client Windows
>> app and it's working great, however I'm now looking at making this an
>> occasionally-connected remote-synchronized store.
>>
>> In other words, I want to use one of the free online storage APIs
>> out there
>> that my users can subscribe to and provide login credentials, and
>> use it to
>> back up entire copies of the index (we're talking relatively small
>> indexes
>> here).
>>
>> The scenario should allow for multiple clients to be simultaneously
>> modifying their local copies of the index, and therefore I will
>> need to
>> merge the indexes to allow for multiple sources of change.
>>
>> My question is first of all if anyone has any experience with this,
>> just for
>> some advice, but in particular I'm concerned with the merging
>> process--does
>> merging two indexes simply concatenate all documents in each, even
>> if they
>> are identical, or is there some kind of logic performed to union
>> duplicates?
>> If not, how should I go about doing that manually in an efficient
>> way?
>>
>> I'm not terribly worried about conflicts or collisions--in the
>> worst case I
>> can simply duplicate the document, but I don't want duplicate
>> copies of
>> documents created when there is no conflict.
>>
>> Thanks for any advice.
>>
>> Logan
>> --
>> View this message in context:
>> http://www.nabble.com/Synchronization-and-merging-indexes-tp21110690p21110690.html
>> Sent from the Lucene - General mailing list archive at Nabble.com.

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads