Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-Dev

Deleting first IndexCommit

 

 

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded


jason.rutherglen at gmail

Jul 3, 2008, 5:11 AM

Post #1 of 4 (228 views)
Permalink
Deleting first IndexCommit

For Ocean I need to be able to delete the latest commit, however currently
using IndexReader.open(final IndexCommit commit, IndexDeletionPolicy
deletionPolicy) I need to open the IndexReader that represents the latest
commit, presumably in IndexDeletionPolicy.onInit delete the IndexCommit,
then close the IndexReader and start over. Wouldn't this be cleaner to call
IndexReader.listCommits and then delete the first one (assuming they are in
generation descending order).


lucene at mikemccandless

Jul 3, 2008, 8:09 AM

Post #2 of 4 (211 views)
Permalink
Re: Deleting first IndexCommit [In reply to]

Actually you'd need to open an IndexWriter to do the delete, unless
you open a reader, make a change, then close the reader (on closing
the reader, which makes a new commit, your deletion policy gets a
chance to delete commits).

One question: why are you needing to delete the most recent commit?
(Just curious about the use case behind this...).

"Normally", deletion of commits happens when a new commit is created,
but I agree that being able to spontaneous delete commits could be
nice and would be more straightforward, for Ocean at least. You're
sort of out in new territory here (not too many people juggle so many
commits in their index)!

To do this we'd have to grab the write lock, create an
IndexFileDeleter, delete the commit, close the IndexFileDeleter and
release the write lock. But I'm not sure we want to go down that road
vs. having IndexWriter be the thing you open to make changes
(including deletion of commits) to an index? Maybe, instead, you open
a writer, then we expose a deleteCommit(IndexCommit) method...

Mike

Jason Rutherglen wrote:

> For Ocean I need to be able to delete the latest commit, however
> currently using IndexReader.open(final IndexCommit commit,
> IndexDeletionPolicy deletionPolicy) I need to open the IndexReader
> that represents the latest commit, presumably in
> IndexDeletionPolicy.onInit delete the IndexCommit, then close the
> IndexReader and start over. Wouldn't this be cleaner to call
> IndexReader.listCommits and then delete the first one (assuming they
> are in generation descending order).


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org


jason.rutherglen at gmail

Jul 3, 2008, 8:25 AM

Post #3 of 4 (209 views)
Permalink
Re: Deleting first IndexCommit [In reply to]

It's for recovery, a commit to an index may have occurred but other commits
as part of a global commit failed, so it's for rollback in the event of
failure.

Are you saying the delete does not immediately happen when
IndexCommit.delete is called using IndexDeletionPolicy?

I don't mind using IndexWriter, I am just looking for a more straightforward
API for managing the index generations than using a callback class like
IndexDeletionPolicy.

On Thu, Jul 3, 2008 at 11:09 AM, Michael McCandless <
lucene[at]mikemccandless.com> wrote:

> Actually you'd need to open an IndexWriter to do the delete, unless you
> open a reader, make a change, then close the reader (on closing the reader,
> which makes a new commit, your deletion policy gets a chance to delete
> commits).
>
> One question: why are you needing to delete the most recent commit? (Just
> curious about the use case behind this...).
>
> "Normally", deletion of commits happens when a new commit is created, but I
> agree that being able to spontaneous delete commits could be nice and would
> be more straightforward, for Ocean at least. You're sort of out in new
> territory here (not too many people juggle so many commits in their index)!
>
> To do this we'd have to grab the write lock, create an IndexFileDeleter,
> delete the commit, close the IndexFileDeleter and release the write lock.
> But I'm not sure we want to go down that road vs. having IndexWriter be the
> thing you open to make changes (including deletion of commits) to an index?
> Maybe, instead, you open a writer, then we expose a
> deleteCommit(IndexCommit) method...
>
> Mike
>
> Jason Rutherglen wrote:
>
> For Ocean I need to be able to delete the latest commit, however currently
>> using IndexReader.open(final IndexCommit commit, IndexDeletionPolicy
>> deletionPolicy) I need to open the IndexReader that represents the latest
>> commit, presumably in IndexDeletionPolicy.onInit delete the IndexCommit,
>> then close the IndexReader and start over. Wouldn't this be cleaner to call
>> IndexReader.listCommits and then delete the first one (assuming they are in
>> generation descending order).
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
> For additional commands, e-mail: java-dev-help[at]lucene.apache.org
>
>


lucene at mikemccandless

Jul 3, 2008, 8:43 AM

Post #4 of 4 (206 views)
Permalink
Re: Deleting first IndexCommit [In reply to]

Jason Rutherglen wrote:

> It's for recovery, a commit to an index may have occurred but other
> commits as part of a global commit failed, so it's for rollback in
> the event of failure.

Ahh got it.

> Are you saying the delete does not immediately happen when
> IndexCommit.delete is called using IndexDeletionPolicy?

Well ... it happens nearly immediately. When you call that delete()
method it merely records that fact that you want to delete that
commit. Then when you return from onInit/onCommit, IndexFileDeleter
walks through the commits you had deleted, decRef's all files
referenced by each commit, deleting any file that hits refCount 0.

> I don't mind using IndexWriter, I am just looking for a more
> straightforward API for managing the index generations than using a
> callback class like IndexDeletionPolicy.

I agree we could push for a more straightforward API here over time,
but I think it's not as high priority as the other stuff we're working
through. Or at least I'm feeling very busy!!

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.