lucene at mikemccandless
Jan 14, 2010, 3:17 AM
Post #2 of 2
(2035 views)
Permalink
|
This is a known limitation of Lucene over NFS.
It's because NFS makes no effort to protect open files from deletion.
Other filesystems prevent (or delay) deletion of still open files, eg
on Unix the "delete on last close" semantics is used, on Windows the
file cannot be deleted until no process has it open anymore.
One way to workaround this is to make a custom IndexDeletionPolicy, so
that your app defers deletion of old commits until you know all
current readers have reopened. Another workaround is to simply catch
that exception (best to screen for Stale NFS file handle, so you don't
mask other IOException cases), and reopen your reader right then --
but this is only viable if it's acceptable that a random Query will be
forced to wait while reopen/warming takes place.
Mike
On Thu, Jan 14, 2010 at 1:25 AM, Claudio Deluca <decla86 [at] gmail> wrote:
> Hi,
> We are using Lucene 2.4.1 in a load-balanced environment.
> The lucene index is stored on server a while server b accesses the index
> though an nfs share.
>
> After creating the instance of IndexWriter, the documents are beeing added
> and the index gets optimized and closed.
> *IndexWriter theIndexWriter = new IndexWriter(new File(indexerPath), new
> WhitespaceAnalyzer(), true, IndexWriter.MaxFieldLength.UNLIMITED);
> ...
> theIndexWriter.optimize();
> theIndexWriter.close();*
>
> For serach we open the index on application startup like this
> *Directory theDirectory = FSDirectory.getDirectory(theConfigPath);
> IndexReader indexReader = IndexReader.open(theDirectory, true);
> IndexSearcher searcher = new IndexSearcher(indexReader);*
>
> The exception appears, when server a finished recreation of index (closed
> the indexwriter) and server b executes a search query over the index.
> Only if we restart the application the problem does not longer appear,
> because at this point the index will be newly opened.
>
> How can we avoid this Exception?
>
> java.io.IOException: Stale NFS file handle
> at java.io.RandomAccessFile.readBytes(Native Method)
> at java.io.RandomAccessFile.read(Unknown Source)
> at org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:596)
> at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:136)
> at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:247)
> at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:157)
> at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
> at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
> at org.apache.lucene.index.SegmentTermDocs.next(SegmentTermDocs.java:110)
> at org.apache.lucene.index.SegmentTermPositions.next(SegmentTermPositions.java:98)
> at org.apache.lucene.search.PhrasePositions.next(PhrasePositions.java:41)
> at org.apache.lucene.search.PhraseScorer.init(PhraseScorer.java:131)
> at org.apache.lucene.search.PhraseScorer.next(PhraseScorer.java:76)
> at org.apache.lucene.search.ConjunctionScorer.init(ConjunctionScorer.java:80)
> at org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:48)
> at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:319)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:136)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:123)
> at org.apache.lucene.search.Searcher.search(Searcher.java:86)
>
>
> Thanks,
> Claudio
>
|
