Mailing List Archive: Lucene: Java-User

Lucene 2.1: java.io.IOException: Lock obtain timed out: SimpleFSLock@<path of index file>

Index | Next | Previous | View Threaded

Jerome.Chauvin at businessinteractif

Mar 1, 2007, 12:23 AM

Post #1 of 4 (11302 views)
Permalink

Lucene 2.1: java.io.IOException: Lock obtain timed out: SimpleFSLock@<path of index file>

All,

We encounter issues while updating the lucene index, here is the stack trace:

Caused by: java.io.IOException: Lock obtain timed out:
SimpleFSLock@/data/www/orcanta/lucene/store1/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:69)
at org.apache.lucene.index.IndexReader.aquireWriteLock(IndexReader.java:526)
at org.apache.lucene.index.IndexReader.deleteDocument(IndexReader.java:551)
at org.apache.lucene.index.IndexReader.deleteDocuments(IndexReader.java:578)
at
com.bi.commerce.service.catalog.spring.lucene.LuceneIndex.deleteFromIndex(Luc
eneIndex.java:692)
... 25 more

Here is the source code of the lucene API invocation where the error occurs:

class com.bi.commerce.service.catalog.spring.lucene.LuceneIndex:

import org.apache.lucene.index.IndexReader;
...

public synchronized void deleteFromIndex(ICatalogEntity entity) {
if(!indexExists()) return;
try {
IndexReader reader = IndexReader.open(store);
String uid = getUID(entity);
try{
line 692=> reader.deleteDocuments(new Term("uid",uid));
}catch(ArrayIndexOutOfBoundsException e){
//CHECK ignore this. Can happen if index has not been built
yet (??)
}
reader.close();
} catch (IOException e) {
throw new SearchEngineException(e);
}catch(RuntimeException e){
throw new SearchEngineException(e);
}
}

Do I something wrong? If somebody already encountered this error, or knows a
fix, I'm really interested!

Thanks in advance,

BRegards.

-Jerome Chauvin-

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

lucene at mikemccandless

Mar 1, 2007, 2:22 AM

Post #2 of 4 (10867 views)
Permalink

Re: Lucene 2.1: java.io.IOException: Lock obtain timed out: SimpleFSLock@<path of index file> [In reply to]

"Jerome Chauvin" <Jerome.Chauvin [at] businessinteractif> wrote:

> We encounter issues while updating the lucene index, here is the stack
> trace:
>
> Caused by: java.io.IOException: Lock obtain timed out:
> SimpleFSLock@/data/www/orcanta/lucene/store1/write.lock
> at org.apache.lucene.store.Lock.obtain(Lock.java:69)
> at
> org.apache.lucene.index.IndexReader.aquireWriteLock(IndexReader.java:526)
> at
> org.apache.lucene.index.IndexReader.deleteDocument(IndexReader.java:551)
> at
> org.apache.lucene.index.IndexReader.deleteDocuments(IndexReader.java:578)
> at
> com.bi.commerce.service.catalog.spring.lucene.LuceneIndex.deleteFromIndex(Luc
> eneIndex.java:692)
> ... 25 more

First off, you have to ensure only one writer (either IndexWriter or
as in this case IndexReader doing deletes) is trying to update the
index at the same time. Lucene only allows one writer on an index, and if
a second writer tries to open it will receive exactly this exception.

(Note that as of 2.1 you can now do deletes with IndexWriter which
simplifies things because you can use a single IndexWriter for
adds/updates/deletes.)

If you are already doing that (single writer) correctly, the other
common cause is that this is a leftover lock file (for example if the
JVM crashed or was killed or even if you didn't close a previous
writer before the JVM exited). There is a better locking
implementation (NativeFSLockFactory) that correctly frees the lock
when the JVM crashes so you may want to use that one instead if you
hit this often (but first explain the root cause of your crashes!).

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Jerome.Chauvin at businessinteractif

Mar 2, 2007, 5:22 AM

Post #3 of 4 (10854 views)
Permalink

RE: Lucene 2.1: java.io.IOException: Lock obtain timed out: SimpleFSLock@<path of index file> [In reply to]

Thanks Michael for your answer, but following check of our processing, it
appears all the updates of the index are made in a single thread. Actually,
this kind of exception is thrown during a heavy batch processing. This
processing is not multi-threaded.

One other think I don't understand is: All the methods accessing the index in
Lucene are synchronized. How can a concurrent access to the index occur?

Thanks in advance,

-Jerome Chauvin-

-----Message d'origine-----
De : Michael McCandless [mailto:lucene [at] mikemccandless]
Envoyé : jeudi 1 mars 2007 11:22
À : java-user [at] lucene
Objet : Re: Lucene 2.1: java.io.IOException: Lock obtain timed out:
SimpleFSLock@<path of index file>

"Jerome Chauvin" <Jerome.Chauvin [at] businessinteractif> wrote:

> We encounter issues while updating the lucene index, here is the stack
> trace:
>
> Caused by: java.io.IOException: Lock obtain timed out:
> SimpleFSLock@/data/www/orcanta/lucene/store1/write.lock
> at org.apache.lucene.store.Lock.obtain(Lock.java:69)
> at
>
> org.apache.lucene.index.IndexReader.aquireWriteLock(IndexReader.java:5
> 26)
> at
>
> org.apache.lucene.index.IndexReader.deleteDocument(IndexReader.java:55
> 1)
> at
>
> org.apache.lucene.index.IndexReader.deleteDocuments(IndexReader.java:5
> 78)
> at
> com.bi.commerce.service.catalog.spring.lucene.LuceneIndex.deleteFromIn
> dex(Luc
> eneIndex.java:692)
> ... 25 more

First off, you have to ensure only one writer (either IndexWriter or as in
this case IndexReader doing deletes) is trying to update the index at the
same time. Lucene only allows one writer on an index, and if a second writer
tries to open it will receive exactly this exception.

(Note that as of 2.1 you can now do deletes with IndexWriter which simplifies
things because you can use a single IndexWriter for
adds/updates/deletes.)

If you are already doing that (single writer) correctly, the other common
cause is that this is a leftover lock file (for example if the JVM crashed or
was killed or even if you didn't close a previous writer before the JVM
exited). There is a better locking implementation (NativeFSLockFactory) that
correctly frees the lock when the JVM crashes so you may want to use that one
instead if you hit this often (but first explain the root cause of your
crashes!).

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

lucene at mikemccandless

Mar 2, 2007, 6:08 AM

Post #4 of 4 (10855 views)
Permalink

RE: Lucene 2.1: java.io.IOException: Lock obtain timed out: SimpleFSLock@<path of index file> [In reply to]

"Jerome Chauvin" wrote:
> Thanks Michael for your answer, but following check of our processing, it
> appears all the updates of the index are made in a single thread.
> Actually,
> this kind of exception is thrown during a heavy batch processing. This
> processing is not multi-threaded.

Do you also have an IndexWriter open at the time that you try to the
do deletes through the IndexReader instance?

You have to close that writer before you try to use the IndexReader to
do deletes (and vice/versa). Only one of these may be opened at a
time.

(This is rather confusing, and is one of reasons that
"deleteDocuments" method was added to IndexWriter in as of 2.1).

> One other think I don't understand is: All the methods accessing the
> index in
> Lucene are synchronized. How can a concurrent access to the index occur?

The synchronization protects Lucene when multiple threads are sharing
a single writer/reader instance (and this use case is totally fine &
correct).

To protect the Index against multiple writers (where an IndexReader
doing deletes is considered a "writer"), Lucene uses the write.lock.
Ie a single threaded program can incorrectly try to open multiple
writers against one index, which leads to exactly this exception.

Since writers can in general be on different JVMs or different
machines, Lucene must guard against that as well. This is why the
"write.lock" [by default] is based in the filesystem and stored [by
default] in the index directory.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads