Mailing List Archive: Lucene: Java-User

problem:lucene did not delete old index file after optimize method called

Index | Next | Previous | View Threaded

luocan19826164 at sohu

Feb 10, 2010, 1:24 AM

Post #1 of 7 (1234 views)
Permalink

problem:lucene did not delete old index file after optimize method called

lucene did not delete old index file after optimize method called.
ps:I call IndexWriter.getReader() and then call old IndexReader.close() every 1 minute,
a long time pass, I watche old index file did not disappear.
after I restart my program, optimize index file disappear,but old index file stay!!

lucene at mikemccandless

Feb 10, 2010, 1:58 AM

Post #2 of 7 (1161 views)
Permalink

Re: problem:lucene did not delete old index file after optimize method called [In reply to]

This happens, on Windows, when there is an IndexReader still reading
those files. IndexWriter will periodically (after a merge completes
or a new segment is flushed) retry deleting those files, but it won't
succeed until no reader has a given file open anymore.

IW also deletes unreferenced files when it's first opened. So if you
want to force it to try to delete, close your current IW and open a
new one (with create=false).

Mike

On Wed, Feb 10, 2010 at 4:24 AM, <luocan19826164 [at] sohu> wrote:
>
>
>
>
> lucene did not delete old index file after optimize method called.
> ps:I call IndexWriter.getReader() and then call old IndexReader.close() every 1 minute,
> a long time pass, I watche old index file did not disappear.
> after I restart my program, optimize index file disappear,but old index file stay!!

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

luocan19826164 at sohu

Feb 10, 2010, 2:35 AM

Post #3 of 7 (1174 views)
Permalink

Re: problem:lucene did not delete old index file after optimize method called [In reply to]

thanks for your reply!
but I don't think there is an IndexReader still reading those files,because I call indexReader close
and reopen every 1 minute .
 
IW also deletes unreferenced files,but why it delete the optimize file,not delete the old index file.
the merged file is what I wanted.
 
((after I restart my program, optimize index file disappear,but old index file stay))
 
 

----- 原文 ----- 发件人: Michael McCandless 主　题: Re: problem:lucene did not delete old index file after optimize method called时　间: 2010年2月10日 17:58:12
This happens, on Windows, when there is an IndexReader still readingthose files. IndexWriter will periodically (after a merge completesor a new segment is flushed) retry deleting those files, but it won'tsucceed until no reader has a given file open anymore.IW also deletes unreferenced files when it's first opened. So if youwant to force it to try to delete, close your current IW and open anew one (with create=false).MikeOn Wed, Feb 10, 2010 at 4:24 AM, <luocan19826164 [at] sohu> wrote:>>>>> lucene did not delete old index file after optimize method called.> ps:I call IndexWriter.getReader() and then call old IndexReader.close() every 1 minute,> a long time pass, I watche old index file did not disappear.> after I restart my program, optimize index file disappear,but old index file stay!!---------------------------------------------------------------------To unsubscribe, e-mail: java-user-unsubscribe [at] lucene additional commands, e-mail: java-user-help [at] lucene

lucene at mikemccandless

Feb 10, 2010, 2:39 AM

Post #4 of 7 (1158 views)
Permalink

Re: problem:lucene did not delete old index file after optimize method called [In reply to]

My guess is there is accidentally still a reader open, at the time
that IW tries to delete these unreferenced files.

Eg if you close & reopen your reader, always, then there is always a
reader open on the index.

Try closing all readers, then close IW, then open & close a new IW,
and see if the files are deleted.

Or... can you make this happen in a small test case?

Mike

On Wed, Feb 10, 2010 at 5:35 AM, <luocan19826164 [at] sohu> wrote:
> thanks for your reply!
> but I don't think there is an IndexReader still reading those files,because I call indexReader close
> and reopen every 1 minute .
>  
> IW also deletes unreferenced files,but why it delete the optimize file,not delete the old index file.
> the merged file is what I wanted.
>  
> ((after I restart my program, optimize index file disappear,but old index file stay))
>  
>  
>
>
>
> ----- ԭ�� ----- ��: Michael McCandless ��: Re: problem:lucene did not delete old index file after optimize method calledʱ��: 2010��2��10�� 17:58:12
> This happens, on Windows, when there is an IndexReader still readingthose files. IndexWriter will periodically (after a merge completesor a new segment is flushed) retry deleting those files, but it won'tsucceed until no reader has a given file open anymore.IW also deletes unreferenced files when it's first opened. So if youwant to force it to try to delete, close your current IW and open anew one (with create=false).MikeOn Wed, Feb 10, 2010 at 4:24 AM, <luocan19826164 [at] sohu> wrote:>>>>> lucene did not delete old index file after optimize method called.> ps:I call IndexWriter.getReader() and then call old IndexReader.close() every 1 minute,> a long time pass, I watche old index file did not disappear.> after I restart my program, optimize index file disappear,but old index file stay!!---------------------------------------------------------------------To unsubscribe, e-mail: java-user-unsubscribe [at] lucene additional commands, e-mail: java-user-help [at] lucene

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

luocan19826164 at sohu

Feb 10, 2010, 6:54 AM

Post #5 of 7 (1148 views)
Permalink

RE: problem:lucene did not delete old index file after optimize method called [In reply to]

here a small test case I watched there are three compound files.
0.cfs 6786kb
1.cfs 2044kb
2.cfs 8790kb(the optimize file)
I think in this testcase only 2.cfs left(the optimize file left),Is that right??

import java.io.File;
import java.io.IOException;

import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.Fieldable;
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.store.LockObtainFailedException;
import org.apache.lucene.util.Version;

public class ReopenTest {
IndexWriter writer;

public static void main(String[] args) throws CorruptIndexException,
LockObtainFailedException, IOException {
final IndexWriter writer = new IndexWriter(FSDirectory.open(new File(
"data")), new StandardAnalyzer(Version.LUCENE_CURRENT),
IndexWriter.MaxFieldLength.LIMITED);
writer.setMergeFactor(16);

Thread writeThread = new Thread(new Runnable() {
@Override
public void run() {
// write test data
try {
for (int i = 0; i < 1000000; i++) {
Document doc = new Document();
Fieldable itemID = new Field("ItemID", String
.valueOf(i), Field.Store.YES,
Field.Index.ANALYZED);

writer.addDocument(doc);
}

Thread.currentThread().sleep(10000);
System.out.println("optimize begin");
writer.optimize();
System.out.println("optimize end");
} catch (CorruptIndexException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

}
});

writeThread.start();

Thread reopenThread = new Thread(new Runnable() {
public void run() {
try {
IndexReader reader = writer.getReader();
while (true) {
Thread.currentThread().sleep(1000);
IndexReader newReader = writer.getReader();
if (newReader != reader) {
reader.close();
reader = newReader;
}
System.out.println("132");
}

} catch (InterruptedException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

});

reopenThread.start();
}

}

-----邮件原件-----
发件人: luocan19826164 [at] sohu [mailto:luocan19826164 [at] sohu]
发送时间: 2010年2月10日 22:49
收件人: java-user
主题: RE: here a small test case problem:lucene did not delete old index file after optimize method called

I watched there are three compound files.
0.cfs  6786kb
1.cfs  2044kb
2.cfs  8790kb(the optimize file)
I think in this testcase only 2.cfs left(the optimize file left),Is that right??
<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p> </o:p>
import java.io.File;
import java.io.IOException;
<o:p> </o:p>
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field; import org.apache.lucene.document.Fieldable;
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.store.LockObtainFailedException;
import org.apache.lucene.util.Version;
<o:p> </o:p>
public class ReopenTest {
         IndexWriter writer;
<o:p> </o:p>
         public static void main(String[] args) throws CorruptIndexException,
                            LockObtainFailedException, IOException {
                   final IndexWriter writer = new
IndexWriter(FSDirectory.open(new File(
                                     "data")), new
StandardAnalyzer(Version.LUCENE_CURRENT),
                                     IndexWriter.MaxFieldLength.LIMITED);
                   writer.setMergeFactor(16);
<o:p> </o:p>
                   Thread writeThread = new Thread(new Runnable() {
                            @Override
                            public void run() {
                                     // write test data
                                     try {
                                               for (int i = 0; i < 1000000; i++) {
                                                        Document doc = new
Document();
                                                        Fieldable itemID = new
Field("ItemID", String
                                                                           .valueOf(i),
Field.Store.YES,
                                                                           Field.Index.
ANALYZED);
<o:p> </o:p>
                                                        writer.addDocument(doc);
                                               }
<o:p> </o:p>
                                               Thread.currentThread().sleep(10000);
                                               System.out.println("optimize
begin");
                                               writer.optimize();
                                               System.out.println("optimize end");
                                     } catch (CorruptIndexException e) {
                                               e.printStackTrace();
                                     } catch (IOException e) {
                                               e.printStackTrace();
                                     } catch (InterruptedException e) {
                                               e.printStackTrace();
                                     }
<o:p> </o:p>
                            }
                   });
<o:p> </o:p>
                   writeThread.start();
                  
                  
                   Thread reopenThread = new Thread(new Runnable() {
                            public void run() {
                                     try {
                                               IndexReader reader =
writer.getReader();
                                               while (true) {
        
Thread.currentThread().sleep(1000);
                                                        IndexReader newReader =
writer.getReader();
                                                        if (newReader != reader) {
                                                                 reader.close();
                                                                 reader = newReader;
                                                        }
                                                        System.out.println("132");
                                               }
<o:p> </o:p>
                                     } catch (InterruptedException e) {
                                               e.printStackTrace();
                                     } catch (IOException e) {
                                               e.printStackTrace();
                                     }
                            }
<o:p> </o:p>
                   });
                  
                   reopenThread.start();
         }
<o:p> </o:p>
}
 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

lucene at mikemccandless

Feb 10, 2010, 7:35 AM

Post #6 of 7 (1146 views)
Permalink

Re: problem:lucene did not delete old index file after optimize method called [In reply to]

From this test, I would expect all 3 files to be left, because
IndexWriter never gets another chance to remove the files.

IndexWriter only attempts to remove unreferenced files in roughly 3 places:

* On open
* On flushing a new segment
* On finishing a merge

So, the moment your optimize finished, IndexWriter wanted to delete
0.cfs and 1.cfs, but could not, because the currently open reader was
using them. Then, because you don't close/open a new IndexWriter, no
further attempt is made to delete.

In a normal app, as you continue adding docs, normal merges kick off,
etc, these unreferenced files would eventually be removed.

It's just taking longer than it should, because IndexWriter doesn't
know exactly when the reader was closed.

I think we should add an explicit
IndexWriter.removeUnreferencedFiles() method, so apps can control the
immediacy if they want to. I'll open an issue.

For your tests, if you add code to stop both threads, close the
writer, then open & close a new writer, you should see 0.cfs and 1.cfs
removed.

Mike

On Wed, Feb 10, 2010 at 9:54 AM, luocanrao <luocan19826164 [at] sohu> wrote:
> here a small test case I watched there are three compound files.
> 0.cfs 6786kb
> 1.cfs 2044kb
> 2.cfs 8790kb(the optimize file)
> I think in this testcase only 2.cfs left(the optimize file left),Is that right??
>
> import java.io.File;
> import java.io.IOException;
>
> import org.apache.lucene.analysis.standard.StandardAnalyzer;
> import org.apache.lucene.document.Document;
> import org.apache.lucene.document.Field;
> import org.apache.lucene.document.Fieldable;
> import org.apache.lucene.index.CorruptIndexException;
> import org.apache.lucene.index.IndexReader;
> import org.apache.lucene.index.IndexWriter;
> import org.apache.lucene.store.FSDirectory;
> import org.apache.lucene.store.LockObtainFailedException;
> import org.apache.lucene.util.Version;
>
> public class ReopenTest {
> IndexWriter writer;
>
> public static void main(String[] args) throws CorruptIndexException,
> LockObtainFailedException, IOException {
> final IndexWriter writer = new IndexWriter(FSDirectory.open(new File(
> "data")), new StandardAnalyzer(Version.LUCENE_CURRENT),
> IndexWriter.MaxFieldLength.LIMITED);
> writer.setMergeFactor(16);
>
> Thread writeThread = new Thread(new Runnable() {
> @Override
> public void run() {
> // write test data
> try {
> for (int i = 0; i < 1000000; i++) {
> Document doc = new Document();
> Fieldable itemID = new Field("ItemID", String
> .valueOf(i), Field.Store.YES,
> Field.Index.ANALYZED);
>
> writer.addDocument(doc);
> }
>
> Thread.currentThread().sleep(10000);
> System.out.println("optimize begin");
> writer.optimize();
> System.out.println("optimize end");
> } catch (CorruptIndexException e) {
> e.printStackTrace();
> } catch (IOException e) {
> e.printStackTrace();
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
>
> }
> });
>
> writeThread.start();
>
>
> Thread reopenThread = new Thread(new Runnable() {
> public void run() {
> try {
> IndexReader reader = writer.getReader();
> while (true) {
> Thread.currentThread().sleep(1000);
> IndexReader newReader = writer.getReader();
> if (newReader != reader) {
> reader.close();
> reader = newReader;
> }
> System.out.println("132");
> }
>
> } catch (InterruptedException e) {
> e.printStackTrace();
> } catch (IOException e) {
> e.printStackTrace();
> }
> }
>
> });
>
> reopenThread.start();
> }
>
> }
>
> -----�ʼ�ԭ��-----
> ��: luocan19826164 [at] sohu [mailto:luocan19826164 [at] sohu]
> ��ʱ��: 2010��2��10�� 22:49
> �ռ��: java-user
> ��: RE: here a small test case problem:lucene did not delete old index file after optimize method called
>
> I watched there are three compound files.
> 0.cfs  6786kb
> 1.cfs  2044kb
> 2.cfs  8790kb(the optimize file)
> I think in this testcase only 2.cfs left(the optimize file left),Is that right??
> <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p> </o:p>
> import java.io.File;
> import java.io.IOException;
> <o:p> </o:p>
> import org.apache.lucene.analysis.standard.StandardAnalyzer;
> import org.apache.lucene.document.Document;
> import org.apache.lucene.document.Field; import org.apache.lucene.document.Fieldable;
> import org.apache.lucene.index.CorruptIndexException;
> import org.apache.lucene.index.IndexReader;
> import org.apache.lucene.index.IndexWriter;
> import org.apache.lucene.store.FSDirectory;
> import org.apache.lucene.store.LockObtainFailedException;
> import org.apache.lucene.util.Version;
> <o:p> </o:p>
> public class ReopenTest {
>          IndexWriter writer;
> <o:p> </o:p>
>          public static void main(String[] args) throws CorruptIndexException,
>                             LockObtainFailedException, IOException {
>                    final IndexWriter writer = new
> IndexWriter(FSDirectory.open(new File(
>                                      "data")), new
> StandardAnalyzer(Version.LUCENE_CURRENT),
>                                      IndexWriter.MaxFieldLength.LIMITED);
>                    writer.setMergeFactor(16);
> <o:p> </o:p>
>                    Thread writeThread = new Thread(new Runnable() {
>                             @Override
>                             public void run() {
>                                      // write test data
>                                      try {
>                                                for (int i = 0; i < 1000000; i++) {
>                                                         Document doc = new
> Document();
>                                                         Fieldable itemID = new
> Field("ItemID", String
>                                                                            .valueOf(i),
> Field.Store.YES,
>                                                                            Field.Index.
> ANALYZED);
> <o:p> </o:p>
>                                                         writer.addDocument(doc);
>                                                }
> <o:p> </o:p>
>                                                Thread.currentThread().sleep(10000);
>                                                System.out.println("optimize
> begin");
>                                                writer.optimize();
>                                                System.out.println("optimize end");
>                                      } catch (CorruptIndexException e) {
>                                                e.printStackTrace();
>                                      } catch (IOException e) {
>                                                e.printStackTrace();
>                                      } catch (InterruptedException e) {
>                                                e.printStackTrace();
>                                      }
> <o:p> </o:p>
>                             }
>                    });
> <o:p> </o:p>
>                    writeThread.start();
>                   
>                   
>                    Thread reopenThread = new Thread(new Runnable() {
>                             public void run() {
>                                      try {
>                                                IndexReader reader =
> writer.getReader();
>                                                while (true) {
>         
> Thread.currentThread().sleep(1000);
>                                                         IndexReader newReader =
> writer.getReader();
>                                                         if (newReader != reader) {
>                                                                  reader.close();
>                                                                  reader = newReader;
>                                                         }
>                                                         System.out.println("132");
>                                                }
> <o:p> </o:p>
>                                      } catch (InterruptedException e) {
>                                                e.printStackTrace();
>                                      } catch (IOException e) {
>                                                e.printStackTrace();
>                                      }
>                             }
> <o:p> </o:p>
>                    });
>                   
>                    reopenThread.start();
>          }
> <o:p> </o:p>
> }
>  
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

lucene at mikemccandless

Feb 10, 2010, 8:27 AM

Post #7 of 7 (1143 views)
Permalink

Re: problem:lucene did not delete old index file after optimize method called [In reply to]

OK I opened this issue:

https://issues.apache.org/jira/browse/LUCENE-2259

And put a patch on. If you can try the patch, that'd be great :) You
should be able to apply the patch, build a new jar, then run your test
again unmodified, and 0.cfs, 1.cfs should then be removed.

Mike

2010/2/10 Michael McCandless <lucene [at] mikemccandless>:
> From this test, I would expect all 3 files to be left, because
> IndexWriter never gets another chance to remove the files.
>
> IndexWriter only attempts to remove unreferenced files in roughly 3 places:
>
> * On open
> * On flushing a new segment
> * On finishing a merge
>
> So, the moment your optimize finished, IndexWriter wanted to delete
> 0.cfs and 1.cfs, but could not, because the currently open reader was
> using them. Then, because you don't close/open a new IndexWriter, no
> further attempt is made to delete.
>
> In a normal app, as you continue adding docs, normal merges kick off,
> etc, these unreferenced files would eventually be removed.
>
> It's just taking longer than it should, because IndexWriter doesn't
> know exactly when the reader was closed.
>
> I think we should add an explicit
> IndexWriter.removeUnreferencedFiles() method, so apps can control the
> immediacy if they want to. I'll open an issue.
>
> For your tests, if you add code to stop both threads, close the
> writer, then open & close a new writer, you should see 0.cfs and 1.cfs
> removed.
>
> Mike
>
> On Wed, Feb 10, 2010 at 9:54 AM, luocanrao <luocan19826164 [at] sohu> wrote:
>> here a small test case I watched there are three compound files.
>> 0.cfs 6786kb
>> 1.cfs 2044kb
>> 2.cfs 8790kb(the optimize file)
>> I think in this testcase only 2.cfs left(the optimize file left),Is that right??
>>
>> import java.io.File;
>> import java.io.IOException;
>>
>> import org.apache.lucene.analysis.standard.StandardAnalyzer;
>> import org.apache.lucene.document.Document;
>> import org.apache.lucene.document.Field;
>> import org.apache.lucene.document.Fieldable;
>> import org.apache.lucene.index.CorruptIndexException;
>> import org.apache.lucene.index.IndexReader;
>> import org.apache.lucene.index.IndexWriter;
>> import org.apache.lucene.store.FSDirectory;
>> import org.apache.lucene.store.LockObtainFailedException;
>> import org.apache.lucene.util.Version;
>>
>> public class ReopenTest {
>> IndexWriter writer;
>>
>> public static void main(String[] args) throws CorruptIndexException,
>> LockObtainFailedException, IOException {
>> final IndexWriter writer = new IndexWriter(FSDirectory.open(new File(
>> "data")), new StandardAnalyzer(Version.LUCENE_CURRENT),
>> IndexWriter.MaxFieldLength.LIMITED);
>> writer.setMergeFactor(16);
>>
>> Thread writeThread = new Thread(new Runnable() {
>> @Override
>> public void run() {
>> // write test data
>> try {
>> for (int i = 0; i < 1000000; i++) {
>> Document doc = new Document();
>> Fieldable itemID = new Field("ItemID", String
>> .valueOf(i), Field.Store.YES,
>> Field.Index.ANALYZED);
>>
>> writer.addDocument(doc);
>> }
>>
>> Thread.currentThread().sleep(10000);
>> System.out.println("optimize begin");
>> writer.optimize();
>> System.out.println("optimize end");
>> } catch (CorruptIndexException e) {
>> e.printStackTrace();
>> } catch (IOException e) {
>> e.printStackTrace();
>> } catch (InterruptedException e) {
>> e.printStackTrace();
>> }
>>
>> }
>> });
>>
>> writeThread.start();
>>
>>
>> Thread reopenThread = new Thread(new Runnable() {
>> public void run() {
>> try {
>> IndexReader reader = writer.getReader();
>> while (true) {
>> Thread.currentThread().sleep(1000);
>> IndexReader newReader = writer.getReader();
>> if (newReader != reader) {
>> reader.close();
>> reader = newReader;
>> }
>> System.out.println("132");
>> }
>>
>> } catch (InterruptedException e) {
>> e.printStackTrace();
>> } catch (IOException e) {
>> e.printStackTrace();
>> }
>> }
>>
>> });
>>
>> reopenThread.start();
>> }
>>
>> }
>>
>> -----�ʼ�ԭ��-----
>> ��: luocan19826164 [at] sohu [mailto:luocan19826164 [at] sohu]
>> ��ʱ��: 2010��2��10�� 22:49
>> �ռ��: java-user
>> ��: RE: here a small test case problem:lucene did not delete old index file after optimize method called
>>
>> I watched there are three compound files.
>> 0.cfs  6786kb
>> 1.cfs  2044kb
>> 2.cfs  8790kb(the optimize file)
>> I think in this testcase only 2.cfs left(the optimize file left),Is that right??
>> <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p> </o:p>
>> import java.io.File;
>> import java.io.IOException;
>> <o:p> </o:p>
>> import org.apache.lucene.analysis.standard.StandardAnalyzer;
>> import org.apache.lucene.document.Document;
>> import org.apache.lucene.document.Field; import org.apache.lucene.document.Fieldable;
>> import org.apache.lucene.index.CorruptIndexException;
>> import org.apache.lucene.index.IndexReader;
>> import org.apache.lucene.index.IndexWriter;
>> import org.apache.lucene.store.FSDirectory;
>> import org.apache.lucene.store.LockObtainFailedException;
>> import org.apache.lucene.util.Version;
>> <o:p> </o:p>
>> public class ReopenTest {
>>          IndexWriter writer;
>> <o:p> </o:p>
>>          public static void main(String[] args) throws CorruptIndexException,
>>                             LockObtainFailedException, IOException {
>>                    final IndexWriter writer = new
>> IndexWriter(FSDirectory.open(new File(
>>                                      "data")), new
>> StandardAnalyzer(Version.LUCENE_CURRENT),
>>                                      IndexWriter.MaxFieldLength.LIMITED);
>>                    writer.setMergeFactor(16);
>> <o:p> </o:p>
>>                    Thread writeThread = new Thread(new Runnable() {
>>                             @Override
>>                             public void run() {
>>                                      // write test data
>>                                      try {
>>                                                for (int i = 0; i < 1000000; i++) {
>>                                                         Document doc = new
>> Document();
>>                                                         Fieldable itemID = new
>> Field("ItemID", String
>>                                                                            .valueOf(i),
>> Field.Store.YES,
>>                                                                            Field.Index.
>> ANALYZED);
>> <o:p> </o:p>
>>                                                         writer.addDocument(doc);
>>                                                }
>> <o:p> </o:p>
>>                                                Thread.currentThread().sleep(10000);
>>                                                System.out.println("optimize
>> begin");
>>                                                writer.optimize();
>>                                                System.out.println("optimize end");
>>                                      } catch (CorruptIndexException e) {
>>                                                e.printStackTrace();
>>                                      } catch (IOException e) {
>>                                                e.printStackTrace();
>>                                      } catch (InterruptedException e) {
>>                                                e.printStackTrace();
>>                                      }
>> <o:p> </o:p>
>>                             }
>>                    });
>> <o:p> </o:p>
>>                    writeThread.start();
>>                   
>>                   
>>                    Thread reopenThread = new Thread(new Runnable() {
>>                             public void run() {
>>                                      try {
>>                                                IndexReader reader =
>> writer.getReader();
>>                                                while (true) {
>>         
>> Thread.currentThread().sleep(1000);
>>                                                         IndexReader newReader =
>> writer.getReader();
>>                                                         if (newReader != reader) {
>>                                                                  reader.close();
>>                                                                  reader = newReader;
>>                                                         }
>>                                                         System.out.println("132");
>>                                                }
>> <o:p> </o:p>
>>                                      } catch (InterruptedException e) {
>>                                                e.printStackTrace();
>>                                      } catch (IOException e) {
>>                                                e.printStackTrace();
>>                                      }
>>                             }
>> <o:p> </o:p>
>>                    });
>>                   
>>                    reopenThread.start();
>>          }
>> <o:p> </o:p>
>> }
>>  
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> For additional commands, e-mail: java-user-help [at] lucene
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads