Mailing List Archive: Lucene: Java-User

回复： Why does this query slow down Lucene?

Index | Next | Previous | View Threaded

zhoucheng2008 at gmail

Aug 15, 2012, 7:09 PM

Post #1 of 4 (303 views)
Permalink

回复： Why does this query slow down Lucene?

The query has been stuck for more than an hour. The total size is less than 1G, and the number of docs is around 100,000. Hardware is ok as it works well with other much more demanding projects.

------------------ 原始邮件 ------------------
发件人: "Li Li"<fancyerii [at] gmail>;
发送时间: 2012年8月16日(星期四) 上午9:59
收件人: "java-user"<java-user [at] lucene>;

主题: Re: Why does this query slow down Lucene?

how slow is it？ are all your searches slow or only that query slow？ how
many docs are indexed and the size of the indexes？ whats the hardware
configuration？
you should describe it clearly to get help.
在 2012-8-16 上午9:28，"zhoucheng2008" <zhoucheng2008 [at] gmail>写道：

> Hi,
>
>
> I have the string "$21 a Day Once a Month" to search on a large index.
>
>
> I escape the $ sign, and the query string looks like:
>
>
> +level:0 +(title:21 title:a title:day title:once title:a title:month)
> release_year:1941 code:movieverycd126087
>
>
> This slows down Lucene. Any idea?
>
>
> Thanks,
> Cheng

fancyerii at gmail

Aug 15, 2012, 7:22 PM

Post #2 of 4 (300 views)
Permalink

Re: 回复： Why does this query slow down Lucene? [In reply to]

use jstack pid to check any deadlock.

On Thu, Aug 16, 2012 at 10:09 AM, zhoucheng2008 <zhoucheng2008 [at] gmail> wrote:
> The query has been stuck for more than an hour. The total size is less than 1G, and the number of docs is around 100,000. Hardware is ok as it works well with other much more demanding projects.
>
>
>
>
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Li Li"<fancyerii [at] gmail>;
> 发送时间: 2012年8月16日(星期四) 上午9:59
> 收件人: "java-user"<java-user [at] lucene>;
>
> 主题: Re: Why does this query slow down Lucene?
>
>
>
> how slow is it？ are all your searches slow or only that query slow？ how
> many docs are indexed and the size of the indexes？ whats the hardware
> configuration？
> you should describe it clearly to get help.
> 在 2012-8-16 上午9:28，"zhoucheng2008" <zhoucheng2008 [at] gmail>写道：
>
>> Hi,
>>
>>
>> I have the string "$21 a Day Once a Month" to search on a large index.
>>
>>
>> I escape the $ sign, and the query string looks like:
>>
>>
>> +level:0 +(title:21 title:a title:day title:once title:a title:month)
>> release_year:1941 code:movieverycd126087
>>
>>
>> This slows down Lucene. Any idea?
>>
>>
>> Thanks,
>> Cheng

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

fancyerii at gmail

Aug 15, 2012, 7:24 PM

Post #3 of 4 (302 views)
Permalink

Re: 回复： Why does this query slow down Lucene? [In reply to]

and also try jmap -heap pid to check whether it runs out of memory
or jstat -gcutil pid 1000

On Thu, Aug 16, 2012 at 10:09 AM, zhoucheng2008 <zhoucheng2008 [at] gmail> wrote:
> The query has been stuck for more than an hour. The total size is less than 1G, and the number of docs is around 100,000. Hardware is ok as it works well with other much more demanding projects.
>
>
>
>
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Li Li"<fancyerii [at] gmail>;
> 发送时间: 2012年8月16日(星期四) 上午9:59
> 收件人: "java-user"<java-user [at] lucene>;
>
> 主题: Re: Why does this query slow down Lucene?
>
>
>
> how slow is it？ are all your searches slow or only that query slow？ how
> many docs are indexed and the size of the indexes？ whats the hardware
> configuration？
> you should describe it clearly to get help.
> 在 2012-8-16 上午9:28，"zhoucheng2008" <zhoucheng2008 [at] gmail>写道：
>
>> Hi,
>>
>>
>> I have the string "$21 a Day Once a Month" to search on a large index.
>>
>>
>> I escape the $ sign, and the query string looks like:
>>
>>
>> +level:0 +(title:21 title:a title:day title:once title:a title:month)
>> release_year:1941 code:movieverycd126087
>>
>>
>> This slows down Lucene. Any idea?
>>
>>
>> Thanks,
>> Cheng

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

1393975679 at qq

Aug 16, 2012, 5:27 PM

Post #4 of 4 (291 views)
Permalink

回复： Why does this query slow down Lucene? [In reply to]

Yes. The query is bit long, which is the major reason.

------------------ 原始邮件 ------------------
发件人: "Trejkaz"<trejkaz [at] trypticon>;
发送时间: 2012年8月16日(星期四) 中午11:09
收件人: "java-user"<java-user [at] lucene>;

主题: Re: Why does this query slow down Lucene?

On Thu, Aug 16, 2012 at 11:27 AM, zhoucheng2008 <zhoucheng2008 [at] gmail> wrote:
>
> +(title:21 title:a title:day title:once title:a title:month)

Looks like you have a fairly big boolean query going on here, and some
of the terms you're using are really common ones like "a".

Are you using AND or OR for the default operator? I think AND was
significantly faster.

I would also consider culling stop words to get things like "a" out of
the picture.

I also find it interesting that Lucene doesn't normalise title:a
title:a to title:a^2 or something. :)

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads