Mailing List Archive: Lucene: Java-User

Highlighter and Shingles...

Index | Next | Previous | View Threaded

dawn at digitorial

Apr 20, 2012, 4:58 AM

Post #1 of 3 (165 views)
Permalink

Highlighter and Shingles...

Hi,

Are there any notes on making the highlighter work consistently with a
shingle generated index?
I have a situation where complete matches highlight OK, but partial
matches do not - leading to a number of blank previews...

Our analyser look like:

TokenStream result =
new StopFilter(Version.LUCENE_36,
new ShingleFilter(
new StopFilter(Version.LUCENE_36,
new LowerCaseFilter(Version.LUCENE_36,
new StandardFilter(Version.LUCENE_36,
new
StandardTokenizer(Version.LUCENE_36, reader)
)
),
STOP_CHARS_SET)
),
STOP_WORDS_SET);

--

Rgds.
*Dawn Raison*

sarowe at syr

Apr 20, 2012, 6:28 AM

Post #2 of 3 (159 views)
Permalink

RE: Highlighter and Shingles... [In reply to]

Hi Dawn,

Can you give an example of a "partial match"?

Steve

-----Original Message-----
From: Dawn Zoë Raison [mailto:dawn [at] digitorial]
Sent: Friday, April 20, 2012 7:59 AM
To: java-user [at] lucene
Subject: Highlighter and Shingles...

Hi,

Are there any notes on making the highlighter work consistently with a shingle generated index?
I have a situation where complete matches highlight OK, but partial matches do not - leading to a number of blank previews...

Our analyser look like:

TokenStream result =
new StopFilter(Version.LUCENE_36,
new ShingleFilter(
new StopFilter(Version.LUCENE_36,
new LowerCaseFilter(Version.LUCENE_36,
new StandardFilter(Version.LUCENE_36,
new StandardTokenizer(Version.LUCENE_36, reader)
)
),
STOP_CHARS_SET)
),
STOP_WORDS_SET);

--

Rgds.
*Dawn Raison*

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

dawn at digitorial

Apr 21, 2012, 4:36 AM

Post #3 of 3 (159 views)
Permalink

Re: Highlighter and Shingles... [In reply to]

Steve,

Exactly the right question...

Prompted by your question, further investigation reveals that I need to
move the "access" part of my lucene query into a filter to prevent
non-matching documents getting scored.
In that situation of course the highlighter finds nothing to highlight -
that's what it's supposed to do.

Thanks,
Dawn

On 20/04/2012 14:28, Steven A Rowe wrote:
> Hi Dawn,
>
> Can you give an example of a "partial match"?
>
> Steve
>
>

--

Rgds.
*Dawn Raison*

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads