simon.willnauer at gmail
Aug 3, 2012, 12:42 AM
Views: 569
Permalink
|
On Thu, Aug 2, 2012 at 11:09 PM, Bill Chesky
<Bill.Chesky [at] learninga-z> wrote:
> Hi,
>
> I understand that generally speaking you should use the same analyzer on querying as was used on indexing. In my code I am using the SnowballAnalyzer on index creation. However, on the query side I am building up a complex BooleanQuery from other BooleanQuerys and/or PhraseQuerys on several fields. None of these require specifying an analyzer anywhere. This is causing some odd results, I think, because a different analyzer (or no analyzer?) is being used for the query.
>
> Question: how do I build my boolean and phrase queries using the SnowballAnalyzer?
>
> One thing I did that seemed to kind of work was to build my complex query normally then build a snowball-analyzed query using a QueryParser instantiated with a SnowballAnalyzer. To do this, I simply pass the string value of the complex query to the QueryParser.parse() method to get the new query. Something like this:
>
> // build a complex query from other BooleanQuerys and PhraseQuerys
> BooleanQuery fullQuery = buildComplexQuery();
> QueryParser parser = new QueryParser(Version.LUCENE_30, "title", new SnowballAnalyzer(Version.LUCENE_30, "English"));
> Query snowballAnalyzedQuery = parser.parse(fullQuery.toString());
>
> TopScoreDocCollector collector = TopScoreDocCollector.create(10000, true);
> indexSearcher.search(snowballAnalyzedQuery, collector);
you can just use the analyzer directly like this:
Analyzer analyzer = new SnowballAnalyzer(Version.LUCENE_30, "English");
TokenStream stream = analyzer.tokenStream("title", new
StringReader(fullQuery.toString()):
CharTermAttribute termAttr = stream.addAttribute(CharTermAttribute.class);
stream.reset();
BooleanQuery q = new BooleanQuery();
while(stream.incrementToken()) {
q.addClause(new BooleanClause(Occur.MUST, new Term("title",
termAttr.toString())));
}
you also have access to the token positions if you want to create
phrase queries etc. just add a PositionIncrementAttribute like this:
PositionIncrementAttribute posAttr =
stream.addAttribute(PositionsIncrementAttribute.class);
pls. doublecheck the code it's straight from the top of my head.
simon
>
> Like I said, this seems to kind of work but it doesn't feel right. Does this make sense? Is there a better way?
>
> thanks in advance,
>
> Bill
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene
|
Re: Analyzer on query question