Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-User

Running query text through an Analyzer without QueryParser?

 

 

Lucene java-user RSS feed   Index | Next | Previous | View Threaded


jattardi at gmail

Jul 30, 2007, 7:52 AM

Post #1 of 6 (542 views)
Permalink
Running query text through an Analyzer without QueryParser?

Following up on my recent question. It has been suggested to me that I can
run the query text through an Analyzer without using the QueryParser. For
example, if I know what field to be searched I can create a PrefixQuery or
WildcardQuery, but still want to process the search text with the same
Analyzer that did the indexing. How do I run a query through an Analyzer
without using the QueryParser... is this possible?


erickerickson at gmail

Jul 30, 2007, 10:49 AM

Post #2 of 6 (510 views)
Permalink
Re: Running query text through an Analyzer without QueryParser? [In reply to]

Would this work?

TokenStream ts = StandardAnalyzer.tokenStream();
while ((Token tok = ts.next()) != null) {
do whatever
}

Best
Erick

On 7/30/07, Joe Attardi <jattardi[at]gmail.com> wrote:
>
> Following up on my recent question. It has been suggested to me that I can
> run the query text through an Analyzer without using the QueryParser. For
> example, if I know what field to be searched I can create a PrefixQuery or
> WildcardQuery, but still want to process the search text with the same
> Analyzer that did the indexing. How do I run a query through an Analyzer
> without using the QueryParser... is this possible?
>


jattardi at gmail

Jul 30, 2007, 10:57 AM

Post #3 of 6 (506 views)
Permalink
Re: Running query text through an Analyzer without QueryParser? [In reply to]

So then would I just concatenate the tokens together to form the query text?

--
Joe Attardi
jattardi[at]gmail.com
http://thinksincode.blogspot.com/

On 7/30/07, Erick Erickson <erickerickson[at]gmail.com> wrote:
>
> Would this work?
>
> TokenStream ts = StandardAnalyzer.tokenStream();
> while ((Token tok = ts.next()) != null) {
> do whatever
> }
>
> Best
> Erick
>
> On 7/30/07, Joe Attardi <jattardi[at]gmail.com> wrote:
> >
> > Following up on my recent question. It has been suggested to me that I
> can
> > run the query text through an Analyzer without using the QueryParser.
> For
> > example, if I know what field to be searched I can create a PrefixQuery
> or
> > WildcardQuery, but still want to process the search text with the same
> > Analyzer that did the indexing. How do I run a query through an Analyzer
> > without using the QueryParser... is this possible?
> >
>


a.schrijvers at hippo

Jul 30, 2007, 11:04 AM

Post #4 of 6 (505 views)
Permalink
RE: Running query text through an Analyzer without QueryParser? [In reply to]

>
> So then would I just concatenate the tokens together to form
> the query text?

You might better create a TermQuery for each token instead of concatenating, and combine them in a BooleanQuery and say wether all terms must or should occur. Very simple, see [1]

Regards Ard

[1] http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/javadoc/org/apache/lucene/search/BooleanQuery.html

>
> --
> Joe Attardi
> jattardi[at]gmail.com
> http://thinksincode.blogspot.com/
>
> On 7/30/07, Erick Erickson <erickerickson[at]gmail.com> wrote:
> >
> > Would this work?
> >
> > TokenStream ts = StandardAnalyzer.tokenStream();
> > while ((Token tok = ts.next()) != null) {
> > do whatever
> > }
> >
> > Best
> > Erick
> >
> > On 7/30/07, Joe Attardi <jattardi[at]gmail.com> wrote:
> > >
> > > Following up on my recent question. It has been suggested
> to me that I
> > can
> > > run the query text through an Analyzer without using the
> QueryParser.
> > For
> > > example, if I know what field to be searched I can create
> a PrefixQuery
> > or
> > > WildcardQuery, but still want to process the search text
> with the same
> > > Analyzer that did the indexing. How do I run a query
> through an Analyzer
> > > without using the QueryParser... is this possible?
> > >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-user-help[at]lucene.apache.org


jattardi at gmail

Jul 30, 2007, 2:25 PM

Post #5 of 6 (504 views)
Permalink
Re: Running query text through an Analyzer without QueryParser? [In reply to]

What about the case where I want to search a MAC address? For example,
00:14:da:81:21:4f will be split by the StandardTokenizer as the tokens
"00", "14", "da", "81", "21", and "4f".

Suppose I want to search for 00:14:da:81:21:4f. In the search box, I type
00:14:da:81:21:4f. But because these are all separate tokens, it would still
find a match if it had all the tokens but in a different order - for
example, that query would also find a MAC of da:14:4f:21:00:81. Is there
some way to enforce the order in which terms appear, or should I just index
a MAC address as UN_TOKENIZED ?

Thanks

--
Joe Attardi
jattardi[at]gmail.com
http://thinksincode.blogspot.com/

On 7/30/07, Ard Schrijvers <a.schrijvers[at]hippo.nl> wrote:
>
>
> >
> > So then would I just concatenate the tokens together to form
> > the query text?
>
> You might better create a TermQuery for each token instead of
> concatenating, and combine them in a BooleanQuery and say wether all terms
> must or should occur. Very simple, see [1]
>
> Regards Ard
>
> [1]
> http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/javadoc/org/apache/lucene/search/BooleanQuery.html
>
> >
> > --
> > Joe Attardi
> > jattardi[at]gmail.com
> > http://thinksincode.blogspot.com/
> >
> > On 7/30/07, Erick Erickson <erickerickson[at]gmail.com> wrote:
> > >
> > > Would this work?
> > >
> > > TokenStream ts = StandardAnalyzer.tokenStream();
> > > while ((Token tok = ts.next()) != null) {
> > > do whatever
> > > }
> > >
> > > Best
> > > Erick
> > >
> > > On 7/30/07, Joe Attardi <jattardi[at]gmail.com> wrote:
> > > >
> > > > Following up on my recent question. It has been suggested
> > to me that I
> > > can
> > > > run the query text through an Analyzer without using the
> > QueryParser.
> > > For
> > > > example, if I know what field to be searched I can create
> > a PrefixQuery
> > > or
> > > > WildcardQuery, but still want to process the search text
> > with the same
> > > > Analyzer that did the indexing. How do I run a query
> > through an Analyzer
> > > > without using the QueryParser... is this possible?
> > > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe[at]lucene.apache.org
> For additional commands, e-mail: java-user-help[at]lucene.apache.org
>
>


erickerickson at gmail

Jul 30, 2007, 5:01 PM

Post #6 of 6 (497 views)
Permalink
Re: Running query text through an Analyzer without QueryParser? [In reply to]

*SpanNearQuery<file:///C:/lucene-2.1.0/docs/api/org/apache/lucene/search/spans/SpanNearQuery.html#SpanNearQuery%28org.apache.lucene.search.spans.SpanQuery%5B%5D,%20int,%20boolean%29>
*(SpanQuery<file:///C:/lucene-2.1.0/docs/api/org/apache/lucene/search/spans/SpanQuery.html>[]
clauses,
int slop, boolean inOrder)

Erick

On 7/30/07, Joe Attardi <jattardi[at]gmail.com> wrote:
>
> What about the case where I want to search a MAC address? For example,
> 00:14:da:81:21:4f will be split by the StandardTokenizer as the tokens
> "00", "14", "da", "81", "21", and "4f".
>
> Suppose I want to search for 00:14:da:81:21:4f. In the search box, I type
> 00:14:da:81:21:4f. But because these are all separate tokens, it would
> still
> find a match if it had all the tokens but in a different order - for
> example, that query would also find a MAC of da:14:4f:21:00:81. Is there
> some way to enforce the order in which terms appear, or should I just
> index
> a MAC address as UN_TOKENIZED ?
>
> Thanks
>
> --
> Joe Attardi
> jattardi[at]gmail.com
> http://thinksincode.blogspot.com/
>
> On 7/30/07, Ard Schrijvers <a.schrijvers[at]hippo.nl> wrote:
> >
> >
> > >
> > > So then would I just concatenate the tokens together to form
> > > the query text?
> >
> > You might better create a TermQuery for each token instead of
> > concatenating, and combine them in a BooleanQuery and say wether all
> terms
> > must or should occur. Very simple, see [1]
> >
> > Regards Ard
> >
> > [1]
> >
> http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/javadoc/org/apache/lucene/search/BooleanQuery.html
> >
> > >
> > > --
> > > Joe Attardi
> > > jattardi[at]gmail.com
> > > http://thinksincode.blogspot.com/
> > >
> > > On 7/30/07, Erick Erickson <erickerickson[at]gmail.com> wrote:
> > > >
> > > > Would this work?
> > > >
> > > > TokenStream ts = StandardAnalyzer.tokenStream();
> > > > while ((Token tok = ts.next()) != null) {
> > > > do whatever
> > > > }
> > > >
> > > > Best
> > > > Erick
> > > >
> > > > On 7/30/07, Joe Attardi <jattardi[at]gmail.com> wrote:
> > > > >
> > > > > Following up on my recent question. It has been suggested
> > > to me that I
> > > > can
> > > > > run the query text through an Analyzer without using the
> > > QueryParser.
> > > > For
> > > > > example, if I know what field to be searched I can create
> > > a PrefixQuery
> > > > or
> > > > > WildcardQuery, but still want to process the search text
> > > with the same
> > > > > Analyzer that did the indexing. How do I run a query
> > > through an Analyzer
> > > > > without using the QueryParser... is this possible?
> > > > >
> > > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe[at]lucene.apache.org
> > For additional commands, e-mail: java-user-help[at]lucene.apache.org
> >
> >
>

Lucene java-user RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.