Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-User

Query for "cache" mechanism to used

 

 

Lucene java-user RSS feed   Index | Next | Previous | View Threaded


hiren.t.shah at gmail

Apr 10, 2012, 1:40 PM

Post #1 of 3 (235 views)
Permalink
Query for "cache" mechanism to used

I have a requirement to get results of search query within 1 second for a
database table.The database table is returning results slowly at this
point.A table has to be moved to a cache and searched from there so that
search results come fast.I want to do google type page refresh on my
existing search page -- which means the page should refresh as the user
types.
In order to acheive this the search results should return within one
second.My database is teradata.Its queries are taking 2 to 3 seconds at
least.Hence i want to look for other options like caching.I want to use
cache so that the resuls come fast.

Columns are
*company , Id , Industry, parent ...4 more*
Its a search page.So if user types "ja" all items starting from ja like

*company ------------- Id ------------- Industry --------------parent*
jaico ------------- 222 -------------paints ------------- Jaico asia
Jammy fruits------------- 232-------------food------------- jammy
International

The table contains 3.2 million rows and there are 8 columns that are
present.The search data need to return all 8 columns.Considering byte wise
there are 150 chars per row.So total bytes are 3.2 million * 150 chars =
480 Megabytes .I need to store this much data in cache and then fire search
queries like sql (grouping ,like ,order by) across them.What would be the
best option to use in this case

1. ehcache
2. jboss cache
3. Inifinispan
4. Apache Lucene

Please suggest which option is good .Is it better to do caching in memory
or to use lucene?
*What need to be cached?-->* It is a table of 3.2 million rows with 8
columns.
*Why it is to be cached?-->* It is to be cached so that search results come
*faster* than sql query.If i use sql query it takes very long time.Hence i
want to move towards caching data.


ivan at brusic

Apr 11, 2012, 3:42 PM

Post #2 of 3 (229 views)
Permalink
Re: Query for "cache" mechanism to used [In reply to]

A cache should be independent of the data store. Ehcache works well in
front of Lucene as well as a (relational) database. However, caches
work great for key/value data, so the cache value would be a result
set. Is caching the grouped result good enough?

--
Ivan

On Tue, Apr 10, 2012 at 1:40 PM, Hiren Shah <hiren.t.shah [at] gmail> wrote:
> I have a requirement to get results of search query within 1 second for a
> database table.The database table is returning results slowly at this
> point.A table has to be moved to a cache and searched from there so that
> search results come fast.I want to do google type page refresh on my
> existing search page -- which means the page should refresh as the user
> types.
> In order to acheive this the search results should return within one
> second.My database is teradata.Its queries are taking 2 to 3 seconds at
> least.Hence i want to look for other options like caching.I want to use
> cache so that the resuls come fast.
>
> Columns are
> *company , Id , Industry, parent ...4 more*
> Its a search page.So if user types "ja" all items starting from ja like
>
> *company ------------- Id ------------- Industry --------------parent*
> jaico ------------- 222 -------------paints ------------- Jaico asia
> Jammy fruits------------- 232-------------food------------- jammy
> International
>
> The table contains 3.2 million rows and there are 8 columns that are
> present.The search data need to return all 8 columns.Considering byte wise
> there are 150 chars per row.So total bytes are 3.2 million * 150 chars =
> 480 Megabytes .I need to store this much data in cache and then fire search
> queries like sql (grouping ,like ,order by) across them.What would be the
> best option to use in this case
>
>   1. ehcache
>   2. jboss cache
>   3. Inifinispan
>   4. Apache Lucene
>
> Please suggest which option is good .Is it better to do caching in memory
> or to use lucene?
> *What need to be cached?-->* It is a table of 3.2 million rows with 8
> columns.
> *Why it is to be cached?-->* It is to be cached so that search results come
> *faster* than sql query.If i use sql query it takes very long time.Hence i
> want to move towards caching data.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


hiren.t.shah at gmail

Apr 11, 2012, 9:08 PM

Post #3 of 3 (228 views)
Permalink
Re: Query for "cache" mechanism to used [In reply to]

Hello Ivan
Thanks for the reply.

1. I tried to use lucene .It stored the data on indexes on the hard
disk.The search works well.How of the ehcache above lucene will help
performane.
2. Will Lucene used alone give better performance. *OR *Will ehcache
used above Lucene give better performance?
3. How to make ehcache above Lucene?Any help on it would be
appreciated.I am using sql table to be stored as cache as i need to fire
sql queries like on "multiple" fields.
4. One proble, i came across ehcahce was that for searchable API it
does not allow disk overflow.API fails if i try to do it.So if my data is
500Mb and jvm memory is configured for it then it works fine.But if the
data exceeds 500Mb then it cannot be cached.Is this not a major
problem.is there any workaround on this?

Thanks
Hiren

On Thu, Apr 12, 2012 at 4:12 AM, Ivan Brusic <ivan [at] brusic> wrote:

> A cache should be independent of the data store. Ehcache works well in
> front of Lucene as well as a (relational) database. However, caches
> work great for key/value data, so the cache value would be a result
> set. Is caching the grouped result good enough?
>
> --
> Ivan
>
> On Tue, Apr 10, 2012 at 1:40 PM, Hiren Shah <hiren.t.shah [at] gmail>
> wrote:
> > I have a requirement to get results of search query within 1 second for a
> > database table.The database table is returning results slowly at this
> > point.A table has to be moved to a cache and searched from there so that
> > search results come fast.I want to do google type page refresh on my
> > existing search page -- which means the page should refresh as the user
> > types.
> > In order to acheive this the search results should return within one
> > second.My database is teradata.Its queries are taking 2 to 3 seconds at
> > least.Hence i want to look for other options like caching.I want to use
> > cache so that the resuls come fast.
> >
> > Columns are
> > *company , Id , Industry, parent ...4 more*
> > Its a search page.So if user types "ja" all items starting from ja like
> >
> > *company ------------- Id ------------- Industry --------------parent*
> > jaico ------------- 222 -------------paints ------------- Jaico asia
> > Jammy fruits------------- 232-------------food------------- jammy
> > International
> >
> > The table contains 3.2 million rows and there are 8 columns that are
> > present.The search data need to return all 8 columns.Considering byte
> wise
> > there are 150 chars per row.So total bytes are 3.2 million * 150 chars =
> > 480 Megabytes .I need to store this much data in cache and then fire
> search
> > queries like sql (grouping ,like ,order by) across them.What would be the
> > best option to use in this case
> >
> > 1. ehcache
> > 2. jboss cache
> > 3. Inifinispan
> > 4. Apache Lucene
> >
> > Please suggest which option is good .Is it better to do caching in memory
> > or to use lucene?
> > *What need to be cached?-->* It is a table of 3.2 million rows with 8
> > columns.
> > *Why it is to be cached?-->* It is to be cached so that search results
> come
> > *faster* than sql query.If i use sql query it takes very long time.Hence
> i
> > want to move towards caching data.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>
>

Lucene java-user RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.