Mailing List Archive by Gossamer Threads http://www.gossamer-threads.com/lists/lucene/java-user/ en-us (c) Gossamer Threads Inc. All rights reserved. 11 Oct 2008 17:25:07 -0800 120 75 23 http://www.gossamer-threads.com/lists/lucene/java-user/ http://www.gossamer-threads.com/images/lists/rss_logo.jpg Release 2.4.0 of Lucene is now available! With 2.4.0 we have relaxed the backwards compatibility policy of the Fieldable interface: we now allow chan 11 Oct 2008 08:50:45 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66225 Hello All, At my university we have over 20,000 small file ranging from 20k to 500k per directory and we would like to index them. I was wondering if 11 Oct 2008 07:22:28 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66223 Hello everyone. I've been fiddeling with the idea of retrieving the top terms from a subset of the index (i.e. top terms from the documents retrie 11 Oct 2008 03:36:04 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66221 Glen, Thank you for the details there. Its really great what you've done and I will study it some more! I too though about using multiple writers i 11 Oct 2008 02:21:08 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66220 IndexWriter is thread-safe and has been for a while (http://www.mail-archive.com/lucene-dev@jakarta.apache.org/msg00157.html) so you don't have to wor 10 Oct 2008 19:17:45 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66219 Hi gang, Wondering how folks have address scaled up indexing. I saw old threads about using clustered webapp with JNDI singleton index writer due to 10 Oct 2008 14:05:16 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66218 Actually looking at this a little deeper maybe Lucene could/should automatically be doing this "short" optimisation here? Given a comparitively small 10 Oct 2008 09:22:13 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66214 I have had a similar problem. What I do is load all the date field values at index startup, convert dates (timestamps) to a Julian date (# of seconds 10 Oct 2008 08:57:19 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66213 That's a really good idea Mark! :) Thanks! Will try to see if can make a quick change with your suggestion. (Too bad quick isn't really a word in my 10 Oct 2008 08:45:02 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66212 Update: The statement "...cost is field size (10 bytes ?) times number of documents" is wrong. What you actually have is the cost of the unique strin 10 Oct 2008 08:07:31 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66209 Yes, I understand that, and I did mean the number of documents, but I read in the javadoc that: "For String fields, the cache is larger: in additio 10 Oct 2008 07:58:14 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66208 I think you have your memory cost calculation wrong. The cost is field size (10 bytes ?) times number of documents NOT number of unique terms. The cac 10 Oct 2008 07:43:35 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66207 Unfortunately no, since the documents that are added may come form a new "source" containing old documents aswell..:/ I tried deploying our webappli 10 Oct 2008 07:25:29 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66206 Assuming content is added in chronological order and with no updates to existing docs couldn't you rely on internal Lucene document id to give a chron 10 Oct 2008 06:18:46 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66205 I'll follow up on my own question... Let's say that we have 4 years of data, meaning that there will be roughly 4 * 365 = 1460 unique terms for our 10 Oct 2008 05:51:50 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66202 True, I guess I was thinking of things from a search-only perspective when I claimed they were identical... But you're absolutely right in that you ca 10 Oct 2008 05:47:34 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66201 The release bits are indeed propagating through all mirrors, but I'm going to wait until tomorrow to do the announcement, to make sure all mirrors 10 Oct 2008 05:31:25 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66200 Hi there, I've just noticed that there is already a 2.4 release available on ibiblio (http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/luce 10 Oct 2008 05:27:40 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66199 Hello, I've read a lot of threads now on memory consumption and sorting, and I think I have a pretty good understanding of how things work, but I 10 Oct 2008 05:09:36 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66198 Ah, sorry. Just saw the bit about the free text query too. A FieldCache is the answer here I suspect in order to quickly retrieve the date values for 10 Oct 2008 03:04:25 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66196 Assuming your date data is held as YYYYMMDD and you want daily totals.... Term startTerm=new Term("date","20080101"); TermEnum termEn 10 Oct 2008 02:40:32 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66195 Hi. Anyone have an idea of how I would create a query which finds the data backing a trend graph where date is X and num(docs) is on Y axis ? This i 10 Oct 2008 02:12:35 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66194 From what I can understand, you want to insert the word "history" and then get proposed "related" terms in combination with your input query. In ess 10 Oct 2008 00:33:58 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66193 Hi Ganesh, Your situation seems pretty straight. I did not really split my database (storage), just that while indexing, I indexed the data into 'n' 09 Oct 2008 21:49:16 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66192 Kalani, They are put into the document as separate fields. That's why there are getFields() and deleteFields() methods on Document. John G. -----Or 09 Oct 2008 19:29:36 -0800 http://www.gossamer-threads.com/lists/lucene/java-user/66191