Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-Dev

[jira] [Updated] (LUCENE-4120) FST should use packed integer arrays

 

 

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded


jira at apache

Jun 11, 2012, 5:19 AM

Post #1 of 4 (52 views)
Permalink
[jira] [Updated] (LUCENE-4120) FST should use packed integer arrays

[ https://issues.apache.org/jira/browse/LUCENE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4120:
---------------------------------

Affects Version/s: (was: 5.0)
Fix Version/s: (was: 5.0)
4.0

> FST should use packed integer arrays
> ------------------------------------
>
> Key: LUCENE-4120
> URL: https://issues.apache.org/jira/browse/LUCENE-4120
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/FSTs
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Fix For: 4.0
>
>
> There are some places where an int[] could be advantageously replaced with a packed integer array.
> I am thinking (at least) of:
> * FST.nodeAddress (GrowableWriter)
> * FST.inCounts (GrowableWriter)
> * FST.nodeRefToAddress (read-only Reader)
> The serialization/deserialization methods should be modified too in order to take advantage of PackedInts.get{Reader,Writer}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jun 11, 2012, 6:18 AM

Post #2 of 4 (48 views)
Permalink
[jira] [Updated] (LUCENE-4120) FST should use packed integer arrays [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4120:
---------------------------------

Attachment: LUCENE-4120.patch

Patch. I don't fully understand how FST packing works so I would appreciate if someone familiar with it could review this patch.

> FST should use packed integer arrays
> ------------------------------------
>
> Key: LUCENE-4120
> URL: https://issues.apache.org/jira/browse/LUCENE-4120
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/FSTs
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4120.patch
>
>
> There are some places where an int[] could be advantageously replaced with a packed integer array.
> I am thinking (at least) of:
> * FST.nodeAddress (GrowableWriter)
> * FST.inCounts (GrowableWriter)
> * FST.nodeRefToAddress (read-only Reader)
> The serialization/deserialization methods should be modified too in order to take advantage of PackedInts.get{Reader,Writer}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jun 11, 2012, 4:48 PM

Post #3 of 4 (47 views)
Permalink
[jira] [Updated] (LUCENE-4120) FST should use packed integer arrays [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4120:
---------------------------------

Attachment: LUCENE-4120.patch

New patch:
- fixed Kuromoji {{TokenInfoDictionaryBuilder}} (but you will need to run ant build-dict to make tests pass),
- moved {{save}} to {{Mutable}}, {{FST}} now cannot be saved if it has been loaded from disk,
- renamed {{getWriter}} to {{getWriterByFormat}},
- fixed docs.

FST docs say that there is no need to have backward compatibility because FSTs are experimental. Is it still accurate? The fact that FSTs are used in {{MemoryPostingsFormat}} and Kuromoji analyzers makes me feel that this is not true anymore (or at least won't be true anymore when 4.0 is released).

> FST should use packed integer arrays
> ------------------------------------
>
> Key: LUCENE-4120
> URL: https://issues.apache.org/jira/browse/LUCENE-4120
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/FSTs
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4120.patch, LUCENE-4120.patch
>
>
> There are some places where an int[] could be advantageously replaced with a packed integer array.
> I am thinking (at least) of:
> * FST.nodeAddress (GrowableWriter)
> * FST.inCounts (GrowableWriter)
> * FST.nodeRefToAddress (read-only Reader)
> The serialization/deserialization methods should be modified too in order to take advantage of PackedInts.get{Reader,Writer}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jun 12, 2012, 12:34 PM

Post #4 of 4 (46 views)
Permalink
[jira] [Updated] (LUCENE-4120) FST should use packed integer arrays [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-4120:
---------------------------------

Attachment: LUCENE-4120.patch

bq. Can you move the imports under the copyright header in GrowableWriter.java?

Patch updated.

> FST should use packed integer arrays
> ------------------------------------
>
> Key: LUCENE-4120
> URL: https://issues.apache.org/jira/browse/LUCENE-4120
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/FSTs
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4120.patch, LUCENE-4120.patch, LUCENE-4120.patch
>
>
> There are some places where an int[] could be advantageously replaced with a packed integer array.
> I am thinking (at least) of:
> * FST.nodeAddress (GrowableWriter)
> * FST.inCounts (GrowableWriter)
> * FST.nodeRefToAddress (read-only Reader)
> The serialization/deserialization methods should be modified too in order to take advantage of PackedInts.get{Reader,Writer}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.