Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-Dev

[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent

 

 

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded


jira at apache

Jul 3, 2012, 9:38 PM

Post #1 of 8 (102 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406270#comment-13406270 ]

David Smiley commented on LUCENE-4188:
--------------------------------------

I think storing a shape should universally be SpatialContext.toString(shape);
This is just a simple method call for the strategy to invoke. If/when Lucene separates out better stored fields from indexed ones then the Strategy need not bother.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 3, 2012, 10:06 PM

Post #2 of 8 (95 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406278#comment-13406278 ]

Chris Male commented on LUCENE-4188:
------------------------------------

I think it is bizarre to first parse the String to a Shape, pass the Shape into the Strategy, and then the Strategy to ask for the String again. I think we should let the user decide what they want to store and how it should be stored. Even if we do decide to use SpatialContext.toString as the default behavior, it doesn't belong inside the Strategys. Solr can create a separate Field instance for the stored value and so can other users of the module.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 4, 2012, 6:26 AM

Post #3 of 8 (93 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406504#comment-13406504 ]

David Smiley commented on LUCENE-4188:
--------------------------------------

Makes sense. Is telling Lucene to store a field independent of indexing now? I forget what issue it was that is trying to separate it; I recall you are involved in that one Chris. Even if that isn't done, if the Strategy gives it to Lucene to index, can the caller (e.g. Solr / the user of the API) add it by the same name as stored? If it can then I agree we can move field storage out.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 4, 2012, 6:36 AM

Post #4 of 8 (86 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406513#comment-13406513 ]

Chris Male commented on LUCENE-4188:
------------------------------------

bq. if the Strategy gives it to Lucene to index, can the caller (e.g. Solr / the user of the API) add it by the same name as stored?

I think the Strategy should continue to return the Fields to the caller so they can be added to a Document and indexed at the appropriate time. But yes, you can add a stored field with the same name and everything is fine. We are definitely decoupling indexed and stored fields more and more beneath IndexWriter, but to Document you can either have one Field instance for both indexed and stored, or two different instances.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 4, 2012, 6:57 PM

Post #5 of 8 (85 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406819#comment-13406819 ]

David Smiley commented on LUCENE-4188:
--------------------------------------

Awesome.
+1

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 4, 2012, 10:36 PM

Post #6 of 8 (93 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406859#comment-13406859 ]

Chris Male commented on LUCENE-4188:
------------------------------------

Thanks for taking this on David.

bq. I added a convenience method strategy.createStoredField()

I don't really like this. It is barely an improvement on the current code. The whole point of this issue is that the storing of Shapes shouldn't be related to Strategys. I think we should be explicit and require the consumer code (Solr or something else) decides how it wants to store Shapes. If you want a convenience method then it should be static, illustrating it is a utility that the Strategys cannot override. Ideally I would like it somewhere else entirely.

bq. Perhaps createFields should be renamed to createIndexedFields() ?

+1 I think that makes its role clearer.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
> Assignee: David Smiley
> Attachments: LUCENE-4188_remove_field_storage_from_createField.patch
>
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 5, 2012, 8:48 AM

Post #7 of 8 (88 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407221#comment-13407221 ]

David Smiley commented on LUCENE-4188:
--------------------------------------

RE createStoredField():
bq. I don't really like this. It is barely an improvement on the current code. The whole point of this issue is that the storing of Shapes shouldn't be related to Strategys. I think we should be explicit and require the consumer code (Solr or something else) decides how it wants to store Shapes. If you want a convenience method then it should be static, illustrating it is a utility that the Strategys cannot override. Ideally I would like it somewhere else entirely.

The client doesn't have to use this method, but in all tests + the Solr adapters I don't think there's a reason not to. I found it to be useful, and to provide a place to document how it is recommended to store the shape (notice I even included the one-liner source in the javadocs). An advantage of it being an instance method on the Strategy is that it has convenient access to both the field name & SpatialContext. I could make this method final, and I could add more documentation that makes it clear that the user is free to store the shape in any way they wish since the spatial module doesn't care.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
> Assignee: David Smiley
> Attachments: LUCENE-4188_remove_field_storage_from_createField.patch
>
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Jul 5, 2012, 7:58 PM

Post #8 of 8 (88 views)
Permalink
[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407668#comment-13407668 ]

Chris Male commented on LUCENE-4188:
------------------------------------

{quote}
The client doesn't have to use this method, but in all tests + the Solr adapters I don't think there's a reason not to. I found it to be useful, and to provide a place to document how it is recommended to store the shape (notice I even included the one-liner source in the javadocs). An advantage of it being an instance method on the Strategy is that it has convenient access to both the field name & SpatialContext. I could make this method final, and I could add more documentation that makes it clear that the user is free to store the shape in any way they wish since the spatial module doesn't care.
{quote}

I think it is a misnomer to say this is how we recommend a shape is stored. That is exactly what I disagree with and why I want it changed from how it is currently. To me the recommended way, if the consumer already has a String representation of the Shape that they used to construct the Shape, is to just store that String rather than re-parsing the Shape. I also dont like the use of SpatialContext. It represents Shapes in a non-standard way, sometimes using WKT (if you're using JtsSpatialContext) and sometimes not. Yes it is symmetrical with the readShape code, but that is only helpful if you use the Context to read your shapes in the first place.

Thinking about it, I think we need to improve the Shape reading code in SpatialContext so that it is clear what representation is used and so it is possible to extend the functionality when you want to use anything other than Strings.

> Storing Shapes shouldn't be Strategy dependent
> ----------------------------------------------
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/spatial
> Reporter: Chris Male
> Assignee: David Smiley
> Attachments: LUCENE-4188_remove_field_storage_from_createField.patch
>
>
> The logic for storing Shape representations seems to be different for each Strategy. The PrefixTreeStrategy impls store the Shape in WKT, which is nice if you're using WKT but not much help if you're not. BBoxStrategy doesn't actually store the Shape itself, but a representation of the bounding box. TwoDoubles seems to follow the PrefixTreeStrategy approach, which is surprising since it only indexes Points and they could be stored without using WKT.
> I think we need to consider what storing a Shape means. If we want to store the Shape itself, then that logic should be standardised and done outside of the Strategys since it is not really related to them. If we want to store the terms being used by the Strategys to make Shapes queryable, then we need to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.