Mailing List Archive: Lucene: Java-Dev

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD)

Index | Next | Previous | View Threaded

jira at apache

Jan 27, 2012, 2:04 AM

Post #1 of 6 (43 views)
Permalink

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD)

[ https://issues.apache.org/jira/browse/SOLR-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194587#comment-13194587 ]

Jan Høydahl commented on SOLR-1758:
-----------------------------------

Hi Mike,

I have not tested your patch yet, but I think better validation and config-error is overdue, so I marked this for 4.0, hoping to revive the discussion again.

I like the flexibility of your patch and that its use is optional.
It won't solve all our validation needs, but perhaps it will catch the majority as well as making assisted schema/config editing with standard XML editors easier.

To go the last mile, such as validating config syntax for custom plugins etc, we could let them to their own validation. E.g. NamedListInitializedPlugin users may validate everything they get in their NamedList in init() instead of just checking for existence of a few selected params.

> schema definition for configuration files (validation, XSD)
> -----------------------------------------------------------
>
> Key: SOLR-1758
> URL: https://issues.apache.org/jira/browse/SOLR-1758
> Project: Solr
> Issue Type: New Feature
> Reporter: Jorg Heymans
> Labels: configuration, schema.xml, solrconfig.xml, validation, xsd
> Fix For: 4.0
>
> Attachments: config-validation-20110523.patch
>
>
> It is too easy to make configuration errors in Solr without getting warnings. We should explore ways of validation configurations. See mailing list discussion at http://search-lucene.com/m/h6xKf1EShE6

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Jan 27, 2012, 4:53 AM

Post #2 of 6 (36 views)
Permalink

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD) [In reply to]

[ https://issues.apache.org/jira/browse/SOLR-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194662#comment-13194662 ]

Mike Sokolov commented on SOLR-1758:
------------------------------------

I'm not sure if plugins use the Config class? It's been almost a year since I looked at this. But if they are getting a NamedList, it's probably too late (in their init()) to validate their input using a formal XML schema. Perhaps there could be a hook though for extending the config.xsd so that plugin config can be validated in the same step as the rest of the config. I believe we could do this using some mechanism of include files in the schemas; plugin authors would put their own schemas in an extension.xsd?

> schema definition for configuration files (validation, XSD)
> -----------------------------------------------------------
>
> Key: SOLR-1758
> URL: https://issues.apache.org/jira/browse/SOLR-1758
> Project: Solr
> Issue Type: New Feature
> Reporter: Jorg Heymans
> Labels: configuration, schema.xml, solrconfig.xml, validation, xsd
> Fix For: 4.0
>
> Attachments: config-validation-20110523.patch
>
>
> It is too easy to make configuration errors in Solr without getting warnings. We should explore ways of validation configurations. See mailing list discussion at http://search-lucene.com/m/h6xKf1EShE6

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Jan 27, 2012, 5:15 AM

Post #3 of 6 (36 views)
Permalink

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD) [In reply to]

[ https://issues.apache.org/jira/browse/SOLR-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194685#comment-13194685 ]

Jan Høydahl commented on SOLR-1758:
-----------------------------------

That's why I think it makes more sense to let plugins validate the contents of their NamedList in Java code, instead of in XSD. But your XSDs will catch 90% and is a great step towards better validation. But of course, if there's an easy way to do addon XSDs for plugins then fine :)

> schema definition for configuration files (validation, XSD)
> -----------------------------------------------------------
>
> Key: SOLR-1758
> URL: https://issues.apache.org/jira/browse/SOLR-1758
> Project: Solr
> Issue Type: New Feature
> Reporter: Jorg Heymans
> Labels: configuration, schema.xml, solrconfig.xml, validation, xsd
> Fix For: 4.0
>
> Attachments: config-validation-20110523.patch
>
>
> It is too easy to make configuration errors in Solr without getting warnings. We should explore ways of validation configurations. See mailing list discussion at http://search-lucene.com/m/h6xKf1EShE6

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Feb 3, 2012, 11:07 AM

Post #4 of 6 (31 views)
Permalink

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD) [In reply to]

[ https://issues.apache.org/jira/browse/SOLR-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199951#comment-13199951 ]

Jan Høydahl commented on SOLR-1758:
-----------------------------------

In light of SOLR-3093 and SOLR-1052, have you thought about how to handle validation of older versions of config/schema? A Solr1.3 user may upgrade to Solr4.0 and simply point it to the old configuration. Some syntax and parameters are deprecated and removed between versions.

Perhaps a solution is to NOT try to validate older versions of configs, and when we go from 4.0 to 4.1, there will be a new config_41.xsd in addition to config_40.xsd, and your code will select xsd based on LuceneMatchVersion?

> schema definition for configuration files (validation, XSD)
> -----------------------------------------------------------
>
> Key: SOLR-1758
> URL: https://issues.apache.org/jira/browse/SOLR-1758
> Project: Solr
> Issue Type: New Feature
> Reporter: Jorg Heymans
> Labels: configuration, schema.xml, solrconfig.xml, validation, xsd
> Fix For: 4.0
>
> Attachments: config-validation-20110523.patch
>
>
> It is too easy to make configuration errors in Solr without getting warnings. We should explore ways of validation configurations. See mailing list discussion at http://search-lucene.com/m/h6xKf1EShE6

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Feb 5, 2012, 2:07 PM

Post #5 of 6 (27 views)
Permalink

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD) [In reply to]

[ https://issues.apache.org/jira/browse/SOLR-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200933#comment-13200933 ]

Mike Sokolov commented on SOLR-1758:
------------------------------------

Yes - the schema will have to evolve as the config files evolve, so there will be a need for a new version, probably with each release.

I think matching using LuceneMatchVersion makes a lot of sense. For versions that are old enough (eg a different major release), the validator could still run, but produce warnings only. Or else it could simply produce a message saying: "warning; stale config version, not validating" or something to that effect. I'm not clear on how reasonable it is to maintain an old config version: isn't this the kind of thing that users will *want* to be prompted to revisit?

> schema definition for configuration files (validation, XSD)
> -----------------------------------------------------------
>
> Key: SOLR-1758
> URL: https://issues.apache.org/jira/browse/SOLR-1758
> Project: Solr
> Issue Type: New Feature
> Reporter: Jorg Heymans
> Labels: configuration, schema.xml, solrconfig.xml, validation, xsd
> Fix For: 4.0
>
> Attachments: config-validation-20110523.patch
>
>
> It is too easy to make configuration errors in Solr without getting warnings. We should explore ways of validation configurations. See mailing list discussion at http://search-lucene.com/m/h6xKf1EShE6

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Feb 5, 2012, 4:37 PM

Post #6 of 6 (29 views)
Permalink

[jira] [Commented] (SOLR-1758) schema definition for configuration files (validation, XSD) [In reply to]

[ https://issues.apache.org/jira/browse/SOLR-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200979#comment-13200979 ]

Jan Høydahl commented on SOLR-1758:
-----------------------------------

A "not validating" warning due to pre Solr4.x version sounds good to me.

Would it make more sense to keep the xsd file(s) inside the WAR instead of in example/solr/conf? This way you don't need to copy all the schemas (for v4.0, 4.1, 4.2 etc) around with your solr config. Then add a JavaOpt which can disable validation -{{Dsolr.validate=false}}

> schema definition for configuration files (validation, XSD)
> -----------------------------------------------------------
>
> Key: SOLR-1758
> URL: https://issues.apache.org/jira/browse/SOLR-1758
> Project: Solr
> Issue Type: New Feature
> Reporter: Jorg Heymans
> Labels: configuration, schema.xml, solrconfig.xml, validation, xsd
> Fix For: 4.0
>
> Attachments: config-validation-20110523.patch
>
>
> It is too easy to make configuration errors in Solr without getting warnings. We should explore ways of validation configurations. See mailing list discussion at http://search-lucene.com/m/h6xKf1EShE6

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads