Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

[nova] Call for Help -- OpenStack API XML Support

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


vishvananda at gmail

Aug 9, 2012, 3:25 PM

Post #1 of 4 (100 views)
Permalink
[nova] Call for Help -- OpenStack API XML Support

Hello Everyone,

We are in the unfortunate position of not knowing how good our OpenStack API XML support is. All of our integration tests use json. Many of the compute extensions don't even have XML deserializers. We also assume that there bugs we don't even know about due to underuse. We need to do something about XML by Folsom, because releasing a buggy api isn't good for anyone.

We don't want to alienate the communtity by dropping XML support, but we also can't recommend it in its current state. If there are people out there who use XML (or want to use XML) we need your help. We need help identifying what works and what doesn't, we need bug reports, and we need merge proposals for the extra serializers and fixes that are important. Nova-core has too much going on right now to tackle XML.

I see a few potential results from this effort.

1) We get a lot of community support and we manage to get XML into usable shape by Folsom.

2) We get enough community support to get the core api working and the most important extensions in place. We release clear documentation on what is expected to work.

3) We get no support, in which case we mark XML support deprecated and encourage people to use JSON only.

Note that other openstack projects only support json, and there are already java bindings that use json so option 3) isn't terrible, but we don't want to go that route without discussing it with the community first. If anyone has alternative solutions or suggestions, feel free to let me know.

Thanks,
Vish
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


berrange at redhat

Aug 10, 2012, 5:47 AM

Post #2 of 4 (90 views)
Permalink
Re: [nova] Call for Help -- OpenStack API XML Support [In reply to]

On Thu, Aug 09, 2012 at 03:25:01PM -0700, Vishvananda Ishaya wrote:
> Hello Everyone,
>
> We are in the unfortunate position of not knowing how good our OpenStack
> API XML support is. All of our integration tests use json. Many of the
> compute extensions don't even have XML deserializers. We also assume that
> there bugs we don't even know about due to underuse. We need to do something
> about XML by Folsom, because releasing a buggy api isn't good for anyone.

I don't know this area of Nova code at all, but I must say I'm
surprised that it actually needs explicit extra work to accept
or output XML, instead of JSON. It should be possible to auto
generate an XML document from a JSON document, and vica-verca.
At which point you'd not need to have 2 sets of tests for JSON
vs XML, all you need test is the XML<->JSON conversions.

I can imagine this isn't done because of a need to maintain
compat with some special XML schema that isn't a direct 1-1
mapping from the JSON schema ? Are there other complications
preventing this being done

> 1) We get a lot of community support and we manage to get XML into usable shape by Folsom.
>
> 2) We get enough community support to get the core api working and the most important extensions in place. We release clear documentation on what is expected to work.
>
> 3) We get no support, in which case we mark XML support deprecated and encourage people to use JSON only.

4) Deprecate the current XML schemas, and declare new XML schemas
which are a 1-1 mapping from the JSON

> Note that other openstack projects only support json, and there are
> already java bindings that use json so option 3) isn't terrible, but
> we don't want to go that route without discussing it with the community
> first. If anyone has alternative solutions or suggestions, feel free to
> let me know.

If we can get to a position where we auto-generate XML from JSON for Nova,
we could do the same for other projects too

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


gabe.westmaas at RACKSPACE

Aug 10, 2012, 6:14 AM

Post #3 of 4 (89 views)
Permalink
Re: [nova] Call for Help -- OpenStack API XML Support [In reply to]

> -----Original Message-----
> From: openstack-
> bounces+gabe.westmaas=rackspace.com [at] lists
> [mailto:openstack-
> bounces+gabe.westmaas=rackspace.com [at] lists] On Behalf Of
> Daniel P. Berrange
> Sent: Friday, August 10, 2012 8:48 AM
> To: Vishvananda Ishaya
> Cc: OpenStack Development Mailing List; openstack [at] lists
> (openstack [at] lists)
> Subject: Re: [Openstack] [nova] Call for Help -- OpenStack API XML Support
>
> On Thu, Aug 09, 2012 at 03:25:01PM -0700, Vishvananda Ishaya wrote:
> > Hello Everyone,
> >
> > We are in the unfortunate position of not knowing how good our
> > OpenStack API XML support is. All of our integration tests use json.
> > Many of the compute extensions don't even have XML deserializers. We
> > also assume that there bugs we don't even know about due to underuse.
> > We need to do something about XML by Folsom, because releasing a buggy
> api isn't good for anyone.
>
> I don't know this area of Nova code at all, but I must say I'm surprised that it
> actually needs explicit extra work to accept or output XML, instead of JSON.
> It should be possible to auto generate an XML document from a JSON
> document, and vica-verca.
> At which point you'd not need to have 2 sets of tests for JSON vs XML, all you
> need test is the XML<->JSON conversions.
>
> I can imagine this isn't done because of a need to maintain compat with some
> special XML schema that isn't a direct 1-1 mapping from the JSON schema ?
> Are there other complications preventing this being done

Correct. Extensions also cause some issues, though it all basically comes back to not being a 1-1 mapping. The reason it's not 1-1 is not because of different data, but just the representation of the same data.

>
> > 1) We get a lot of community support and we manage to get XML into
> usable shape by Folsom.
> >
> > 2) We get enough community support to get the core api working and the
> most important extensions in place. We release clear documentation on
> what is expected to work.
> >
> > 3) We get no support, in which case we mark XML support deprecated and
> encourage people to use JSON only.
>
> 4) Deprecate the current XML schemas, and declare new XML schemas
> which are a 1-1 mapping from the JSON

The biggest problem with this is that either json or XML ends up feeling second class. If you were writing XML for XML's sake it wouldn't come out looking like XML generated from json schemas. I will say that the reason it came out like this in the current API is that XML was first class with auto-generated JSON, which was pretty bad. Rather than going the other way, we opted to keep both optimized for development against the API rather than for development of the API.

Anyway, not saying it can't be done, but I am saying that XML will feel like a second class format. This mostly has to do with figuring out when something should be an attribute of an element vs when it should be a value of an element. I think we ran into some issues around collections that were non-auto-translatable as well, but I'm struggling to remember them.

We can decide to just pick a translation and go with it, of course. Although perhaps we need an expert on enterprise tools to make sure that the XML we generate properly translates into sane objects in those enterprise tools.

>
> > Note that other openstack projects only support json, and there are
> > already java bindings that use json so option 3) isn't terrible, but
> > we don't want to go that route without discussing it with the
> > community first. If anyone has alternative solutions or suggestions,
> > feel free to let me know.
>
> If we can get to a position where we auto-generate XML from JSON for
> Nova, we could do the same for other projects too
>
> Regards,
> Daniel
> --
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


gabe.westmaas at RACKSPACE

Aug 10, 2012, 6:18 AM

Post #4 of 4 (91 views)
Permalink
Re: [nova] Call for Help -- OpenStack API XML Support [In reply to]

> -----Original Message-----
> From: openstack-
> bounces+gabe.westmaas=rackspace.com [at] lists
> [mailto:openstack-
> bounces+gabe.westmaas=rackspace.com [at] lists] On Behalf Of
> Gabe Westmaas
> Sent: Friday, August 10, 2012 9:15 AM
> To: Daniel P. Berrange; Vishvananda Ishaya
> Cc: OpenStack Development Mailing List; openstack [at] lists
> (openstack [at] lists)
> Subject: Re: [Openstack] [nova] Call for Help -- OpenStack API XML Support
>
>
>
> > -----Original Message-----
> > From: openstack-
> > bounces+gabe.westmaas=rackspace.com [at] lists
> > [mailto:openstack-
> > bounces+gabe.westmaas=rackspace.com [at] lists] On Behalf
> Of
> > Daniel P. Berrange
> > Sent: Friday, August 10, 2012 8:48 AM
> > To: Vishvananda Ishaya
> > Cc: OpenStack Development Mailing List; openstack [at] lists
> > (openstack [at] lists)
> > Subject: Re: [Openstack] [nova] Call for Help -- OpenStack API XML
> > Support
> >
> > On Thu, Aug 09, 2012 at 03:25:01PM -0700, Vishvananda Ishaya wrote:
> > > Hello Everyone,
> > >
> > > We are in the unfortunate position of not knowing how good our
> > > OpenStack API XML support is. All of our integration tests use json.
> > > Many of the compute extensions don't even have XML deserializers. We
> > > also assume that there bugs we don't even know about due to underuse.
> > > We need to do something about XML by Folsom, because releasing a
> > > buggy
> > api isn't good for anyone.
> >
> > I don't know this area of Nova code at all, but I must say I'm
> > surprised that it actually needs explicit extra work to accept or output XML,
> instead of JSON.
> > It should be possible to auto generate an XML document from a JSON
> > document, and vica-verca.
> > At which point you'd not need to have 2 sets of tests for JSON vs XML,
> > all you need test is the XML<->JSON conversions.
> >
> > I can imagine this isn't done because of a need to maintain compat
> > with some special XML schema that isn't a direct 1-1 mapping from the
> JSON schema ?
> > Are there other complications preventing this being done
>
> Correct. Extensions also cause some issues, though it all basically comes back
> to not being a 1-1 mapping. The reason it's not 1-1 is not because of different
> data, but just the representation of the same data.
>
> >
> > > 1) We get a lot of community support and we manage to get XML into
> > usable shape by Folsom.
> > >
> > > 2) We get enough community support to get the core api working and
> > > the
> > most important extensions in place. We release clear documentation on
> > what is expected to work.
> > >
> > > 3) We get no support, in which case we mark XML support deprecated
> > > and
> > encourage people to use JSON only.
> >
> > 4) Deprecate the current XML schemas, and declare new XML schemas
> > which are a 1-1 mapping from the JSON
>
> The biggest problem with this is that either json or XML ends up feeling
> second class. If you were writing XML for XML's sake it wouldn't come out
> looking like XML generated from json schemas. I will say that the reason it
> came out like this in the current API is that XML was first class with auto-
> generated JSON, which was pretty bad. Rather than going the other way, we
> opted to keep both optimized for development against the API rather than
> for development of the API.
>
> Anyway, not saying it can't be done, but I am saying that XML will feel like a
> second class format. This mostly has to do with figuring out when something
> should be an attribute of an element vs when it should be a value of an
> element. I think we ran into some issues around collections that were non-
> auto-translatable as well, but I'm struggling to remember them.
>
> We can decide to just pick a translation and go with it, of course. Although
> perhaps we need an expert on enterprise tools to make sure that the XML
> we generate properly translates into sane objects in those enterprise tools.
>

Oh, and, even if we do this, that does not remove the need for additional tests around XML, of course. We still need to make sure that is working correctly.

> >
> > > Note that other openstack projects only support json, and there are
> > > already java bindings that use json so option 3) isn't terrible, but
> > > we don't want to go that route without discussing it with the
> > > community first. If anyone has alternative solutions or suggestions,
> > > feel free to let me know.
> >
> > If we can get to a position where we auto-generate XML from JSON for
> > Nova, we could do the same for other projects too
> >
> > Regards,
> > Daniel
> > --
> > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/
> :|
> > |: http://libvirt.org -o- http://virt-manager.org :|
> > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
> > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack [at] lists
> > Unsubscribe : https://launchpad.net/~openstack
> > More help : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.