Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

Image API v2 Draft 4

 

 

First page Previous page 1 2 Next page Last page  View All OpenStack dev RSS feed   Index | Next | Previous | View Threaded


brian.waldon at rackspace

Apr 8, 2012, 9:31 PM

Post #1 of 36 (591 views)
Permalink
Image API v2 Draft 4

The next draft of the v2 OpenStack Image API is available for review:

https://docs.google.com/document/d/1rb7ZVn0Du_5NZqUyQpqUZSmv7Qd66TMHYAtvsow7LH4/edit.

Unfortunately, there won't be time for an official session at the summit, but feel free to start a discussion through email or by using the comment system in the doc itself. I'll also be available at the summit to discuss this in an unofficial capacity. There are still a few kinks to work out, but we're finally getting started implementing this thing. Thanks to everyone who has helped out so far!

Brian Waldon


APPENDIX A: Major Changes From Draft 3

1) Removed concept of image locations - Replication and multiple image locations are not yet well-formed. It's unclear what information/capabilities we need in the API at the moment, so I've simplified it down to just GET /images/<IMAGE_ID>/file. I would prefer to keep the public-facing API as simple as possible, so maybe this belongs in an Admin-only API. We can iterate on this for the next 6 months if we need to…

2) Replaced access_type with visibility - Rather than 'public', 'private' and 'shared', visibility can be either 'public' or 'private'. Added some explanation of how 'visibility' plays with 'owner' in the concepts section of the doc.


APPENDIX B: Outstanding issues

Some of this stuff is hard, some I just haven't put the time into figuring out yet:

1) Does it make sense to use tags, metadata, or both? I can see cases for both, but is that overkill?

2) How do we fit the existing 'copy_from' functionality in?

3) Need to properly convey link objects in json schemas

4) Need to write xsds :(

5) What link relation do we use for subcollections like tags, access, etc? http://www.iana.org/assignments/link-relations/link-relations.xml


justin at fathomdb

Apr 9, 2012, 9:56 AM

Post #2 of 36 (583 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

>
> APPENDIX B: Outstanding issues
>
> 4) Need to write xsds :(
>

This is easy if you design a model which works with XML. If you have an
XML compatible model, you can generate an XSD and a JSON model from that.
Also, it means you can just use common middleware to map XML to JSON,
rather than coding XML parsing functions.

I think the one thing you'd have to change is that XML won't allow you to
serialize a dictionary with elements as keys (e.g. { "key1": "value1",
"key2": "value2" }, you have to do this: [. { "key": "key1", "value":
"value1" }, { "key": "key2", "value": "value2" } ]

I think this is only done in the image properties.

On the downside: The JSON isn't as JSON-ic as it could be.
On the upside: You need never worry about XML again


jaypipes at gmail

Apr 9, 2012, 10:11 AM

Post #3 of 36 (580 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On 04/09/2012 12:56 PM, Justin Santa Barbara wrote:
> APPENDIX B: Outstanding issues
>
> 4) Need to write xsds :(
>
> This is easy if you design a model which works with XML. If you have an
> XML compatible model, you can generate an XSD and a JSON model from
> that. Also, it means you can just use common middleware to map XML to
> JSON, rather than coding XML parsing functions.

Justin, what does "design a model which works with XML" mean?

> I think the one thing you'd have to change is that XML won't allow you
> to serialize a dictionary with elements as keys (e.g. { "key1":
> "value1", "key2": "value2" }, you have to do this: [. { "key": "key1",
> "value": "value1" }, { "key": "key2", "value": "value2" } ]
>
> I think this is only done in the image properties.

No, the image properties have been removed in the 2.0 Images API and
replaced with the standard concept of "tags":

https://docs.google.com/document/d/1rb7ZVn0Du_5NZqUyQpqUZSmv7Qd66TMHYAtvsow7LH4/edit#heading=h.y10rq9pdt27y

> On the downside: The JSON isn't as JSON-ic as it could be.

In what ways?

> On the upside: You need never worry about XML again

We haven't worried about it at all up until now. And Glance has been
perfectly fine without it. ;)

See you in a week!

-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


justin at fathomdb

Apr 9, 2012, 11:16 AM

Post #4 of 36 (581 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

>
> Justin, what does "design a model which works with XML" mean?


Simply avoiding the handful of things that are specific to JSON (or
specific to XML). Nothing too onerous (no angle brackets)!


> I think this is only done in the image properties.
>>
>
> No, the image properties have been removed in the 2.0 Images API and
> replaced with the standard concept of "tags":
>
> https://docs.google.com/**document/d/1rb7ZVn0Du_**
> 5NZqUyQpqUZSmv7Qd66TMHYAtvsow7**LH4/edit#heading=h.**y10rq9pdt27y<https://docs.google.com/document/d/1rb7ZVn0Du_5NZqUyQpqUZSmv7Qd66TMHYAtvsow7LH4/edit#heading=h.y10rq9pdt27y>


I was looking at properties in "GET /schemas/image/access" and "GET
/schemas/access". In general, the Glance 2.0 model is very close to being
XML-compatible.

On the downside: The JSON isn't as JSON-ic as it could be.
>>
>
> In what ways?


When you're designing JSON considering only JSON, you'd probably use {
key1: value1 } - as you have done. If you're designing generically, you'd
probably use { key: key1, value: value1 }.

On the upside: You need never worry about XML again
>>
>
> We haven't worried about it at all up until now. And Glance has been
> perfectly fine without it. ;)
>

Well yes, I agree that our XML support generally is sufficiently buggy as
to be unusable, hence unused. I don't actually care about XML, I just care
about having a well-specified API that works.

In my opinion, the way to do that is to specify a model, and we have JSON
and XML representations of that model. If we design the model right, JSON
& XML both "just work" without any code (and less code = less bugs).
Moreover, when someone introduces another data interchange format, we
again just write one piece of middleware, one documentation generator, and
we're done. Doing this by hand for each format is simply busywork.

I don't care whether we specify the model as a Python class, Java class,
XML schema, JSON schema, or using stick figures with long arms pointing at
boxes. You can even specify it in JSON if you stick to a "lowest common
denominator" subset. I think the only thing you need to avoid is "no
changing-at-runtime keys"; I think this makes it compatible with XML, Avro,
ProtocolBuffers & Thrift.

I don't think it should be a big deal for the JSON format to avoid dynamic
keys, given how much you win whenever you have to support a new format.

Incidentally, I just heard about yet another new format - apparently this
one is going to be the standard for use with node.js
- Hypertext-Powered-State-Transfer-Representation. Apparently JSON is too
complicated now - with its strict need for quote marks and multiple data
types and all that over engineering, so HPSTR just sends the values; you
just use convention over configuration to infer the keys. I trust it's not
going to be a lot of work for OpenStack to support it.

> See you in a week!

Looking forward to it!


jaypipes at gmail

Apr 9, 2012, 1:27 PM

Post #5 of 36 (585 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On 04/09/2012 02:16 PM, Justin Santa Barbara wrote:
> Justin, what does "design a model which works with XML" mean?
>
> Simply avoiding the handful of things that are specific to JSON (or
> specific to XML). Nothing too onerous (no angle brackets)!

I see, gotcha.

> I think this is only done in the image properties.
>
> No, the image properties have been removed in the 2.0 Images API and
> replaced with the standard concept of "tags":
>
> https://docs.google.com/__document/d/1rb7ZVn0Du___5NZqUyQpqUZSmv7Qd66TMHYAtvsow7__LH4/edit#heading=h.__y10rq9pdt27y
> <https://docs.google.com/document/d/1rb7ZVn0Du_5NZqUyQpqUZSmv7Qd66TMHYAtvsow7LH4/edit#heading=h.y10rq9pdt27y>
>
> I was looking at properties in "GET /schemas/image/access" and "GET
> /schemas/access". In general, the Glance 2.0 model is very close to
> being XML-compatible.

OK.

> On the downside: The JSON isn't as JSON-ic as it could be.
>
> In what ways?
>
> When you're designing JSON considering only JSON, you'd probably use {
> key1: value1 } - as you have done. If you're designing generically,
> you'd probably use { key: key1, value: value1 }.

You mean we'd have to do dumb crap because XML doesn't have the native
concept of a list? ;)

> On the upside: You need never worry about XML again
>
> We haven't worried about it at all up until now. And Glance has been
> perfectly fine without it. ;)
>
> Well yes, I agree that our XML support generally is sufficiently buggy
> as to be unusable, hence unused. I don't actually care about XML, I
> just care about having a well-specified API that works.

I wasn't actually talking about Nova. :) Glance doesn't have ANY support
for XML at all. It's been using JSON and only JSON from its inception...
And for the record, we will absolutely NOT be generating JSON from XML
schemas. That is pure lunacy.

> In my opinion, the way to do that is to specify a model, and we have
> JSON and XML representations of that model. If we design the model
> right, JSON & XML both "just work" without any code (and less code =
> less bugs).

I shall refer you to Keystone Legacy to see how well that strategy works
in practice.

> Moreover, when someone introduces another data interchange
> format, we again just write one piece of middleware, one documentation
> generator, and we're done. Doing this by hand for each format is simply
> busywork.

You'd *think* this would work. In practice, however, it really doesn't.
Neither does (good, valid) code generation...

> I don't care whether we specify the model as a Python class, Java class,
> XML schema, JSON schema, or using stick figures with long arms pointing
> at boxes. You can even specify it in JSON if you stick to a "lowest
> common denominator" subset.

In the 2.0 API we *are* specifying it in JSON. JSON Schema, specifically...

> I think the only thing you need to avoid is
> "no changing-at-runtime keys"; I think this makes it compatible
> with XML, Avro, ProtocolBuffers & Thrift.

That is antithetical to having dynamic, but discoverable, schemas. JSON
Schema (and XSD, fwiw) provide things like additionalProperties and
<xsd:any> for just this sort of thing. Making a schema entirely static
is really only useful for generating (bad and soon-to-be-outdated)
client code.

Having dynamic and discoverable schemas enables clients to respond to
backwards-compatible schema changes (like the addition of standard
properties or the addition of "extra-standard" additionalProperties)
without having to recompile a client or change any client code at all...

> I don't think it should be a big deal for the JSON format to avoid
> dynamic keys, given how much you win whenever you have to support a new
> format.

By dynamic keys, are you referring to the additionalProperties? If so, I
disagree.

> Incidentally, I just heard about yet another new format - apparently
> this one is going to be the standard for use with node.js
> - Hypertext-Powered-State-Transfer-Representation.

HPSTR? H(i)PST(e)R? Are you sure that wasn't an April Fool's Joke? :)

Best,
-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


kiall at managedit

Apr 9, 2012, 2:08 PM

Post #6 of 36 (583 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On Mon, Apr 9, 2012 at 7:16 PM, Justin Santa Barbara <justin [at] fathomdb>wrote:
>
> When you're designing JSON considering only JSON, you'd probably use {
> key1: value1 } - as you have done. If you're designing generically, you'd
> probably use { key: key1, value: value1 }.
>

I, literally. die a little inside each and every time I see this kind of
JSON. Once you start producing JSON like this, you may as well just stick
with XML. Parsing it will be easier.

JSON and XML have different strengths and weaknesses, if you only allow the
common ground of functionality to be used you end up with a mediocre at
best implementation of both.

Thanks,
Kiall


justin at fathomdb

Apr 9, 2012, 2:14 PM

Post #7 of 36 (583 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

>
> When you're designing JSON considering only JSON, you'd probably use {
>>
> key1: value1 } - as you have done. If you're designing generically,
>> you'd probably use { key: key1, value: value1 }.
>>
>
> You mean we'd have to do dumb crap because XML doesn't have the native
> concept of a list? ;)


XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports
extensible lists, which is why the syntax is different.



> You'd *think* this would work. In practice, however, it really doesn't.
>> Neither does (good, valid) code generation...
>
>
Of course it works! Every JAX-RS webserver does this. You just can't
start with JSON first and expect everything to magically be OK.

If you think it doesn't work, can you provide an example?

You start with an abstract model, and then check what it looks like in
JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR, etc... If
you start with JSON, then of course it won't work. If we're going to treat
XML as an afterthought, then I'd rather we just didn't support XML at all
(and yes, I absolutely mean that - it is good that Glance is honest that
they don't support XML.)

Even ignoring XML, I can't help but think that not having a strict
delineation between user-provided data and the structure of your document
is a pretty risky idea.


In the 2.0 API we *are* specifying it in JSON. JSON Schema, specifically...
>

Are JSON schemas an April Fool's joke? Once you introduce schemas, you
might as well just go with XML ;-)


> > I think the only thing you need to avoid is
>
>> "no changing-at-runtime keys"; I think this makes it compatible
>> with XML, Avro, ProtocolBuffers & Thrift.
>>
>
> That is antithetical to having dynamic, but discoverable, schemas. JSON
> Schema (and XSD, fwiw) provide things like additionalProperties and
> <xsd:any> for just this sort of thing. Making a schema entirely static is
> really only useful for generating (bad and soon-to-be-outdated) client code.
>

> Having dynamic and discoverable schemas enables clients to respond to
> backwards-compatible schema changes (like the addition of standard
> properties or the addition of "extra-standard" additionalProperties)
> without having to recompile a client or change any client code at all...


I couldn't disagree more: "what does it mean"? There's the implicit
contract underlying the interface; the semantics that underpin the syntax.
e.g. syntax: a glance image id is a string, semantics: the id is unique to
a glance installation and is used to refer to an image in REST calls.

xsd:any allows you to put elements _from another schema_ into your XML
document. That foreign schema defines the semantics of those elements.
It's schemas all the way down, giving semantics to your syntax.

If your additional properties in Glance are properly schematized, then
that's great. But true cross-representation schemas are an open problem, I
believe, so you're really painting yourself into a corner (how do you
support XML output, if you let people upload arbitrary JSON schemas?)


Incidentally, I just heard about yet another new format - apparently
>> this one is going to be the standard for use with node.js
>> - Hypertext-Powered-State-**Transfer-Representation.
>>
>
> HPSTR? H(i)PST(e)R? Are you sure that wasn't an April Fool's Joke? :)
>

Are you sure it wasn't mine? ;-)


jaypipes at gmail

Apr 9, 2012, 2:43 PM

Post #8 of 36 (589 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On 04/09/2012 05:14 PM, Justin Santa Barbara wrote:
> When you're designing JSON considering only JSON, you'd probably
> use {
>
> key1: value1 } - as you have done. If you're designing generically,
> you'd probably use { key: key1, value: value1 }.
>
>
> You mean we'd have to do dumb crap because XML doesn't have the
> native concept of a list? ;)
>
>
> XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports
> extensible lists, which is why the syntax is different.

Extensible lists are pointless. Lists have no attributes other than
their length. I made this point a couple design summits ago... but
whatever :)

> You'd *think* this would work. In practice, however, it really
> doesn't. Neither does (good, valid) code generation...
>
> Of course it works! Every JAX-RS webserver does this. You just can't
> start with JSON first and expect everything to magically be OK.

I'm not aware that I said to "start with JSON first and expect
everything to magically be OK". I believe I put both JSON Schema and
XSDs in the Images API 2.0 proposal?

> If you think it doesn't work, can you provide an example?

I was referring specifically to the Keystone Legacy code that attempted
to generate JSON from XSDs and WADLs. But for the life of me, I can't
now find the code in GitHub that shows what I'm talking about... meh.

> You start with an abstract model, and then check what it looks like in
> JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR, etc...

I'm not opposed to using an abstract model, I was just saying that the
"easy as pie" solution you offered of "just having to create a new
generator, etc" isn't in practice so easy and often leads to
hard-to-debug subtle differences in output format that aren't fixable
just be "editing the abstract model"...

> If you start with JSON, then of course it won't work. If we're going
> to treat XML as an afterthought, then I'd rather we just didn't support
> XML at all (and yes, I absolutely mean that - it is good that Glance is
> honest that they don't support XML.)

Glad we took the high road ;P

> Even ignoring XML, I can't help but think that not having a strict
> delineation between user-provided data and the structure of your
> document is a pretty risky idea.

Not quite sure I follow you here... are you referring to dynamic schemas
here?

> In the 2.0 API we *are* specifying it in JSON. JSON Schema,
> specifically...
>
> Are JSON schemas an April Fool's joke? Once you introduce schemas, you
> might as well just go with XML ;-)

I personally find JSON Schema to be as robust, and infinitely more
readable, as XSD.

> > I think the only thing you need to avoid is
>
> "no changing-at-runtime keys"; I think this makes it compatible
> with XML, Avro, ProtocolBuffers & Thrift.
>
> That is antithetical to having dynamic, but discoverable, schemas.
> JSON Schema (and XSD, fwiw) provide things like additionalProperties
> and <xsd:any> for just this sort of thing. Making a schema entirely
> static is really only useful for generating (bad and
> soon-to-be-outdated) client code.
>
> Having dynamic and discoverable schemas enables clients to respond
> to backwards-compatible schema changes (like the addition of
> standard properties or the addition of "extra-standard"
> additionalProperties) without having to recompile a client or change
> any client code at all...
>
> I couldn't disagree more: "what does it mean"? There's the implicit
> contract underlying the interface; the semantics that underpin the
> syntax. e.g. syntax: a glance image id is a string, semantics: the id
> is unique to a glance installation and is used to refer to an image in
> REST calls.

I think you're muddying WADL and XSD above?

> xsd:any allows you to put elements _from another schema_ into your XML
> document. That foreign schema defines the semantics of those elements.
> It's schemas all the way down, giving semantics to your syntax.

That's exactly what JSON Schema's additionalProperties allows...

> If your additional properties in Glance are properly schematized, then
> that's great. But true cross-representation schemas are an open
> problem, I believe, so you're really painting yourself into a corner
> (how do you support XML output, if you let people upload arbitrary JSON
> schemas?)

They are absolutely properly schematized. They are nothing more than
false (indicating the schema is fixed) or a list of other schemas...

> Incidentally, I just heard about yet another new format - apparently
> this one is going to be the standard for use with node.js
> - Hypertext-Powered-State-__Transfer-Representation.
>
> HPSTR? H(i)PST(e)R? Are you sure that wasn't an April Fool's Joke? :)
>
> Are you sure it wasn't mine? ;-)

:)

-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jorge.williams at rackspace

Apr 9, 2012, 3:42 PM

Post #9 of 36 (587 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

Justin,

>From a JAX-RS / Java persecutive, starting with an XML schema and having that dictate what the JSON will look like -- doesn't just make sense -- it makes life *A LOT* easier. And a lot of services written in Java do just that. Unfortunately, as you pointed out, this approach has the tendency to create very unJSONic JSON -- in that case JSON is just an afterthought..it's a second class citizen -- essentially it's XML with curly braces...and that doesn't jive well with dynamic languages.

You can go the other route and start from JSON to create XML -- and some well intended folks do this, but you end up in the other direction...essentially you have <JSON/>, which is essentially JSON in XML form. That kind of XML sucks and it's pretty useless if you work with static languages like Java and .Net -- where having a static (but extensible) schema is really useful.

That's the world we live in today...static languages work really well with XML and dynamic languages work really well with JSON. In my opinion, you have to account for both and treat both as first class citizens. I'd vote for having an abstract model and treat the XML and JSON as different renderings. Doing so is not easy as pie -- but it's not rocket science either.

I am totally with you on a couple of points though:

1. We do need to differentiate between user data and core attributes. Jay and I have had debates about this before.
2. Having entirely dynamic discoverable schema, seems nice, but it has the potential to introduce a lot of complexity. In particular, i'm not exactly sure how to process the schema and keep the goodness that comes from using a statically typed language. And, as you point out, it's not clear how one would support multiple media types given that approach.

How about we discuss this further at the summit :-)

-jOrGe W.


On Apr 9, 2012, at 4:14 PM, Justin Santa Barbara wrote:

When you're designing JSON considering only JSON, you'd probably use {
key1: value1 } - as you have done. If you're designing generically,
you'd probably use { key: key1, value: value1 }.

You mean we'd have to do dumb crap because XML doesn't have the native concept of a list? ;)

XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports extensible lists, which is why the syntax is different.


You'd *think* this would work. In practice, however, it really doesn't. Neither does (good, valid) code generation...

Of course it works! Every JAX-RS webserver does this. You just can't start with JSON first and expect everything to magically be OK.

If you think it doesn't work, can you provide an example?

You start with an abstract model, and then check what it looks like in JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR, etc... If you start with JSON, then of course it won't work. If we're going to treat XML as an afterthought, then I'd rather we just didn't support XML at all (and yes, I absolutely mean that - it is good that Glance is honest that they don't support XML.)

Even ignoring XML, I can't help but think that not having a strict delineation between user-provided data and the structure of your document is a pretty risky idea.


In the 2.0 API we *are* specifying it in JSON. JSON Schema, specifically...

Are JSON schemas an April Fool's joke? Once you introduce schemas, you might as well just go with XML ;-)

> I think the only thing you need to avoid is
"no changing-at-runtime keys"; I think this makes it compatible
with XML, Avro, ProtocolBuffers & Thrift.

That is antithetical to having dynamic, but discoverable, schemas. JSON Schema (and XSD, fwiw) provide things like additionalProperties and <xsd:any> for just this sort of thing. Making a schema entirely static is really only useful for generating (bad and soon-to-be-outdated) client code.

Having dynamic and discoverable schemas enables clients to respond to backwards-compatible schema changes (like the addition of standard properties or the addition of "extra-standard" additionalProperties) without having to recompile a client or change any client code at all...

I couldn't disagree more: "what does it mean"? There's the implicit contract underlying the interface; the semantics that underpin the syntax. e.g. syntax: a glance image id is a string, semantics: the id is unique to a glance installation and is used to refer to an image in REST calls.

xsd:any allows you to put elements _from another schema_ into your XML document. That foreign schema defines the semantics of those elements. It's schemas all the way down, giving semantics to your syntax.

If your additional properties in Glance are properly schematized, then that's great. But true cross-representation schemas are an open problem, I believe, so you're really painting yourself into a corner (how do you support XML output, if you let people upload arbitrary JSON schemas?)


Incidentally, I just heard about yet another new format - apparently
this one is going to be the standard for use with node.js
- Hypertext-Powered-State-Transfer-Representation.

HPSTR? H(i)PST(e)R? Are you sure that wasn't an April Fool's Joke? :)

Are you sure it wasn't mine? ;-)

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


justin at fathomdb

Apr 9, 2012, 4:03 PM

Post #10 of 36 (582 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

>
> How about we discuss this further at the summit :-)
>

I think that's a sensible proposal. We're not likely to reach a good
conclusion here. I think my viewpoint is that even json-dressed-as-xml is
fine; no end-user gives two hoots what our JSON/XML/HPSTR looks like. I'd
wager most users of the EC2 API have never even seen the data
representation!


justin at fathomdb

Apr 9, 2012, 4:05 PM

Post #11 of 36 (586 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

>
> Extensible lists are pointless. Lists have no attributes other than their
> length. I made this point a couple design summits ago... but whatever :)


Looks like the Sapir-Whorf hypothesis might be true after all ;-)

Let's dust off the pugil-sticks for the design summit..


jorge.williams at rackspace

Apr 9, 2012, 4:07 PM

Post #12 of 36 (582 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On Apr 9, 2012, at 6:03 PM, Justin Santa Barbara wrote:

How about we discuss this further at the summit :-)

I think that's a sensible proposal. We're not likely to reach a good conclusion here. I think my viewpoint is that even json-dressed-as-xml is fine; no end-user gives two hoots what our JSON/XML/HPSTR looks like. I'd wager most users of the EC2 API have never even seen the data representation!


I take it you didn't attend the glorious JSON debate of a couple of summits ago :-)

I'm up for round two,

-jOrGe W.


jaypipes at gmail

Apr 9, 2012, 4:11 PM

Post #13 of 36 (582 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On 04/09/2012 07:07 PM, Jorge Williams wrote:
>
> On Apr 9, 2012, at 6:03 PM, Justin Santa Barbara wrote:
>
>> How about we discuss this further at the summit :-)
>>
>>
>> I think that's a sensible proposal. We're not likely to reach a good
>> conclusion here. I think my viewpoint is that even json-dressed-as-xml
>> is fine; no end-user gives two hoots what our JSON/XML/HPSTR looks
>> like. I'd wager most users of the EC2 API have never even seen the
>> data representation!
>
>
> I take it you didn't attend the glorious JSON debate of a couple of
> summits ago :-)

Glorious it was indeed.

> I'm up for round two,

Only with beer. :) I still owe you a couple I think!

> -jOrGe W.
>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


luis at woorea

Apr 9, 2012, 5:10 PM

Post #14 of 36 (589 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

Hi,

I have been fighting with these issues.

Here is the proposed solution i am currently using on OpenStack Java SDK.

*Every representation should implement a common interface*

The jaxb annotations for marshalling and unmarshalling XML reside on xml
implementation class
The gson annotations for serialization and deserialization of JSON reside
on json implementation class

This approach is unobstrussive with XML/JSON representations

I think 95% of the API works well mixing jaxb/gson annotations on the same
implementation classes.

This has another advantage, you can switch from one to another service
provider sharing the same model,
that's the interfaces model but the implentation.

Does it make sense for you?

Luis

On Tue, Apr 10, 2012 at 1:11 AM, Jay Pipes <jaypipes [at] gmail> wrote:

> On 04/09/2012 07:07 PM, Jorge Williams wrote:
>
>>
>> On Apr 9, 2012, at 6:03 PM, Justin Santa Barbara wrote:
>>
>> How about we discuss this further at the summit :-)
>>>
>>>
>>> I think that's a sensible proposal. We're not likely to reach a good
>>> conclusion here. I think my viewpoint is that even json-dressed-as-xml
>>> is fine; no end-user gives two hoots what our JSON/XML/HPSTR looks
>>> like. I'd wager most users of the EC2 API have never even seen the
>>> data representation!
>>>
>>
>>
>> I take it you didn't attend the glorious JSON debate of a couple of
>> summits ago :-)
>>
>
> Glorious it was indeed.
>
>
> I'm up for round two,
>>
>
> Only with beer. :) I still owe you a couple I think!
>
> -jOrGe W.
>>
>>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>



--
-------------------------------------------
Luis Alberto Gervaso Martin
Woorea Solutions, S.L
CEO & CTO
mobile: (+34) 627983344
luis@ <luis.gervaso [at] gmail>woorea.es


mordred at inaugust

Apr 9, 2012, 9:05 PM

Post #15 of 36 (579 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On 04/09/2012 04:11 PM, Jay Pipes wrote:
> On 04/09/2012 07:07 PM, Jorge Williams wrote:
>>
>> On Apr 9, 2012, at 6:03 PM, Justin Santa Barbara wrote:
>>
>>> How about we discuss this further at the summit :-)
>>>
>>>
>>> I think that's a sensible proposal. We're not likely to reach a good
>>> conclusion here. I think my viewpoint is that even json-dressed-as-xml
>>> is fine; no end-user gives two hoots what our JSON/XML/HPSTR looks
>>> like. I'd wager most users of the EC2 API have never even seen the
>>> data representation!
>>
>>
>> I take it you didn't attend the glorious JSON debate of a couple of
>> summits ago :-)
>
> Glorious it was indeed.
>
>> I'm up for round two,
>
> Only with beer. :) I still owe you a couple I think!

I refuse to not be there for that. Please make sure I'm in the room.
With a video camera. And a bottle of whiskey.

Monty

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


thierry at openstack

Apr 10, 2012, 2:26 AM

Post #16 of 36 (584 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

Jay Pipes wrote:
>> I take it you didn't attend the glorious JSON debate of a couple of
>> summits ago :-)
>
> Glorious it was indeed.

I think the key quote was something like:
"Please don't bastardize my JSON with your XML crap"

--
Thierry Carrez (ttx)
Release Manager, OpenStack

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


doug.hellmann at dreamhost

Apr 10, 2012, 3:51 AM

Post #17 of 36 (585 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On Mon, Apr 9, 2012 at 5:14 PM, Justin Santa Barbara <justin [at] fathomdb>wrote:

> When you're designing JSON considering only JSON, you'd probably use {
>>>
>> key1: value1 } - as you have done. If you're designing generically,
>>> you'd probably use { key: key1, value: value1 }.
>>>
>>
>> You mean we'd have to do dumb crap because XML doesn't have the native
>> concept of a list? ;)
>
>
> XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports
> extensible lists, which is why the syntax is different.
>
>
>
>> You'd *think* this would work. In practice, however, it really doesn't.
>>> Neither does (good, valid) code generation...
>>
>>
> Of course it works! Every JAX-RS webserver does this. You just can't
> start with JSON first and expect everything to magically be OK.
>
> If you think it doesn't work, can you provide an example?
>
> You start with an abstract model, and then check what it looks like in
> JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR, etc... If
> you start with JSON, then of course it won't work. If we're going to treat
> XML as an afterthought, then I'd rather we just didn't support XML at all
> (and yes, I absolutely mean that - it is good that Glance is honest that
> they don't support XML.)
>

Kevin Dangoor and Christophe de Vienne have done some work on abstracting
the view of data inside and outside of the API with TGWebServices [1] (a
TurboGears add-on) and the more recent "Web Services Made Easy" [2], which
is framework agnostic. I have used TGWS in the past to create an API using
SOAP and JSON (it also supports generic XML, but we didn't need that). I
found that it worked well for our purposes at the time.

[1] http://code.google.com/p/tgws/
[2] http://packages.python.org/WSME/


eglynn at redhat

Apr 10, 2012, 6:21 AM

Post #18 of 36 (583 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

> APPENDIX B: Outstanding issues
> ...
> 2) How do we fit the existing 'copy_from' functionality in?


Is the v2 API retaining some equivalent of the existing
x-image-meta-location header, to allow an externally-stored
image be registered with glance?

e.g. via an image field specified on create or update:

POST /images HTTP/1.1
{"external-location": "s3://access:secret [at] s3/image", ...}

or:

PUT /images/<IMAGE_ID> HTTP/1.1
{"external-location": "s3://access:secret [at] s3/image", ...}

If so, the most straight-forward approach for copy-from would be to
follow a similar pattern with an image field such as:

POST /images HTTP/1.1
{"copy-from": "s3://access:secret [at] s3/image", ...}

... etc.

Cheers,
Eoghan


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jaypipes at gmail

Apr 10, 2012, 7:47 AM

Post #19 of 36 (585 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

FWIW, Nova already has this kind of abstraction, with views and
serializers... I wasn't planning on reinventing any wheels with the 2.0
Images API implementation; just using what Nova had (and hopefully
moving it to openstack-common before bringing the code into Glance).

Best,
-jay

On 04/10/2012 06:51 AM, Doug Hellmann wrote:
>
>
> On Mon, Apr 9, 2012 at 5:14 PM, Justin Santa Barbara
> <justin [at] fathomdb <mailto:justin [at] fathomdb>> wrote:
>
> When you're designing JSON considering only JSON, you'd
> probably use {
>
> key1: value1 } - as you have done. If you're designing
> generically,
> you'd probably use { key: key1, value: value1 }.
>
>
> You mean we'd have to do dumb crap because XML doesn't have the
> native concept of a list? ;)
>
>
> XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports
> extensible lists, which is why the syntax is different.
>
> You'd *think* this would work. In practice, however, it
> really doesn't. Neither does (good, valid) code generation...
>
>
> Of course it works! Every JAX-RS webserver does this. You just
> can't start with JSON first and expect everything to magically be OK.
>
> If you think it doesn't work, can you provide an example?
>
> You start with an abstract model, and then check what it looks like
> in JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR,
> etc... If you start with JSON, then of course it won't work. If
> we're going to treat XML as an afterthought, then I'd rather we just
> didn't support XML at all (and yes, I absolutely mean that - it is
> good that Glance is honest that they don't support XML.)
>
>
> Kevin Dangoor and Christophe de Vienne have done some work on
> abstracting the view of data inside and outside of the API with
> TGWebServices [1] (a TurboGears add-on) and the more recent "Web
> Services Made Easy" [2], which is framework agnostic. I have used TGWS
> in the past to create an API using SOAP and JSON (it also supports
> generic XML, but we didn't need that). I found that it worked well for
> our purposes at the time.
>
> [1] http://code.google.com/p/tgws/
> [2] http://packages.python.org/WSME/

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


jaypipes at gmail

Apr 10, 2012, 7:48 AM

Post #20 of 36 (581 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

I'll let Waldon answer this, but I know that it is marked as "to be
determined" currently in his notes on the API...

On 04/10/2012 09:21 AM, Eoghan Glynn wrote:
>> APPENDIX B: Outstanding issues
>> ...
>> 2) How do we fit the existing 'copy_from' functionality in?
>
>
> Is the v2 API retaining some equivalent of the existing
> x-image-meta-location header, to allow an externally-stored
> image be registered with glance?
>
> e.g. via an image field specified on create or update:
>
> POST /images HTTP/1.1
> {"external-location": "s3://access:secret [at] s3/image", ...}
>
> or:
>
> PUT /images/<IMAGE_ID> HTTP/1.1
> {"external-location": "s3://access:secret [at] s3/image", ...}
>
> If so, the most straight-forward approach for copy-from would be to
> follow a similar pattern with an image field such as:
>
> POST /images HTTP/1.1
> {"copy-from": "s3://access:secret [at] s3/image", ...}
>
> ... etc.
>
> Cheers,
> Eoghan
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


doug.hellmann at dreamhost

Apr 10, 2012, 8:01 AM

Post #21 of 36 (581 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

+1 on reusing existing code and the move

On Tue, Apr 10, 2012 at 10:47 AM, Jay Pipes <jaypipes [at] gmail> wrote:

> FWIW, Nova already has this kind of abstraction, with views and
> serializers... I wasn't planning on reinventing any wheels with the 2.0
> Images API implementation; just using what Nova had (and hopefully moving
> it to openstack-common before bringing the code into Glance).
>
> Best,
> -jay
>
>
> On 04/10/2012 06:51 AM, Doug Hellmann wrote:
>
>>
>>
>> On Mon, Apr 9, 2012 at 5:14 PM, Justin Santa Barbara
>> <justin [at] fathomdb <mailto:justin [at] fathomdb>> wrote:
>>
>> When you're designing JSON considering only JSON, you'd
>> probably use {
>>
>> key1: value1 } - as you have done. If you're designing
>> generically,
>> you'd probably use { key: key1, value: value1 }.
>>
>>
>> You mean we'd have to do dumb crap because XML doesn't have the
>> native concept of a list? ;)
>>
>>
>> XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports
>> extensible lists, which is why the syntax is different.
>>
>> You'd *think* this would work. In practice, however, it
>> really doesn't. Neither does (good, valid) code generation...
>>
>>
>> Of course it works! Every JAX-RS webserver does this. You just
>> can't start with JSON first and expect everything to magically be OK.
>>
>> If you think it doesn't work, can you provide an example?
>>
>> You start with an abstract model, and then check what it looks like
>> in JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR,
>> etc... If you start with JSON, then of course it won't work. If
>> we're going to treat XML as an afterthought, then I'd rather we just
>> didn't support XML at all (and yes, I absolutely mean that - it is
>> good that Glance is honest that they don't support XML.)
>>
>>
>> Kevin Dangoor and Christophe de Vienne have done some work on
>> abstracting the view of data inside and outside of the API with
>> TGWebServices [1] (a TurboGears add-on) and the more recent "Web
>> Services Made Easy" [2], which is framework agnostic. I have used TGWS
>> in the past to create an API using SOAP and JSON (it also supports
>> generic XML, but we didn't need that). I found that it worked well for
>> our purposes at the time.
>>
>> [1] http://code.google.com/p/tgws/
>> [2] http://packages.python.org/**WSME/ <http://packages.python.org/WSME/>
>>
>


justin at fathomdb

Apr 10, 2012, 10:05 AM

Post #22 of 36 (582 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

I'd really rather we supported one format, if they're not going to be equal
citizens (i.e. both generated from a common model).

I wasted a lot of time with nova's XML "support"; I'm sure the Java binding
was the only project ever to try to use it; we'd have been able to proceed
much faster if we'd just stuck with JSON - we now have a horrible hybrid,
where JSON is used for some calls because the XML has/had bugs.

You prefer working from a JSON model; you're doing the work; so JSON it is.
Please spend the time you would have spent on XML support on making glance
2.0 the best image server there is.

Justin


On Tue, Apr 10, 2012 at 7:47 AM, Jay Pipes <jaypipes [at] gmail> wrote:

> FWIW, Nova already has this kind of abstraction, with views and
> serializers... I wasn't planning on reinventing any wheels with the 2.0
> Images API implementation; just using what Nova had (and hopefully moving
> it to openstack-common before bringing the code into Glance).
>
> Best,
> -jay
>
>
> On 04/10/2012 06:51 AM, Doug Hellmann wrote:
>
>>
>>
>> On Mon, Apr 9, 2012 at 5:14 PM, Justin Santa Barbara
>> <justin [at] fathomdb <mailto:justin [at] fathomdb>> wrote:
>>
>> When you're designing JSON considering only JSON, you'd
>> probably use {
>>
>> key1: value1 } - as you have done. If you're designing
>> generically,
>> you'd probably use { key: key1, value: value1 }.
>>
>>
>> You mean we'd have to do dumb crap because XML doesn't have the
>> native concept of a list? ;)
>>
>>
>> XML has lists, as does Avro, ProtocolBuffers & Thrift. XML supports
>> extensible lists, which is why the syntax is different.
>>
>> You'd *think* this would work. In practice, however, it
>> really doesn't. Neither does (good, valid) code generation...
>>
>>
>> Of course it works! Every JAX-RS webserver does this. You just
>> can't start with JSON first and expect everything to magically be OK.
>>
>> If you think it doesn't work, can you provide an example?
>>
>> You start with an abstract model, and then check what it looks like
>> in JSON, in XML, in ProtocolBuffers, in Avro, in Thrift, in HPSTR,
>> etc... If you start with JSON, then of course it won't work. If
>> we're going to treat XML as an afterthought, then I'd rather we just
>> didn't support XML at all (and yes, I absolutely mean that - it is
>> good that Glance is honest that they don't support XML.)
>>
>>
>> Kevin Dangoor and Christophe de Vienne have done some work on
>> abstracting the view of data inside and outside of the API with
>> TGWebServices [1] (a TurboGears add-on) and the more recent "Web
>> Services Made Easy" [2], which is framework agnostic. I have used TGWS
>> in the past to create an API using SOAP and JSON (it also supports
>> generic XML, but we didn't need that). I found that it worked well for
>> our purposes at the time.
>>
>> [1] http://code.google.com/p/tgws/
>> [2] http://packages.python.org/**WSME/ <http://packages.python.org/WSME/>
>>
>


brian.waldon at rackspace

Apr 10, 2012, 10:11 AM

Post #23 of 36 (588 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

The ability to add an external image was dropped when I removed the concept of image locations. I wanted to rethink how locations worked and didn't realize how much I was actually removing! 'copy_from' just hasn't been fit into the spec yet. I want both of the features to be exposed through the API, but I want it done cleanly. This is something that we will figure out as we implement the API, as there isn't a perfect solution right now. We definitely won't be releasing the v2 API without these features!

Waldon


On Apr 10, 2012, at 6:21 AM, Eoghan Glynn wrote:

>
>
>> APPENDIX B: Outstanding issues
>> ...
>> 2) How do we fit the existing 'copy_from' functionality in?
>
>
> Is the v2 API retaining some equivalent of the existing
> x-image-meta-location header, to allow an externally-stored
> image be registered with glance?
>
> e.g. via an image field specified on create or update:
>
> POST /images HTTP/1.1
> {"external-location": "s3://access:secret [at] s3/image", ...}
>
> or:
>
> PUT /images/<IMAGE_ID> HTTP/1.1
> {"external-location": "s3://access:secret [at] s3/image", ...}
>
> If so, the most straight-forward approach for copy-from would be to
> follow a similar pattern with an image field such as:
>
> POST /images HTTP/1.1
> {"copy-from": "s3://access:secret [at] s3/image", ...}
>
> ... etc.
>
> Cheers,
> Eoghan
>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


kevin.mitchell at rackspace

Apr 10, 2012, 10:22 AM

Post #24 of 36 (585 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

On Tue, 2012-04-10 at 10:05 -0700, Justin Santa Barbara wrote:
> I wasted a lot of time with nova's XML "support"; I'm sure the Java
> binding was the only project ever to try to use it; we'd have been
> able to proceed much faster if we'd just stuck with JSON - we now have
> a horrible hybrid, where JSON is used for some calls because the XML
> has/had bugs.

Well, hopefully the XML support has been a little better since my
templates stuff went in. Either way, though, if I had the choice, I'd
rip all of nova's XML support out tomorrow…

--
Kevin L. Mitchell <kevin.mitchell [at] rackspace>


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


justin at fathomdb

Apr 10, 2012, 10:57 AM

Post #25 of 36 (582 views)
Permalink
Re: Image API v2 Draft 4 [In reply to]

It definitely has improved - thank you for all your work; I didn't mean to
put down anyone's work here. It's simply a Sisyphean task.

Either way, though, if I had the choice, I'd rip all of nova's XML support
> out tomorrow…
>

As a strong supporter of XML, who thinks JSON is for kids that haven't
figured out that the Easter bunny isn't real yet :-)... +1

Justin

First page Previous page 1 2 Next page Last page  View All OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.