Mailing List Archive: OpenStack: Dev

OVF vs. bare container formats for qcow2 images

Index | Next | Previous | View Threaded

lorin at nimbisservices

Mar 28, 2012, 1:41 PM

Post #1 of 11 (492 views)
Permalink

OVF vs. bare container formats for qcow2 images

All:

Given that I have a qcow2 image from somewhere (e.g., downloaded it from a uec-images.ubuntu.com, created one from a raw image using qemu-img) that i want to add to glance:

1. How can I tell whether it's an "ovf" or "bare" container format?
2. Why does it matter?

Whenever I add a qcow2 image to glance, I always choose "ovf", even though it's probably "bare", because I saw an example somewhere, and it just works, so I keep doing it. But I don't know how to inspect a binary file to determine what its container is (if "file image.qcow2" says it's a QEMU QCOW2 Image (v2), does that mean it's "bare"?). In particular, why does the user need to specify this information?

Also, are there any Linux command-line tools for inspecting/manipulating OVF containers?

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com

smoser at ubuntu

Mar 29, 2012, 9:00 AM

Post #2 of 11 (493 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On Wed, 28 Mar 2012, Lorin Hochstein wrote:

> All:
>
> Given that I have a qcow2 image from somewhere (e.g., downloaded it from a uec-images.ubuntu.com, created one from a raw image using qemu-img) that i want to add to glance:
>
> 1. How can I tell whether it's an "ovf" or "bare" container format?
> 2. Why does it matter?

I dont know either, but I do know what you want, taken from stack.sh in
devstack, which should probably be updated to not use '-A'
glance add -A "$TOKEN" \
name="${IMAGE_NAME%.img}" is_public=true
container_format=ami disk_format=ami <${IMAGE}

You can also add "ramdisk_id=...." and "kernel_id=...." tags in the upload
if you want that.

Note, you can do this with the Ubuntu cloud-images, and that is the best
way to put images for openstack into glance.
$ url=https://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
$ wget "$url" -O "${url##*/}"
$ glance add name=${url##*/} is_public=true container_format=ami \
disk_format=ami < "${url##*/}"

> Whenever I add a qcow2 image to glance, I always choose "ovf", even
> though it's probably "bare", because I saw an example somewhere, and it
> just works, so I keep doing it. But I don't know how to inspect a binary
> file to determine what its container is (if "file image.qcow2" says it's
> a QEMU QCOW2 Image (v2), does that mean it's "bare"?). In particular,
> why does the user need to specify this information?
>
> Also, are there any Linux command-line tools for inspecting/manipulating OVF containers?

Theres open-ovf, which has my name on it, but is really abandoned.

from my perspective, the OVF support in glance/openstack is really to be
ignored.

berrange at redhat

Mar 29, 2012, 9:40 AM

Post #3 of 11 (482 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
> All:
>
> Given that I have a qcow2 image from somewhere (e.g., downloaded
> it from a uec-images.ubuntu.com, created one from a raw image using
> qemu-img) that i want to add to glance:
>
> 1. How can I tell whether it's an "ovf" or "bare" container format?

You are mixing up terminology here. Disk image formats are things like
raw, qcow2, vmdk, etc.

OVF refers to the format of a metadata file provided alongside the
disk image, which describes various requirements for running the
image.

The two are not tied together at all, merely complementary to
each other.

> 2. Why does it matter?

OVF provides metadata that is useful to virt/cloud mgmt applications
when deploying a prebuilt disk image. I've no idea what use OpenStack
makes of the OVF metadata though.

> Whenever I add a qcow2 image to glance, I always choose "ovf",
> even though it's probably "bare", because I saw an example
> somewhere, and it just works, so I keep doing it. But I don't
> know how to inspect a binary file to determine what its container
> is (if "file image.qcow2" says it's a QEMU QCOW2 Image (v2), does
> that mean it's "bare"?). In particular, why does the user need to
> specify this information?

If you simply have a single someimage.qcow2 file, then you simply
have a disk image. Thus there is no OVF metadata involved at all.

eg, this is the (qcow2) disk image:

http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img

While this is an OVF metadata file that optionally accompanies the disk image

http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64.ovf

Sometimes, people may create a zip/tar.gz file that contains both the
disk image and OVF file in one convenient download.

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

smoser at ubuntu

Mar 29, 2012, 12:45 PM

Post #4 of 11 (484 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On Thu, 29 Mar 2012, Daniel P. Berrange wrote:

> On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
> > All:
> >
> > Given that I have a qcow2 image from somewhere (e.g., downloaded
> > it from a uec-images.ubuntu.com, created one from a raw image using
> > qemu-img) that i want to add to glance:
> >
> > 1. How can I tell whether it's an "ovf" or "bare" container format?
>
> You are mixing up terminology here. Disk image formats are things like
> raw, qcow2, vmdk, etc.
>
> OVF refers to the format of a metadata file provided alongside the
> disk image, which describes various requirements for running the
> image.
>
> The two are not tied together at all, merely complementary to
> each other.

Well, the implementation in glance is not really aligned with reality.
There was discussion on the list a while ago.
http://www.mail-archive.com/openstack [at] lists/msg05803.html
that has more info.

lorin at nimbisservices

Apr 1, 2012, 8:15 AM

Post #5 of 11 (480 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On Mar 29, 2012, at 12:40 PM, Daniel P. Berrange wrote:

> On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
>> All:
>>
>> Given that I have a qcow2 image from somewhere (e.g., downloaded
>> it from a uec-images.ubuntu.com, created one from a raw image using
>> qemu-img) that i want to add to glance:
>>
>> 1. How can I tell whether it's an "ovf" or "bare" container format?
>
> You are mixing up terminology here. Disk image formats are things like
> raw, qcow2, vmdk, etc.
>
> OVF refers to the format of a metadata file provided alongside the
> disk image, which describes various requirements for running the
> image.
>
> The two are not tied together at all, merely complementary to
> each other.
>

Thanks, that clears things up. I was confused by this language, which sounded to me like the metadata was embedded in the disk image file:

http://glance.openstack.org/formats.html

"The container format refers to whether the virtual machine image is in a file format that also contains metadata about the actual virtual machine."

In addition, the docs have examples like this, which clearly aren't meaningful:
http://glance.openstack.org/glance.html#important-information-about-uploading-images

$> glance add name="My Image" is_public=true \
container_format=ovf disk_format=raw < /tmp/images/myimage.iso

I'll propose a change to the docs for that.

>
>> Whenever I add a qcow2 image to glance, I always choose "ovf",
>> even though it's probably "bare", because I saw an example
>> somewhere, and it just works, so I keep doing it. But I don't
>> know how to inspect a binary file to determine what its container
>> is (if "file image.qcow2" says it's a QEMU QCOW2 Image (v2), does
>> that mean it's "bare"?). In particular, why does the user need to
>> specify this information?
>
> If you simply have a single someimage.qcow2 file, then you simply
> have a disk image. Thus there is no OVF metadata involved at all.
>
> eg, this is the (qcow2) disk image:
>
> http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
>
> While this is an OVF metadata file that optionally accompanies the disk image
>
> http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64.ovf
>

Gotcha.

It's not clear to me how you would specify the OVF metadata file when adding an image file to glance.

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com

lorin at nimbisservices

Apr 1, 2012, 8:22 AM

Post #6 of 11 (486 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On Mar 29, 2012, at 12:00 PM, Scott Moser wrote:

> On Wed, 28 Mar 2012, Lorin Hochstein wrote:
>
>> All:
>>
>> Given that I have a qcow2 image from somewhere (e.g., downloaded it from a uec-images.ubuntu.com, created one from a raw image using qemu-img) that i want to add to glance:
>>
>> 1. How can I tell whether it's an "ovf" or "bare" container format?
>> 2. Why does it matter?
>
> I dont know either, but I do know what you want, taken from stack.sh in
> devstack, which should probably be updated to not use '-A'
> glance add -A "$TOKEN" \
> name="${IMAGE_NAME%.img}" is_public=true
> container_format=ami disk_format=ami <${IMAGE}
>
> You can also add "ramdisk_id=...." and "kernel_id=...." tags in the upload
> if you want that.
>
> Note, you can do this with the Ubuntu cloud-images, and that is the best
> way to put images for openstack into glance.
> $ url=https://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
> $ wget "$url" -O "${url##*/}"
> $ glance add name=${url##*/} is_public=true container_format=ami \
> disk_format=ami < "${url##*/}"
>

Isn't that a qcow2 image? Why does devstack use "container_format=ami disk_format=ami" and not "container_format=bare disk_format=qcow2"?

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com

ayoung at redhat

Jun 29, 2012, 6:53 PM

Post #7 of 11 (421 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On 04/01/2012 11:15 AM, Lorin Hochstein wrote:
>
>
> On Mar 29, 2012, at 12:40 PM, Daniel P. Berrange wrote:
>
>> On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
>>> All:
>>>
>>> Given that I have a qcow2 image from somewhere (e.g., downloaded
>>> it from a uec-images.ubuntu.com <http://uec-images.ubuntu.com>,
>>> created one from a raw image using
>>> qemu-img) that i want to add to glance:
>>>
>>> 1. How can I tell whether it's an "ovf" or "bare" container format?
>>
>> You are mixing up terminology here. Disk image formats are things like
>> raw, qcow2, vmdk, etc.
>>
>> OVF refers to the format of a metadata file provided alongside the
>> disk image, which describes various requirements for running the
>> image.
>>
>> The two are not tied together at all, merely complementary to
>> each other.
>>
>
> Thanks, that clears things up. I was confused by this language, which
> sounded to me like the metadata was embedded in the disk image file:
>
> http://glance.openstack.org/formats.html
>
> "The container format refers to whether the virtual machine image is
> in a file format that also contains metadata about the actual virtual
> machine."
>
> In addition, the docs have examples like this, which clearly aren't
> meaningful:
> http://glance.openstack.org/glance.html#important-information-about-uploading-images

Just to add to the confusion the OVF can contain both the metadata file
and the disk image file in a single archived file.

"An OVF package consists of several files, placed in one directory. A
one-file alternative is the OVA package, which is a TAR file with the
OVF directory inside."

http://en.wikipedia.org/wiki/Open_Virtualization_Format#Technical_description

I think that what you are reading above refers to the single file
alternative.
>
>
> $> glance add name="My Image" is_public=true \
> container_format=ovf disk_format=raw < /tmp/images/myimage.iso
>
> I'll propose a change to the docs for that.
>
>>
>>> Whenever I add a qcow2 image to glance, I always choose "ovf",
>>> even though it's probably "bare", because I saw an example
>>> somewhere, and it just works, so I keep doing it. But I don't
>>> know how to inspect a binary file to determine what its container
>>> is (if "file image.qcow2" says it's a QEMU QCOW2 Image (v2), does
>>> that mean it's "bare"?). In particular, why does the user need to
>>> specify this information?
>>
>> If you simply have a single someimage.qcow2 file, then you simply
>> have a disk image. Thus there is no OVF metadata involved at all.
>>
>> eg, this is the (qcow2) disk image:
>>
>> http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
>>
>> While this is an OVF metadata file that optionally accompanies the
>> disk image
>>
>> http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64.ovf
>>
>
> Gotcha.
>
>
> It's not clear to me how you would specify the OVF metadata file when
> adding an image file to glance.
>
>
> Take care,
>
> Lorin
> --
> Lorin Hochstein
> Lead Architect - Cloud Services
> Nimbis Services, Inc.
> www.nimbisservices.com <https://www.nimbisservices.com/>
>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

lorin at nimbisservices

Jul 3, 2012, 7:07 PM

Post #8 of 11 (419 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On Jun 29, 2012, at 9:53 PM, Adam Young wrote:

> On 04/01/2012 11:15 AM, Lorin Hochstein wrote:
>>
>>
>> On Mar 29, 2012, at 12:40 PM, Daniel P. Berrange wrote:
>>
>>> On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
>>>> All:
>>>>
>>>> Given that I have a qcow2 image from somewhere (e.g., downloaded
>>>> it from a uec-images.ubuntu.com, created one from a raw image using
>>>> qemu-img) that i want to add to glance:
>>>>
>>>> 1. How can I tell whether it's an "ovf" or "bare" container format?
>>>
>>> You are mixing up terminology here. Disk image formats are things like
>>> raw, qcow2, vmdk, etc.
>>>
>>> OVF refers to the format of a metadata file provided alongside the
>>> disk image, which describes various requirements for running the
>>> image.
>>>
>>> The two are not tied together at all, merely complementary to
>>> each other.
>>>
>>
>> Thanks, that clears things up. I was confused by this language, which sounded to me like the metadata was embedded in the disk image file:
>>
>> http://glance.openstack.org/formats.html
>>
>> "The container format refers to whether the virtual machine image is in a file format that also contains metadata about the actual virtual machine."
>>
>> In addition, the docs have examples like this, which clearly aren't meaningful:
>> http://glance.openstack.org/glance.html#important-information-about-uploading-images
>
> Just to add to the confusion the OVF can contain both the metadata file and the disk image file in a single archived file.
>
> "An OVF package consists of several files, placed in one directory. A one-file alternative is the OVA package, which is a TAR file with the OVF directory inside."
>
> http://en.wikipedia.org/wiki/Open_Virtualization_Format#Technical_description

Does anybody know if OpenStack (nova+glance) currently supports OVA packages?

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com

jaypipes at gmail

Jul 5, 2012, 7:58 AM

Post #9 of 11 (412 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

On 07/03/2012 10:07 PM, Lorin Hochstein wrote:
> Does anybody know if OpenStack (nova+glance) currently supports OVA
> packages?

No, not really. Glance will store pretty much anything you throw at it,
but the virt driver(s) in Nova will need to know how to handle what gets
returned from Glance. It is this piece that is missing from Nova (but
really shouldn't be that difficult to add).

Best,
-jay

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

lorin at nimbisservices

Jul 8, 2012, 6:01 PM

Post #10 of 11 (407 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

Paul:

I assume you're using XenServer? When I grepped through the code, there appears to be a XenServer plugin for glance for supporting tar balls that could be OVAs, but don't necessarily contain the OVF file (?).

Here's the code:
https://github.com/openstack/nova/blob/stable/essex/plugins/xenserver/xenapi/etc/xapi.d/plugins/glance#L365

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com

On Jul 5, 2012, at 11:22 AM, Paul Voccio wrote:

> Lorin,
>
> We've been using OVA packages since the beginning. I believe there is a flag in glance for this.
>
>
> Thanks,
> ~pvo
>
> Paul Voccio
> paul.voccio [at] rackspace
> 770-335-2143 (c)
> pvo on #openstack
> ಠ_ಠ
>
> On Jul 3, 2012, at 9:07 PM, Lorin Hochstein wrote:
>
>>
>> On Jun 29, 2012, at 9:53 PM, Adam Young wrote:
>>
>>> On 04/01/2012 11:15 AM, Lorin Hochstein wrote:
>>>>
>>>>
>>>> On Mar 29, 2012, at 12:40 PM, Daniel P. Berrange wrote:
>>>>
>>>>> On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
>>>>>> All:
>>>>>>
>>>>>> Given that I have a qcow2 image from somewhere (e.g., downloaded
>>>>>> it from a uec-images.ubuntu.com, created one from a raw image using
>>>>>> qemu-img) that i want to add to glance:
>>>>>>
>>>>>> 1. How can I tell whether it's an "ovf" or "bare" container format?
>>>>>
>>>>> You are mixing up terminology here. Disk image formats are things like
>>>>> raw, qcow2, vmdk, etc.
>>>>>
>>>>> OVF refers to the format of a metadata file provided alongside the
>>>>> disk image, which describes various requirements for running the
>>>>> image.
>>>>>
>>>>> The two are not tied together at all, merely complementary to
>>>>> each other.
>>>>>
>>>>
>>>> Thanks, that clears things up. I was confused by this language, which sounded to me like the metadata was embedded in the disk image file:
>>>>
>>>> http://glance.openstack.org/formats.html
>>>>
>>>> "The container format refers to whether the virtual machine image is in a file format that also contains metadata about the actual virtual machine."
>>>>
>>>> In addition, the docs have examples like this, which clearly aren't meaningful:
>>>> http://glance.openstack.org/glance.html#important-information-about-uploading-images
>>>
>>> Just to add to the confusion the OVF can contain both the metadata file and the disk image file in a single archived file.
>>>
>>> "An OVF package consists of several files, placed in one directory. A one-file alternative is the OVA package, which is a TAR file with the OVF directory inside."
>>>
>>> http://en.wikipedia.org/wiki/Open_Virtualization_Format#Technical_description
>>
>>
>> Does anybody know if OpenStack (nova+glance) currently supports OVA packages?
>>
>> Take care,
>>
>> Lorin
>> --
>> Lorin Hochstein
>> Lead Architect - Cloud Services
>> Nimbis Services, Inc.
>> www.nimbisservices.com
>>
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>

chris.behrens at RACKSPACE

Jul 9, 2012, 12:55 PM

Post #11 of 11 (394 views)
Permalink

Re: OVF vs. bare container formats for qcow2 images [In reply to]

That's correct. If there's an .ovf, it's currently ignored. It's also not generated when we upload images (snapshots). I'm sure there's plans in someone's head to implement that support at some point.. but right now we expect the .vhds to have specific names in the tar file.

- Chris

On Jul 8, 2012, at 6:01 PM, Lorin Hochstein wrote:

Paul:

I assume you're using XenServer? When I grepped through the code, there appears to be a XenServer plugin for glance for supporting tar balls that could be OVAs, but don't necessarily contain the OVF file (?).

Here's the code:
https://github.com/openstack/nova/blob/stable/essex/plugins/xenserver/xenapi/etc/xapi.d/plugins/glance#L365

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com<https://www.nimbisservices.com/>

On Jul 5, 2012, at 11:22 AM, Paul Voccio wrote:

Lorin,

We've been using OVA packages since the beginning. I believe there is a flag in glance for this.

Thanks,
~pvo

Paul Voccio
paul.voccio [at] rackspace<mailto:paul.voccio [at] rackspace>
770-335-2143 (c)
pvo on #openstack
ಠ_ಠ

On Jul 3, 2012, at 9:07 PM, Lorin Hochstein wrote:

On Jun 29, 2012, at 9:53 PM, Adam Young wrote:

On 04/01/2012 11:15 AM, Lorin Hochstein wrote:

On Mar 29, 2012, at 12:40 PM, Daniel P. Berrange wrote:

On Wed, Mar 28, 2012 at 04:41:28PM -0400, Lorin Hochstein wrote:
All:

Given that I have a qcow2 image from somewhere (e.g., downloaded
it from a uec-images.ubuntu.com<http://uec-images.ubuntu.com/>, created one from a raw image using
qemu-img) that i want to add to glance:

1. How can I tell whether it's an "ovf" or "bare" container format?

You are mixing up terminology here. Disk image formats are things like
raw, qcow2, vmdk, etc.

OVF refers to the format of a metadata file provided alongside the
disk image, which describes various requirements for running the
image.

The two are not tied together at all, merely complementary to
each other.

Thanks, that clears things up. I was confused by this language, which sounded to me like the metadata was embedded in the disk image file:

http://glance.openstack.org/formats.html

"The container format refers to whether the virtual machine image is in a file format that also contains metadata about the actual virtual machine."

In addition, the docs have examples like this, which clearly aren't meaningful:
http://glance.openstack.org/glance.html#important-information-about-uploading-images

Just to add to the confusion the OVF can contain both the metadata file and the disk image file in a single archived file.

"An OVF package consists of several files, placed in one directory. A one-file alternative is the OVA package, which is a TAR file with the OVF directory inside."

http://en.wikipedia.org/wiki/Open_Virtualization_Format#Technical_description

Does anybody know if OpenStack (nova+glance) currently supports OVA packages?

Take care,

Lorin
--
Lorin Hochstein
Lead Architect - Cloud Services
Nimbis Services, Inc.
www.nimbisservices.com<https://www.nimbisservices.com/>

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists<mailto:openstack [at] lists>
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads