Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

File injection support

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


n.paladi at gmail

May 29, 2012, 11:35 AM

Post #1 of 22 (3487 views)
Permalink
File injection support

Hello,

I am looking for more information about file injection support in OpenStack
and find it increasingly inconsistent and incomplete. I have several
questions:

* This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix)
claims
that file injection is supported in XenServer, is NOT supported inn KVM
it's not clear about the others. is that still the case? At least there is
some mentioning of file injection in the openstack xenserver wiki:
http://wiki.openstack.org/XenServer/Development

* Is there any guide or man page where file injection is explained? in this
use case I would like to inject a file to e.g. /etc/security/zzzz.conf at
launch
time -- is that possible, and if yes how can it be done in the Essex
release.

* Are there any discussions about "the future" of file injection?

Thanks a lot!
/Nico.


matt.joyce at cloudscaling

May 29, 2012, 1:21 PM

Post #2 of 22 (3449 views)
Permalink
Re: File injection support [In reply to]

https://help.ubuntu.com/community/CloudInit

Might want to check this little beastie out. Folks using kvm seem to end
up here for certain use cases.

-Matt

On Tue, May 29, 2012 at 11:35 AM, Nicolae Paladi <n.paladi [at] gmail> wrote:

> Hello,
>
> I am looking for more information about file injection support in OpenStack
> and find it increasingly inconsistent and incomplete. I have several
> questions:
>
> * This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix)
> claims
> that file injection is supported in XenServer, is NOT supported inn KVM
> it's not clear about the others. is that still the case? At least there is
> some mentioning of file injection in the openstack xenserver wiki:
> http://wiki.openstack.org/XenServer/Development
>
> * Is there any guide or man page where file injection is explained? in this
> use case I would like to inject a file to e.g. /etc/security/zzzz.conf at
> launch
> time -- is that possible, and if yes how can it be done in the Essex
> release.
>
> * Are there any discussions about "the future" of file injection?
>
> Thanks a lot!
> /Nico.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>


P at draigBrady

May 29, 2012, 4:21 PM

Post #3 of 22 (3450 views)
Permalink
Re: File injection support [In reply to]

On 05/29/2012 07:35 PM, Nicolae Paladi wrote:
> Hello,
>
> I am looking for more information about file injection support in OpenStack
> and find it increasingly inconsistent and incomplete. I have several questions:
>
> * This wiki article (http://wiki.openstack.org/HypervisorSupportMatrix) claims
> that file injection is supported in XenServer, is NOT supported inn KVM
> it's not clear about the others. is that still the case?

One of the key Essex Features was "Hypervisor Feature Parity", including:
Disk Config, resize and file injection.

I've updated the wiki accordingly.

So there is support in "kvm" for injecting ssh keys, net info, root password,
or arbitrary files.

By default there are 3 methods tried in turn to do this:
loopback mount, qemu-nbd, libguestfs.

In Essex injection is supported to the first partition of guest images.
Folsom enables further libguestfs functionality to inspect arbitrarily
complex guest images and find the root partition to inject to.

> At least there is
> some mentioning of file injection in the openstack xenserver wiki:
> http://wiki.openstack.org/XenServer/Development
>
> * Is there any guide or man page where file injection is explained? in this
> use case I would like to inject a file to e.g. /etc/security/zzzz.conf at launch
> time -- is that possible, and if yes how can it be done in the Essex release.

The API docs explain the "personality" parameter to achieve this:
http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html
This is exposed on the nova command line tool though the --injected-files option.

> * Are there any discussions about "the future" of file injection?

There are other options, like cloud-init or using a config drive.
You enable a config drive with the nova --config-drive option,
with value as a boolean or a volume ID. See also:
https://blueprints.launchpad.net/nova/+spec/config-drive-v2

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


fredric.morenius at ericsson

May 30, 2012, 7:47 AM

Post #4 of 22 (3456 views)
Permalink
Re: File injection support [In reply to]

Hello Pádraig,

I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878

The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img )

Would it be possible to backport this: https://github.com/openstack/nova/commit/2b3a1e7
So that file injection as I am trying to do it will work? Or is there any other way to make it work?

Thanks and Best Regards,
Fredric

***************** NOT A CONTRIBUTION *****************
The content of this email shall not be considered as a contribution to OpenStack

-----Original Message-----
From: openstack-bounces+fredric.morenius=ericsson.com [at] lists [mailto:openstack-bounces+fredric.morenius=ericsson.com [at] lists] On Behalf Of Pádraig Brady
Sent: den 30 maj 2012 01:22
To: Nicolae Paladi
Cc: openstack
Subject: Re: [Openstack] File injection support

On 05/29/2012 07:35 PM, Nicolae Paladi wrote:
> Hello,
>
> I am looking for more information about file injection support in
> OpenStack and find it increasingly inconsistent and incomplete. I have several questions:
>
> * This wiki article
> (http://wiki.openstack.org/HypervisorSupportMatrix) claims that file
> injection is supported in XenServer, is NOT supported inn KVM it's not clear about the others. is that still the case?

One of the key Essex Features was "Hypervisor Feature Parity", including:
Disk Config, resize and file injection.

I've updated the wiki accordingly.

So there is support in "kvm" for injecting ssh keys, net info, root password, or arbitrary files.

By default there are 3 methods tried in turn to do this:
loopback mount, qemu-nbd, libguestfs.

In Essex injection is supported to the first partition of guest images.
Folsom enables further libguestfs functionality to inspect arbitrarily complex guest images and find the root partition to inject to.

> At least there is
> some mentioning of file injection in the openstack xenserver wiki:
> http://wiki.openstack.org/XenServer/Development
>
> * Is there any guide or man page where file injection is explained? in
> this use case I would like to inject a file to e.g.
> /etc/security/zzzz.conf at launch time -- is that possible, and if yes how can it be done in the Essex release.

The API docs explain the "personality" parameter to achieve this:
http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html
This is exposed on the nova command line tool though the --injected-files option.

> * Are there any discussions about "the future" of file injection?

There are other options, like cloud-init or using a config drive.
You enable a config drive with the nova --config-drive option, with value as a boolean or a volume ID. See also:
https://blueprints.launchpad.net/nova/+spec/config-drive-v2

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


smoser at ubuntu

May 30, 2012, 8:58 AM

Post #5 of 22 (3447 views)
Permalink
Re: File injection support [In reply to]

On Wed, 30 May 2012, Fredric Morenius wrote:

> Hello Pádraig,
>
> I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878
>
> The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img )

I generally think that "file injection" is significantly less preferable
to use of user-data or config-drive (an api/convention for providing
information to an instance).

cirros supports executing user-data starting with '#!', so you can
probably accomplish what you're after via passing data like this:

#!/bin/sh
cat > "/path/to/file" <<"EOF"
file contents here
EOF

wget http://some.url/my.tar | ( cd / && tar xvzf - )

base64 -d > /usr/bin/my-exe <<EOF
IyEvYmluL3NoCmVjaG8gImhpIHdvcmxkIgo=
EOF

chmod 755 /usr/bin/my-exe
#### end user data ###

You could also use 'shar' to do this.


P at draigBrady

May 30, 2012, 9:03 AM

Post #6 of 22 (3451 views)
Permalink
Re: File injection support [In reply to]

On 05/30/2012 03:47 PM, Fredric Morenius wrote:
> Hello Pádraig,
>
> I am also trying to get file injection to work in Essex, but have run into some issues, as stated here: https://answers.launchpad.net/nova/+question/198878

Igor Laskovy also had that "device-mapper: resume ioctl failed:" issue
with the "qemu-nbd" on a cirros image too,
though he didn't need the injection though and just avoided it.

> The image I am launching is a simple bare container qcow2 image (CirrOS, this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img )

That image is a simple single partition image,
and so should not need the patch referenced below.
What system are you using? If you installed libguestfs then
that should be tried as a method without need for the patch below.

> Would it be possible to backport this: https://github.com/openstack/nova/commit/2b3a1e7
> So that file injection as I am trying to do it will work? Or is there any other way to make it work?

The backport is trivial and already done in the Fedora/EPEL Essex packages.

I was thinking though that this was extra functionality
and so not appropriate for the official stable branch?

> ***************** NOT A CONTRIBUTION *****************
> The content of this email shall not be considered as a contribution to OpenStack

:)

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


fredric.morenius at ericsson

Jun 8, 2012, 1:20 AM

Post #7 of 22 (3422 views)
Permalink
Re: File injection support [In reply to]

Hello All,

An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images:

After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools.

BR Fredric

-----Original Message-----
From: Pádraig Brady [mailto:P [at] draigBrady]
Sent: den 30 maj 2012 18:03
To: Fredric Morenius
Cc: openstack; igor.laskovy [at] gmail
Subject: Re: [Openstack] File injection support

On 05/30/2012 03:47 PM, Fredric Morenius wrote:
> Hello Pádraig,
>
> I am also trying to get file injection to work in Essex, but have run
> into some issues, as stated here:
> https://answers.launchpad.net/nova/+question/198878

Igor Laskovy also had that "device-mapper: resume ioctl failed:" issue with the "qemu-nbd" on a cirros image too, though he didn't need the injection though and just avoided it.

> The image I am launching is a simple bare container qcow2 image
> (CirrOS, this:
> https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64
> -disk.img )

That image is a simple single partition image, and so should not need the patch referenced below.
What system are you using? If you installed libguestfs then that should be tried as a method without need for the patch below.

> Would it be possible to backport this:
> https://github.com/openstack/nova/commit/2b3a1e7
> So that file injection as I am trying to do it will work? Or is there any other way to make it work?

The backport is trivial and already done in the Fedora/EPEL Essex packages.

I was thinking though that this was extra functionality and so not appropriate for the official stable branch?

> ***************** NOT A CONTRIBUTION ***************** The content of
> this email shall not be considered as a contribution to OpenStack

:)

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


igor.laskovy at gmail

Jun 8, 2012, 2:32 AM

Post #8 of 22 (3412 views)
Permalink
Re: File injection support [In reply to]

Thank for this

Igor Laskovy
Kiev, Ukraine
On Jun 8, 2012 11:20 AM, "Fredric Morenius" <fredric.morenius [at] ericsson>
wrote:

> Hello All,
>
> An update on the use of the qemu-nbd/kpartx based solution to inject files
> into VM images:
>
> After some more testing it has turned out that injection into the UEC
> version of CirrOS (this:
> https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz)
> works fine, but injection into the qcow2 version of the image produces the
> error given in the previous mail, so there seems to be robustness problems
> with these tools.
>
> BR Fredric
>
> -----Original Message-----
> From: Pádraig Brady [mailto:P [at] draigBrady]
> Sent: den 30 maj 2012 18:03
> To: Fredric Morenius
> Cc: openstack; igor.laskovy [at] gmail
> Subject: Re: [Openstack] File injection support
>
> On 05/30/2012 03:47 PM, Fredric Morenius wrote:
> > Hello Pádraig,
> >
> > I am also trying to get file injection to work in Essex, but have run
> > into some issues, as stated here:
> > https://answers.launchpad.net/nova/+question/198878
>
> Igor Laskovy also had that "device-mapper: resume ioctl failed:" issue
> with the "qemu-nbd" on a cirros image too, though he didn't need the
> injection though and just avoided it.
>
> > The image I am launching is a simple bare container qcow2 image
> > (CirrOS, this:
> > https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64
> > -disk.img )
>
> That image is a simple single partition image, and so should not need the
> patch referenced below.
> What system are you using? If you installed libguestfs then that should be
> tried as a method without need for the patch below.
>
> > Would it be possible to backport this:
> > https://github.com/openstack/nova/commit/2b3a1e7
> > So that file injection as I am trying to do it will work? Or is there
> any other way to make it work?
>
> The backport is trivial and already done in the Fedora/EPEL Essex packages.
>
> I was thinking though that this was extra functionality and so not
> appropriate for the official stable branch?
>
> > ***************** NOT A CONTRIBUTION ***************** The content of
> > this email shall not be considered as a contribution to OpenStack
>
> :)
>
> cheers,
> Pádraig.
>


P at draigBrady

Jun 8, 2012, 2:49 AM

Post #9 of 22 (3417 views)
Permalink
Re: File injection support [In reply to]

On 06/08/2012 09:20 AM, Fredric Morenius wrote:
> Hello All,
>
> An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images:
>
> After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools.

Yes it's seems that qemu-nbd has issues with this image?

# rpm -qf $(which qemu-nbd)
qemu-common-1.0-17.fc17.x86_64
# qemu-img info cirros-0.3.0-x86_64-disk.img
image: cirros-0.3.0-x86_64-disk.img
file format: qcow2
virtual size: 39M (41126400 bytes)
disk size: 9.3M
cluster_size: 65536
# qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
# ls -la /sys/block/nbd15/pid
-r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid
# kpartx -a /dev/nbd15
device-mapper: resume ioctl on nbd15p1 failed: Invalid argument
create/reload failed on nbd15p1

If I convert to raw with qemu-img, I can mount loopback fine.

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


smoser at ubuntu

Jun 8, 2012, 8:35 AM

Post #10 of 22 (3426 views)
Permalink
Re: File injection support [In reply to]

On Fri, 8 Jun 2012, Pádraig Brady wrote:

> On 06/08/2012 09:20 AM, Fredric Morenius wrote:
> > Hello All,
> >
> > An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images:
> >
> > After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools.
>
> Yes it's seems that qemu-nbd has issues with this image?
>
> # rpm -qf $(which qemu-nbd)
> qemu-common-1.0-17.fc17.x86_64
> # qemu-img info cirros-0.3.0-x86_64-disk.img
> image: cirros-0.3.0-x86_64-disk.img
> file format: qcow2
> virtual size: 39M (41126400 bytes)
> disk size: 9.3M
> cluster_size: 65536
> # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
> # ls -la /sys/block/nbd15/pid
> -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid
> # kpartx -a /dev/nbd15
> device-mapper: resume ioctl on nbd15p1 failed: Invalid argument
> create/reload failed on nbd15p1
>
> If I convert to raw with qemu-img, I can mount loopback fine.

Well, interesting. It does seem to work here:
$ dpkg-query --show qemu-utils
qemu-utils 1.0+noroms-0ubuntu13
$ cd /tmp
$ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
$ md5sum cirros-0.3.0-x86_64-disk.img
6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img
$ qemu-img info cirros-0.3.0-x86_64-disk.img
image: cirros-0.3.0-x86_64-disk.img
file format: qcow2
virtual size: 39M (41126400 bytes)
disk size: 11M
cluster_size: 65536
$ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
$ ls -l /dev/nbd15*
brw-rw---- 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15
brw-rw---- 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1
$ sudo mount /dev/nbd15p1 /mnt
$ ls /mnt
bin etc initrd.img lost+found opt run tmp vmlinuz
boot home lib mnt proc sbin usr
dev init linuxrc old-root root sys var
$ sudo umount /mnt
$ sudo qemu-nbd -d /dev/nbd15
/dev/nbd15 disconnected


smoser at ubuntu

Jun 8, 2012, 8:48 AM

Post #11 of 22 (3432 views)
Permalink
Re: File injection support [In reply to]

On Fri, 8 Jun 2012, Scott Moser wrote:

> On Fri, 8 Jun 2012, Pádraig Brady wrote:
>
> > On 06/08/2012 09:20 AM, Fredric Morenius wrote:
> > > Hello All,
> > >
> > > An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images:
> > >
> > > After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools.
> >
> > Yes it's seems that qemu-nbd has issues with this image?
> >
> > # rpm -qf $(which qemu-nbd)
> > qemu-common-1.0-17.fc17.x86_64
> > # qemu-img info cirros-0.3.0-x86_64-disk.img
> > image: cirros-0.3.0-x86_64-disk.img
> > file format: qcow2
> > virtual size: 39M (41126400 bytes)
> > disk size: 9.3M
> > cluster_size: 65536
> > # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
> > # ls -la /sys/block/nbd15/pid
> > -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid
> > # kpartx -a /dev/nbd15
> > device-mapper: resume ioctl on nbd15p1 failed: Invalid argument
> > create/reload failed on nbd15p1
> >
> > If I convert to raw with qemu-img, I can mount loopback fine.
>
> Well, interesting. It does seem to work here:
> $ dpkg-query --show qemu-utils
> qemu-utils 1.0+noroms-0ubuntu13
> $ cd /tmp
> $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
> $ md5sum cirros-0.3.0-x86_64-disk.img
> 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img

Whoops. I had a local image that was different (must have been laying
around from long ago, the wget saved to .1). It does still work though.

$ md5sum cirros-0.3.0-x86_64-disk.img
50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img

$ qemu-img info cirros-0.3.0-x86_64-disk.img
image: cirros-0.3.0-x86_64-disk.img
file format: qcow2
virtual size: 39M (41126400 bytes)
disk size: 9.3M
cluster_size: 65536

The same nbd -c... mount work as shown.


P at draigBrady

Jun 8, 2012, 9:03 AM

Post #12 of 22 (3430 views)
Permalink
Re: File injection support [In reply to]

On 06/08/2012 04:35 PM, Scott Moser wrote:
> On Fri, 8 Jun 2012, Pádraig Brady wrote:
>
>> On 06/08/2012 09:20 AM, Fredric Morenius wrote:
>>> Hello All,
>>>
>>> An update on the use of the qemu-nbd/kpartx based solution to inject files into VM images:
>>>
>>> After some more testing it has turned out that injection into the UEC version of CirrOS (this: https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz) works fine, but injection into the qcow2 version of the image produces the error given in the previous mail, so there seems to be robustness problems with these tools.
>>
>> Yes it's seems that qemu-nbd has issues with this image?
>>
>> # rpm -qf $(which qemu-nbd)
>> qemu-common-1.0-17.fc17.x86_64
>> # qemu-img info cirros-0.3.0-x86_64-disk.img
>> image: cirros-0.3.0-x86_64-disk.img
>> file format: qcow2
>> virtual size: 39M (41126400 bytes)
>> disk size: 9.3M
>> cluster_size: 65536
>> # qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
>> # ls -la /sys/block/nbd15/pid
>> -r--r--r--. 1 root root 4096 Jun 8 10:19 /sys/block/nbd15/pid
>> # kpartx -a /dev/nbd15
>> device-mapper: resume ioctl on nbd15p1 failed: Invalid argument
>> create/reload failed on nbd15p1
>>
>> If I convert to raw with qemu-img, I can mount loopback fine.
>
> Well, interesting. It does seem to work here:
> $ dpkg-query --show qemu-utils
> qemu-utils 1.0+noroms-0ubuntu13
> $ cd /tmp
> $ wget -q https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
> $ md5sum cirros-0.3.0-x86_64-disk.img
> 6654705afc4b74fda3e1f4330559d066 cirros-0.3.0-x86_64-disk.img

I downloaded again and get 50bdc35edb03a38d91b1b071afb20a3c
I presume the above md5sum is for a mounted/modified image?

> $ qemu-img info cirros-0.3.0-x86_64-disk.img
> image: cirros-0.3.0-x86_64-disk.img
> file format: qcow2
> virtual size: 39M (41126400 bytes)
> disk size: 11M
> cluster_size: 65536
> $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
> $ ls -l /dev/nbd15*
> brw-rw---- 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15
> brw-rw---- 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1

Very interesting. I don't get the ...p1 device here
on a 3.3.7.fc17.x86_64 kernel

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


vishvananda at gmail

Jun 10, 2012, 1:21 PM

Post #13 of 22 (3401 views)
Permalink
Re: File injection support [In reply to]

On Jun 8, 2012, at 9:03 AM, Pádraig Brady wrote:
>
>> $ qemu-img info cirros-0.3.0-x86_64-disk.img
>> image: cirros-0.3.0-x86_64-disk.img
>> file format: qcow2
>> virtual size: 39M (41126400 bytes)
>> disk size: 11M
>> cluster_size: 65536
>> $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img
>> $ ls -l /dev/nbd15*
>> brw-rw---- 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15
>> brw-rw---- 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1
>
> Very interesting. I don't get the ...p1 device here
> on a 3.3.7.fc17.x86_64 kernel


did you modrpobe nbd with max_part=8 or some such? It won't detect partitions automatically unless you do so.

We really should stop using kpartx and force people to modprobe nbd with max_part=X

There seem to be various issues with kpartx

Vish

>
> cheers,
> Pádraig.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


fredric.morenius at ericsson

Jun 11, 2012, 11:48 AM

Post #14 of 22 (3427 views)
Permalink
Re: File injection support [In reply to]

Hello again,

I did some further tests on this issue with the qcow2 image, the qcow2 image converted to raw and the _blank (raw) image from the UEC tar archive.

I loaded nbd with

$ sudo modprobe nbd max_part=8

*** qcow2 image ***

$ k$ md5sum cirros-0.3.0-x86_64-disk.img
50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img

$ qemu-img info cirros-0.3.0-x86_64-disk.img
image: cirros-0.3.0-x86_64-disk.img
file format: qcow2
virtual size: 39M (41126400 bytes)
disk size: 9.3M
cluster_size: 65536

$ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img

That creates the device /dev/nbd15p1

kpartx on top of that:

$ sudo kpartx -a /dev/nbd15
device-mapper: resume ioctl failed: Invalid argument
create/reload failed on nbd15p1

$ sudo qemu-nbd -d /dev/nbd15
/dev/nbd15 disconnected

*** UEC image ***

$ qemu-img info cirros-0.3.0-x86_64-blank.img
image: cirros-0.3.0-x86_64-blank.img
file format: raw
virtual size: 24M (25165824 bytes)
disk size: 24M

$ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-blank.img

That DOES NOT create the device /dev/nbd15p1

kpartx on top of that:

$ sudo kpartx -a /dev/nbd15

kpartx executes without error

$ sudo qemu-nbd -d /dev/nbd15
/dev/nbd15 disconnected

*** raw image ***

$ qemu-img convert -p -O raw cirros-0.3.0-x86_64-disk.img cirros-0.3.0-x86_64-disk_raw.img

$ qemu-img info cirros-0.3.0-x86_64-disk_raw.img
image: cirros-0.3.0-x86_64-disk_raw.img
file format: raw
virtual size: 39M (41126400 bytes)
disk size: 12M

$ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk_raw.img

That creates the device /dev/nbd15p1

kpartx on top of that:

$ sudo kpartx -a /dev/nbd15
device-mapper: resume ioctl failed: Invalid argument
create/reload failed on nbd15p1

$ sudo qemu-nbd -d /dev/nbd15
/dev/nbd15 disconnected

So, the issue is not with the qcow2 vs raw image formats. Instead kpartx only executes without error provided that the device /dev/nbd15p1 is not created.
When p1 is created by qemu-nbd, kpartx is not needed to mount the image device, but the problem is qemu-nbd does not create p1 in all cases. I have not checked the details of the OpenStack code, but an (ugly) work around could be to only execute kpartx when it is needed.

Best Regards,
Fredric

-----Original Message-----
From: openstack-bounces+fredric.morenius=ericsson.com [at] lists [mailto:openstack-bounces+fredric.morenius=ericsson.com [at] lists] On Behalf Of Vishvananda Ishaya
Sent: den 10 juni 2012 22:22
To: Pádraig Brady
Cc: igor.laskovy [at] gmail; openstack
Subject: Re: [Openstack] File injection support


On Jun 8, 2012, at 9:03 AM, Pádraig Brady wrote:
>
>> $ qemu-img info cirros-0.3.0-x86_64-disk.img
>> image: cirros-0.3.0-x86_64-disk.img
>> file format: qcow2
>> virtual size: 39M (41126400 bytes)
>> disk size: 11M
>> cluster_size: 65536
>> $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk.img $ ls
>> -l /dev/nbd15*
>> brw-rw---- 1 root disk 43, 240 Jun 8 11:32 /dev/nbd15
>> brw-rw---- 1 root disk 43, 241 Jun 8 11:32 /dev/nbd15p1
>
> Very interesting. I don't get the ...p1 device here on a
> 3.3.7.fc17.x86_64 kernel


did you modrpobe nbd with max_part=8 or some such? It won't detect partitions automatically unless you do so.

We really should stop using kpartx and force people to modprobe nbd with max_part=X

There seem to be various issues with kpartx

Vish

>
> cheers,
> Pádraig.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


smoser at ubuntu

Jun 11, 2012, 2:16 PM

Post #15 of 22 (3416 views)
Permalink
Re: File injection support [In reply to]

On Mon, 11 Jun 2012, Fredric Morenius wrote:

> Hello again,
>
> I did some further tests on this issue with the qcow2 image, the qcow2 image converted to raw and the _blank (raw) image from the UEC tar archive.
>
> I loaded nbd with
>
> $ sudo modprobe nbd max_part=8
>
> *** qcow2 image ***
>
> $ k$ md5sum cirros-0.3.0-x86_64-disk.img
> 50bdc35edb03a38d91b1b071afb20a3c cirros-0.3.0-x86_64-disk.img

this image is a qcow compressed disk image.
The disk contains a partition table, grub (0.97) installed on the MBR, and
a root filesystem on the first partition.

> That creates the device /dev/nbd15p1
>
> kpartx on top of that:
>
> $ sudo kpartx -a /dev/nbd15
> device-mapper: resume ioctl failed: Invalid argument
> create/reload failed on nbd15p1
>
> $ sudo qemu-nbd -d /dev/nbd15
> /dev/nbd15 disconnected

That seems like "working as designed". at least on ubuntu 12.04, the kpartx is
no longer necessary. I'd have to dig more to figure out exactly why, but
some combination of kernel partition scanner and udev "do the right
thing" when this new network block device is set up.

> *** UEC image ***
>
> $ qemu-img info cirros-0.3.0-x86_64-blank.img
> image: cirros-0.3.0-x86_64-blank.img
> file format: raw
> virtual size: 24M (25165824 bytes)
> disk size: 24M
>
> $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-blank.img

This image is essentially the same as you'd get if you did:
truncate --size 24M disk.img
mkfs.ext3 -F disk.img -L cirros-rootfs

Ie, it is a unpartitioned disk with an (empty) filesystem on it.
There are no partitions, or data in the filesystem.

On boot, cirros initramfs copies itself to that disk, and then pivot_roots
to it. Cirros will run perfectly fine as an initramfs with no root
filesystem too, fwiw.

> That DOES NOT create the device /dev/nbd15p1
>
> kpartx on top of that:
>
> $ sudo kpartx -a /dev/nbd15
>
> kpartx executes without error
>
> $ sudo qemu-nbd -d /dev/nbd15
> /dev/nbd15 disconnected

This also seems like "working as designed".

> *** raw image ***
>
> $ qemu-img convert -p -O raw cirros-0.3.0-x86_64-disk.img cirros-0.3.0-x86_64-disk_raw.img
>
> $ qemu-img info cirros-0.3.0-x86_64-disk_raw.img
> image: cirros-0.3.0-x86_64-disk_raw.img
> file format: raw
> virtual size: 39M (41126400 bytes)
> disk size: 12M
>
> $ sudo qemu-nbd -c /dev/nbd15 $PWD/cirros-0.3.0-x86_64-disk_raw.img
>
> That creates the device /dev/nbd15p1
>
> kpartx on top of that:
>
> $ sudo kpartx -a /dev/nbd15
> device-mapper: resume ioctl failed: Invalid argument
> create/reload failed on nbd15p1
>
> $ sudo qemu-nbd -d /dev/nbd15
> /dev/nbd15 disconnected
>
> So, the issue is not with the qcow2 vs raw image formats. Instead kpartx only executes without error provided that the device /dev/nbd15p1 is not created.
> When p1 is created by qemu-nbd, kpartx is not needed to mount the image
> device, but the problem is qemu-nbd does not create p1 in all cases. I
> have not checked the details of the OpenStack code, but an (ugly) work
> around could be to only execute kpartx when it is needed.

Without digging around on older versions of OS's and their included
kernel/udev/kpartx.... i'd have to say that from your message above, and
my testing that running 'kpartx' is no longer necessary. That can
reasonably be expected to be created by the kernel or some other plumbing
bits automatically.

If it just happens magically,though, I'm not sure how you would know if it
hasn't happened yet. That would imply that this might be racey. I ran
into similar bugs http://pad.lv/937352 and http://pad.lv/942788 that
affected the growing of the root partition from an initramfs.

The thing we ended up settling on was that after re-writing the partition
table, we had to run 'udevadm --settle'. its quite possible that
'qemu-nbd --connect' sets off a similar flurry of events that would need
to be waited on before a /dev/nbdXpY would exist.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


P at draigBrady

Jun 12, 2012, 7:01 AM

Post #16 of 22 (3501 views)
Permalink
Re: File injection support [In reply to]

On 06/11/2012 10:16 PM, Scott Moser wrote:
> Without digging around on older versions of OS's and their included
> kernel/udev/kpartx.... i'd have to say that from your message above, and
> my testing that running 'kpartx' is no longer necessary. That can
> reasonably be expected to be created by the kernel or some other plumbing
> bits automatically.

It would be good to get away from kpartx. I've noticed these issues:

1. The git repo on kernel.org is no longer available.

2. kpartx -l had side effects:
$ kpartx -l /bin/ls
$ ls
text file busy
To fix you need to run losetup -a to find the assigned loopback device
and then losetup -d /dev/loop...

3. On an unconnected loop device we get warnings, but an EXIT_SUCCESS ?
# kpartx -a /dev/loop1 && echo EXIT_SUCCESS
read error, sector 0
llseek error
llseek error
llseek error
EXIT_SUCCESS

4. Also for a loop device that is connected,
I get a "failed" warning, but the EXIT_SUCCESS
is appropriate in that case as the mapped device
is present and usable
# kpartx -a /dev/loop0
/dev/mapper/loop0p1: mknod for loop0p1 failed: File exists

That last item is related to the new code for auto parsing partitions.

That's only available since kernel 3.2 I think so we'll have to
be wary on relying on it.

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


smoser at ubuntu

Jun 12, 2012, 7:46 AM

Post #17 of 22 (3407 views)
Permalink
Re: File injection support [In reply to]

On Tue, 12 Jun 2012, Pádraig Brady wrote:


> 4. Also for a loop device that is connected,
> I get a "failed" warning, but the EXIT_SUCCESS
> is appropriate in that case as the mapped device
> is present and usable
> # kpartx -a /dev/loop0
> /dev/mapper/loop0p1: mknod for loop0p1 failed: File exists
>
> That last item is related to the new code for auto parsing partitions.
>
> That's only available since kernel 3.2 I think so we'll have to
> be wary on relying on it.

Its worth noting that mounting untrusted filesystems from the host
is not really safe. See "Security of mounting filesystems" at
http://libguestfs.org/guestfs.3.html .

The libguestfs path for this is safer. So really we should be looking to
fix this issue by stronger recommendation or requirement on libguestfs.
Or some other non-kernel level filesystem modification.


fredric.morenius at ericsson

Jun 12, 2012, 8:07 AM

Post #18 of 22 (3401 views)
Permalink
Re: File injection support [In reply to]

From: Scott Moser [mailto:ssmoser2 [at] gmail] On Behalf Of Scott Moser
Sent: den 11 juni 2012 23:16

>...Without digging around on older versions of OS's and their included kernel/udev/kpartx.... i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by >the kernel or some other plumbing bits automatically...

I just now tested to remove the kpartx invocation from /nova/virt/disk/mount.py, like this:

diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
index 11959b2..4d9527b 100644
--- a/nova/virt/disk/mount.py
+++ b/nova/virt/disk/mount.py
@@ -61,25 +61,9 @@ class Mount(object):
if self.partition == -1:
self.error = _('partition search unsupported with %s') % self.mode
elif self.partition:
- map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
- self.partition)
- assert(not os.path.exists(map_path))
-
- # Note kpartx can output warnings to stderr and succeed
- # Also it can output failures to stderr and "succeed"
- # So we just go on the existence of the mapped device
- _out, err = utils.trycmd('kpartx', '-a', self.device,
- run_as_root=True, discard_warnings=True)
-
- # Note kpartx does nothing when presented with a raw image,
- # so given we only use it when we expect a partitioned image, fail
- if not os.path.exists(map_path):
- if not err:
- err = _('partition %s not found') % self.partition
- self.error = _('Failed to map partitions: %s') % err
- else:
- self.mapped_device = map_path
- self.mapped = True
+ #qemu-nbd already has mapped the partition for mounting
+ self.mapped_device = '%sp%s' % (self.device, self.partition)
+ self.mapped = True
else:
self.mapped_device = self.device
self.mapped = True
@@ -90,8 +74,7 @@ class Mount(object):
"""Remove partitions of the device from the file system namespace."""
if not self.mapped:
return
- if self.partition:
- utils.execute('kpartx', '-d', self.device, run_as_root=True)
+ #No need to run kpartx
self.mapped = False

to rely on the qemu-nbd mapping. With the above change, file injection works fine with the CirrOS qcow2 image (I have not tested with any other image yet).
I run Ubuntu 12.04 Desktop with the 3.2.0-24-generic kernel and qemu-utils package version 1.0+noroms-0ubuntu13

Best Regards,
Fredric

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


P at draigBrady

Jun 12, 2012, 8:20 AM

Post #19 of 22 (3394 views)
Permalink
Re: File injection support [In reply to]

On 06/12/2012 04:07 PM, Fredric Morenius wrote:
> From: Scott Moser [mailto:ssmoser2 [at] gmail] On Behalf Of Scott Moser
> Sent: den 11 juni 2012 23:16
>
>> ...Without digging around on older versions of OS's and their included kernel/udev/kpartx.... i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by >the kernel or some other plumbing bits automatically...
>
> I just now tested to remove the kpartx invocation from /nova/virt/disk/mount.py, like this:
>
> diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
> index 11959b2..4d9527b 100644
> --- a/nova/virt/disk/mount.py
> +++ b/nova/virt/disk/mount.py
> @@ -61,25 +61,9 @@ class Mount(object):
> if self.partition == -1:
> self.error = _('partition search unsupported with %s') % self.mode
> elif self.partition:
> - map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
> - self.partition)
> - assert(not os.path.exists(map_path))
> -
> - # Note kpartx can output warnings to stderr and succeed
> - # Also it can output failures to stderr and "succeed"
> - # So we just go on the existence of the mapped device
> - _out, err = utils.trycmd('kpartx', '-a', self.device,
> - run_as_root=True, discard_warnings=True)
> -
> - # Note kpartx does nothing when presented with a raw image,
> - # so given we only use it when we expect a partitioned image, fail
> - if not os.path.exists(map_path):
> - if not err:
> - err = _('partition %s not found') % self.partition
> - self.error = _('Failed to map partitions: %s') % err
> - else:
> - self.mapped_device = map_path
> - self.mapped = True
> + #qemu-nbd already has mapped the partition for mounting
> + self.mapped_device = '%sp%s' % (self.device, self.partition)
> + self.mapped = True

Looks good. That should also support raw through the /dev/loop%s devices.

It might be safer to fallback to kpartx if not exists ...
'%sp%s' % (self.device, self.partition)
That would support kernels before 3.2
I'd also add a comment in the code, that nbd must be mounted with param
max_part=16 or whatever.

cheers,
Pádraig.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


moseleymark at gmail

Jul 13, 2012, 4:09 PM

Post #20 of 22 (3303 views)
Permalink
Re: File injection support [In reply to]

On Tue, Jun 12, 2012 at 8:20 AM, Pádraig Brady <P [at] draigbrady> wrote:
> On 06/12/2012 04:07 PM, Fredric Morenius wrote:
>> From: Scott Moser [mailto:ssmoser2 [at] gmail] On Behalf Of Scott Moser
>> Sent: den 11 juni 2012 23:16
>>
>>> ...Without digging around on older versions of OS's and their included kernel/udev/kpartx.... i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by >the kernel or some other plumbing bits automatically...
>>
>> I just now tested to remove the kpartx invocation from /nova/virt/disk/mount.py, like this:
>>
>> diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
>> index 11959b2..4d9527b 100644
>> --- a/nova/virt/disk/mount.py
>> +++ b/nova/virt/disk/mount.py
>> @@ -61,25 +61,9 @@ class Mount(object):
>> if self.partition == -1:
>> self.error = _('partition search unsupported with %s') % self.mode
>> elif self.partition:
>> - map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
>> - self.partition)
>> - assert(not os.path.exists(map_path))
>> -
>> - # Note kpartx can output warnings to stderr and succeed
>> - # Also it can output failures to stderr and "succeed"
>> - # So we just go on the existence of the mapped device
>> - _out, err = utils.trycmd('kpartx', '-a', self.device,
>> - run_as_root=True, discard_warnings=True)
>> -
>> - # Note kpartx does nothing when presented with a raw image,
>> - # so given we only use it when we expect a partitioned image, fail
>> - if not os.path.exists(map_path):
>> - if not err:
>> - err = _('partition %s not found') % self.partition
>> - self.error = _('Failed to map partitions: %s') % err
>> - else:
>> - self.mapped_device = map_path
>> - self.mapped = True
>> + #qemu-nbd already has mapped the partition for mounting
>> + self.mapped_device = '%sp%s' % (self.device, self.partition)
>> + self.mapped = True
>
> Looks good. That should also support raw through the /dev/loop%s devices.
>
> It might be safer to fallback to kpartx if not exists ...
> '%sp%s' % (self.device, self.partition)
> That would support kernels before 3.2
> I'd also add a comment in the code, that nbd must be mounted with param
> max_part=16 or whatever.


Might be totally unrelated to the above issues but I figured I'd throw
it out there. I was messing with file injection today and was getting
odd errors.

Some were due to hitting errors on a previous boot and partitions not
getting cleaned up (e.g. kpartx -d never ran and the /dev/mapper/nbd*
entries were still there, or qemu-nbd was still running). Some of
those were likely a result of my own circumstances.

More interesting though, and what might be of use to other people, the
"kpartx -a" calls get run and then the code in nova/virt/disk/mount.py
immediately checks for whether or not the newly created
/dev/mapper/nbdXXpX partitions exist. They'd actually get created but
the os.exists call would fail. Apparently the os.exists call was
getting run too soon. I added a time.sleep() after both the 'kpartx
-a' and 'kpartx -d' calls, to give things time to catch up before the
os.exists calls and things worked much better.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


P at draigBrady

Jul 13, 2012, 5:35 PM

Post #21 of 22 (3302 views)
Permalink
Re: File injection support [In reply to]

On 07/14/2012 12:09 AM, Mark Moseley wrote:
> On Tue, Jun 12, 2012 at 8:20 AM, Pádraig Brady <P [at] draigbrady> wrote:
>> On 06/12/2012 04:07 PM, Fredric Morenius wrote:
>>> From: Scott Moser [mailto:ssmoser2 [at] gmail] On Behalf Of Scott Moser
>>> Sent: den 11 juni 2012 23:16
>>>
>>>> ...Without digging around on older versions of OS's and their included kernel/udev/kpartx.... i'd have to say that from your message above, and my testing that running 'kpartx' is no longer necessary. That can reasonably be expected to be created by >the kernel or some other plumbing bits automatically...
>>>
>>> I just now tested to remove the kpartx invocation from /nova/virt/disk/mount.py, like this:
>>>
>>> diff --git a/nova/virt/disk/mount.py b/nova/virt/disk/mount.py
>>> index 11959b2..4d9527b 100644
>>> --- a/nova/virt/disk/mount.py
>>> +++ b/nova/virt/disk/mount.py
>>> @@ -61,25 +61,9 @@ class Mount(object):
>>> if self.partition == -1:
>>> self.error = _('partition search unsupported with %s') % self.mode
>>> elif self.partition:
>>> - map_path = '/dev/mapper/%sp%s' % (os.path.basename(self.device),
>>> - self.partition)
>>> - assert(not os.path.exists(map_path))
>>> -
>>> - # Note kpartx can output warnings to stderr and succeed
>>> - # Also it can output failures to stderr and "succeed"
>>> - # So we just go on the existence of the mapped device
>>> - _out, err = utils.trycmd('kpartx', '-a', self.device,
>>> - run_as_root=True, discard_warnings=True)
>>> -
>>> - # Note kpartx does nothing when presented with a raw image,
>>> - # so given we only use it when we expect a partitioned image, fail
>>> - if not os.path.exists(map_path):
>>> - if not err:
>>> - err = _('partition %s not found') % self.partition
>>> - self.error = _('Failed to map partitions: %s') % err
>>> - else:
>>> - self.mapped_device = map_path
>>> - self.mapped = True
>>> + #qemu-nbd already has mapped the partition for mounting
>>> + self.mapped_device = '%sp%s' % (self.device, self.partition)
>>> + self.mapped = True
>>
>> Looks good. That should also support raw through the /dev/loop%s devices.
>>
>> It might be safer to fallback to kpartx if not exists ...
>> '%sp%s' % (self.device, self.partition)
>> That would support kernels before 3.2
>> I'd also add a comment in the code, that nbd must be mounted with param
>> max_part=16 or whatever.
>
>
> Might be totally unrelated to the above issues but I figured I'd throw
> it out there. I was messing with file injection today and was getting
> odd errors.
>
> Some were due to hitting errors on a previous boot and partitions not
> getting cleaned up (e.g. kpartx -d never ran and the /dev/mapper/nbd*
> entries were still there, or qemu-nbd was still running). Some of
> those were likely a result of my own circumstances.
>
> More interesting though, and what might be of use to other people, the
> "kpartx -a" calls get run and then the code in nova/virt/disk/mount.py
> immediately checks for whether or not the newly created
> /dev/mapper/nbdXXpX partitions exist. They'd actually get created but
> the os.exists call would fail. Apparently the os.exists call was
> getting run too soon. I added a time.sleep() after both the 'kpartx
> -a' and 'kpartx -d' calls, to give things time to catch up before the
> os.exists calls and things worked much better.

Sigh.

The amount of synchronization bugs I've noticed in lower
Linux layers lately is worrying.

Anyway I've created a bug for this:
https://bugs.launchpad.net/nova/+bug/1024586

cheers,
Pádraig

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp


eric at cloudscaling

Jul 13, 2012, 5:56 PM

Post #22 of 22 (3320 views)
Permalink
Re: File injection support [In reply to]

On Friday, July 13, 2012 at 20:35 PM, Pádraig Brady wrote:

> >
> > More interesting though, and what might be of use to other people, the
> > "kpartx -a" calls get run and then the code in nova/virt/disk/mount.py
> > immediately checks for whether or not the newly created
> > /dev/mapper/nbdXXpX partitions exist. They'd actually get created but
> > the os.exists call would fail. Apparently the os.exists call was
> > getting run too soon. I added a time.sleep() after both the 'kpartx
> > -a' and 'kpartx -d' calls, to give things time to catch up before the
> > os.exists calls and things worked much better.
>
> Sigh.
>
> The amount of synchronization bugs I've noticed in lower
> Linux layers lately is worrying.
>


The kernel tools can certainly be asynchronous at times.

Perhaps try to see if using inotify would solve this. It is platform-dependent, but it is the best way to solve these problems without race conditions. If we're calling kpartx, platform independence is unlikely to be an issue anyway.

However, if compatibility is desired, BSD/MacOS provide similar functionality via kqueue, and Windows FindFirstChangeNotification.

Regards,
Eric Windisch


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.