Mailing List Archive: OpenStack: Dev

[OpenStack][Nova] Live Migration + NFSv4 - Permission issues

Index | Next | Previous | View Threaded

leanderbb at gmail

Jul 3, 2012, 6:48 AM

Post #1 of 12 (256 views)
Permalink

[OpenStack][Nova] Live Migration + NFSv4 - Permission issues

Hello all,

I've been trying to get the live migration to work according to the guide
http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html.

So far i've setup 2 compute nodes and 1 controller node. They all share the
/var/lib/nova/instances dir. I've already verified that the nova user id is
the same across all the servers.

Currently i'm running into this error when i launch an instance:
http://paste.openstack.org/show/19221/

It's certainly a permission issue, so i tried adding the group "nova" to
the user "libvirt-qemu". However, it still doesn't work. To which user must
i give the nova group permission in order to be able to write in that
directory?

Regards,
Leander

han.sebastien at gmail

Jul 3, 2012, 8:23 AM

Post #2 of 12 (234 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Which permissions did you set on /var/lib/nova/instances?

On Tue, Jul 3, 2012 at 3:48 PM, Leander Bessa Beernaert <leanderbb [at] gmail
> wrote:

> Hello all,
>
> I've been trying to get the live migration to work according to the guide
> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html.
>
> So far i've setup 2 compute nodes and 1 controller node. They all share
> the /var/lib/nova/instances dir. I've already verified that the nova user
> id is the same across all the servers.
>
> Currently i'm running into this error when i launch an instance:
> http://paste.openstack.org/show/19221/
>
> It's certainly a permission issue, so i tried adding the group "nova" to
> the user "libvirt-qemu". However, it still doesn't work. To which user must
> i give the nova group permission in order to be able to write in that
> directory?
>
> Regards,
> Leander
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>

leanderbb at gmail

Jul 3, 2012, 8:25 AM

Post #3 of 12 (235 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Currently it's using the default permission. Everything belongs to user
"nova" and the group "nova".

On Tue, Jul 3, 2012 at 4:23 PM, Sébastien Han <han.sebastien [at] gmail>wrote:

> Which permissions did you set on /var/lib/nova/instances?
>
>
> On Tue, Jul 3, 2012 at 3:48 PM, Leander Bessa Beernaert <
> leanderbb [at] gmail> wrote:
>
>> Hello all,
>>
>> I've been trying to get the live migration to work according to the guide
>> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html.
>>
>> So far i've setup 2 compute nodes and 1 controller node. They all share
>> the /var/lib/nova/instances dir. I've already verified that the nova user
>> id is the same across all the servers.
>>
>> Currently i'm running into this error when i launch an instance:
>> http://paste.openstack.org/show/19221/
>>
>> It's certainly a permission issue, so i tried adding the group "nova" to
>> the user "libvirt-qemu". However, it still doesn't work. To which user must
>> i give the nova group permission in order to be able to write in that
>> directory?
>>
>> Regards,
>> Leander
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>

leanderbb at gmail

Jul 3, 2012, 8:30 AM

Post #4 of 12 (234 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Here's an output from ls -l:
drwxr-xr-x 3 nova nova 4096 Jul 3 14:10 instances

On Tue, Jul 3, 2012 at 4:25 PM, Leander Bessa Beernaert <leanderbb [at] gmail
> wrote:

> Currently it's using the default permission. Everything belongs to user
> "nova" and the group "nova".
>
>
> On Tue, Jul 3, 2012 at 4:23 PM, Sébastien Han <han.sebastien [at] gmail>wrote:
>
>> Which permissions did you set on /var/lib/nova/instances?
>>
>>
>> On Tue, Jul 3, 2012 at 3:48 PM, Leander Bessa Beernaert <
>> leanderbb [at] gmail> wrote:
>>
>>> Hello all,
>>>
>>> I've been trying to get the live migration to work according to the
>>> guide
>>> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html.
>>>
>>> So far i've setup 2 compute nodes and 1 controller node. They all share
>>> the /var/lib/nova/instances dir. I've already verified that the nova user
>>> id is the same across all the servers.
>>>
>>> Currently i'm running into this error when i launch an instance:
>>> http://paste.openstack.org/show/19221/
>>>
>>> It's certainly a permission issue, so i tried adding the group "nova" to
>>> the user "libvirt-qemu". However, it still doesn't work. To which user must
>>> i give the nova group permission in order to be able to write in that
>>> directory?
>>>
>>> Regards,
>>> Leander
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack [at] lists
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>

m at mjvn

Jul 3, 2012, 8:46 AM

Post #5 of 12 (235 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Have you tried setting the ownership of /var/lib/nova/instances to the
nova user?

sudo chown -R nova:nova /var/lib/nova/instances

M

On 03/07/2012 15:48, Leander Bessa Beernaert wrote:
> Hello all,
>
> I've been trying to get the live migration to work according to the
> guide
> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html
> .
>
> So far i've setup 2 compute nodes and 1 controller node. They all
> share the /var/lib/nova/instances dir. I've already verified that the
> nova user id is the same across all the servers.
>
> Currently i'm running into this error when i launch an instance:
> http://paste.openstack.org/show/19221/
>
> It's certainly a permission issue, so i tried adding the group "nova"
> to the user "libvirt-qemu". However, it still doesn't work. To which
> user must i give the nova group permission in order to be able to
> write in that directory?
>
> Regards,
> Leander
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

leanderbb at gmail

Jul 3, 2012, 8:51 AM

Post #6 of 12 (235 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Still the same problem :S

On Tue, Jul 3, 2012 at 4:46 PM, Marnus van Niekerk <m [at] mjvn> wrote:

> Have you tried setting the ownership of /var/lib/nova/instances to the
> nova user?
>
> sudo chown -R nova:nova /var/lib/nova/instances
>
> M
>
>
> On 03/07/2012 15:48, Leander Bessa Beernaert wrote:
>
>> Hello all,
>>
>> I've been trying to get the live migration to work according to the guide
>> http://docs.openstack.org/**trunk/openstack-compute/admin/**
>> content/configuring-live-**migrations.html<http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html>.
>>
>> So far i've setup 2 compute nodes and 1 controller node. They all share
>> the /var/lib/nova/instances dir. I've already verified that the nova user
>> id is the same across all the servers.
>>
>> Currently i'm running into this error when i launch an instance:
>> http://paste.openstack.org/**show/19221/<http://paste.openstack.org/show/19221/>
>>
>> It's certainly a permission issue, so i tried adding the group "nova" to
>> the user "libvirt-qemu". However, it still doesn't work. To which user must
>> i give the nova group permission in order to be able to write in that
>> directory?
>>
>> Regards,
>> Leander
>>
>>
>> ______________________________**_________________
>> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
>> Post to : openstack [at] lists
>> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
>> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>>
>
>
>

leanderbb at gmail

Jul 4, 2012, 8:00 AM

Post #7 of 12 (234 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Shouldn't libvirt have write permission in a nova directory, since it's
executed by root?

On Tue, Jul 3, 2012 at 4:51 PM, Leander Bessa Beernaert <leanderbb [at] gmail
> wrote:

> Still the same problem :S
>
>
> On Tue, Jul 3, 2012 at 4:46 PM, Marnus van Niekerk <m [at] mjvn> wrote:
>
>> Have you tried setting the ownership of /var/lib/nova/instances to the
>> nova user?
>>
>> sudo chown -R nova:nova /var/lib/nova/instances
>>
>> M
>>
>>
>> On 03/07/2012 15:48, Leander Bessa Beernaert wrote:
>>
>>> Hello all,
>>>
>>> I've been trying to get the live migration to work according to the
>>> guide http://docs.openstack.org/**trunk/openstack-compute/admin/**
>>> content/configuring-live-**migrations.html<http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html>.
>>>
>>> So far i've setup 2 compute nodes and 1 controller node. They all share
>>> the /var/lib/nova/instances dir. I've already verified that the nova user
>>> id is the same across all the servers.
>>>
>>> Currently i'm running into this error when i launch an instance:
>>> http://paste.openstack.org/**show/19221/<http://paste.openstack.org/show/19221/>
>>>
>>> It's certainly a permission issue, so i tried adding the group "nova" to
>>> the user "libvirt-qemu". However, it still doesn't work. To which user must
>>> i give the nova group permission in order to be able to write in that
>>> directory?
>>>
>>> Regards,
>>> Leander
>>>
>>>
>>> ______________________________**_________________
>>> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
>>> Post to : openstack [at] lists
>>> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
>>> More help : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>>>
>>
>>
>>
>

berrange at redhat

Jul 4, 2012, 8:18 AM

Post #8 of 12 (244 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

On Tue, Jul 03, 2012 at 02:48:05PM +0100, Leander Bessa Beernaert wrote:
> Hello all,
>
> I've been trying to get the live migration to work according to the guide
> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html.
>
> So far i've setup 2 compute nodes and 1 controller node. They all share the
> /var/lib/nova/instances dir. I've already verified that the nova user id is
> the same across all the servers.
>
> Currently i'm running into this error when i launch an instance:
> http://paste.openstack.org/show/19221/
>
> It's certainly a permission issue, so i tried adding the group "nova" to
> the user "libvirt-qemu". However, it still doesn't work. To which user must
> i give the nova group permission in order to be able to write in that
> directory?

So the error message you get here:

libvirtError: Unable to pre-create chardev file '/var/lib/nova/instances/instance-00000004/console.log': Permission denied

is coming from the libvirt QEMU driver code. This code runs as root:root
in the libvirtd daemon. As such, ordinarily permissions would not be a
problem that blocks libvirtd doing what it wants. NFS is horrible /
special though, and has this capability "root squash" which causes any
client host processes running as root to be treated as if they were
running as 'nfsnobody' or a similar unprivileged user. So if you want
libvirtd to be able to use this mount point you'll need to make sure
that your NFS server has root squash disabled. eg by adding the option
'no_root_squash' to the /etc/exports file for that mount.

Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

leanderbb at gmail

Jul 4, 2012, 8:24 AM

Post #9 of 12 (240 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Ok. so i have added the option you mentioned, restarted nfs and remounted
on the compute nodes.

Now i get this error: http://paste.openstack.org/show/19260/

:S

On Wed, Jul 4, 2012 at 4:18 PM, Daniel P. Berrange <berrange [at] redhat>wrote:

> On Tue, Jul 03, 2012 at 02:48:05PM +0100, Leander Bessa Beernaert wrote:
> > Hello all,
> >
> > I've been trying to get the live migration to work according to the guide
> >
> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html
> .
> >
> > So far i've setup 2 compute nodes and 1 controller node. They all share
> the
> > /var/lib/nova/instances dir. I've already verified that the nova user id
> is
> > the same across all the servers.
> >
> > Currently i'm running into this error when i launch an instance:
> > http://paste.openstack.org/show/19221/
> >
> > It's certainly a permission issue, so i tried adding the group "nova" to
> > the user "libvirt-qemu". However, it still doesn't work. To which user
> must
> > i give the nova group permission in order to be able to write in that
> > directory?
>
> So the error message you get here:
>
> libvirtError: Unable to pre-create chardev file
> '/var/lib/nova/instances/instance-00000004/console.log': Permission denied
>
> is coming from the libvirt QEMU driver code. This code runs as root:root
> in the libvirtd daemon. As such, ordinarily permissions would not be a
> problem that blocks libvirtd doing what it wants. NFS is horrible /
> special though, and has this capability "root squash" which causes any
> client host processes running as root to be treated as if they were
> running as 'nfsnobody' or a similar unprivileged user. So if you want
> libvirtd to be able to use this mount point you'll need to make sure
> that your NFS server has root squash disabled. eg by adding the option
> 'no_root_squash' to the /etc/exports file for that mount.
>
>
> Daniel
> --
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/:|
> |: http://libvirt.org -o- http://virt-manager.org:|
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/:|
> |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc:|
>

berrange at redhat

Jul 4, 2012, 8:35 AM

Post #10 of 12 (235 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

On Wed, Jul 04, 2012 at 04:24:42PM +0100, Leander Bessa Beernaert wrote:
> Ok. so i have added the option you mentioned, restarted nfs and remounted
> on the compute nodes.
>
> Now i get this error: http://paste.openstack.org/show/19260/

libvirtError: internal error Process exited while reading console log output: chardev: opening backend "file" failed: Permission denied

This is an error from QEMU itself. So it shows libvirtd has successfully
done its part of the work. Unless you have changed your qemu.conf file,
then QEMU will normally run as the 'qemu:qemu' user/group pair. So you
should make sure that user account has r-x permission on the directory
where the images are stored. QEMU doesn't need write permission on
the directory - libvirtd makes sure the files themselves are writable

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack [at] lists
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp

leanderbb at gmail

Jul 4, 2012, 8:49 AM

Post #11 of 12 (233 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

Can't seem to get around it >.<. Below are the current permissions on the
folder in one of the compute nodes.
I've already tried chmod o+x /var/lib/nova/instances/ but nothing happend :/

gsd [at] cloud3:~$ ls -l /var/lib/nova/instances/
> total 4
> drwxrwxr-x 2 nova nova 4096 Jul 3 13:33 _base
> gsd [at] cloud3:~$ ls -l /var/lib/nova/
> total 136
> drwxr-xr-x 6 nova nova 4096 Jul 3 13:11 CA
> drwxr-xr-x 2 nova nova 4096 Jun 25 20:11 buckets
> drwxr-xr-x 2 nova nova 4096 Jun 25 20:11 images
> drwxr-xr-x 3 nova nova 4096 Jul 4 15:45 instances
> drwxr-xr-x 2 nova nova 4096 Jun 25 20:11 keys
> drwxr-xr-x 2 nova nova 4096 Jun 25 20:11 networks
> -rw------- 1 nova nova 104448 Jul 3 13:13 nova.sqlite
> drwxr-xr-x 2 nova nova 4096 Jun 25 20:11 tmp

On Wed, Jul 4, 2012 at 4:35 PM, Daniel P. Berrange <berrange [at] redhat>wrote:

> On Wed, Jul 04, 2012 at 04:24:42PM +0100, Leander Bessa Beernaert wrote:
> > Ok. so i have added the option you mentioned, restarted nfs and remounted
> > on the compute nodes.
> >
> > Now i get this error: http://paste.openstack.org/show/19260/
>
> libvirtError: internal error Process exited while reading console log
> output: chardev: opening backend "file" failed: Permission denied
>
>
> This is an error from QEMU itself. So it shows libvirtd has successfully
> done its part of the work. Unless you have changed your qemu.conf file,
> then QEMU will normally run as the 'qemu:qemu' user/group pair. So you
> should make sure that user account has r-x permission on the directory
> where the images are stored. QEMU doesn't need write permission on
> the directory - libvirtd makes sure the files themselves are writable
>
> Regards,
> Daniel
> --
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/:|
> |: http://libvirt.org -o- http://virt-manager.org:|
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/:|
> |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc:|
>

mandarvaze at gmail

Jul 4, 2012, 9:13 PM

Post #12 of 12 (242 views)
Permalink

Re: [OpenStack][Nova] Live Migration + NFSv4 - Permission issues [In reply to]

See if mounting as nfsv3 helps
See https://answers.launchpad.net/nova/+question/164689,
specifically comment #11

-Mandar

On Tue, Jul 3, 2012 at 7:18 PM, Leander Bessa Beernaert <leanderbb [at] gmail
> wrote:

> Hello all,
>
> I've been trying to get the live migration to work according to the guide
> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html.
>
> So far i've setup 2 compute nodes and 1 controller node. They all share
> the /var/lib/nova/instances dir. I've already verified that the nova user
> id is the same across all the servers.
>
> Currently i'm running into this error when i launch an instance:
> http://paste.openstack.org/show/19221/
>
> It's certainly a permission issue, so i tried adding the group "nova" to
> the user "libvirt-qemu". However, it still doesn't work. To which user must
> i give the nova group permission in order to be able to write in that
> directory?
>
> Regards,
> Leander
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads