Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Dev

[OpenStack][Nova] Live-Migration ---Libvirt related problem

 

 

OpenStack dev RSS feed   Index | Next | Previous | View Threaded


nextfool at gmail

May 27, 2012, 11:24 PM

Post #1 of 2 (1226 views)
Permalink
[OpenStack][Nova] Live-Migration ---Libvirt related problem

Hello ,all
first ,let me introduce my develop environment:
I have a computer with following lshw information

*-cpu
description: CPU
product: AMD FX(tm)-6100 Six-Core Processor
vendor: Hynix Semiconductor (Hyundai Electronics)
physical id: 4
bus info: cpu@0
version: AMD FX(tm)-6100 Six-Core Processor
slot: Socket M2
size: 1400MHz
capacity: 1400MHz
width: 64 bits
clock: 200MHz
*-memory
description: System Memory
physical id: 29
slot: System board or motherboard
size: 16GiB
Then ,I installed ubuntu1204 LTS .
Then,I started 3 VMs with virt-manager as my OpenStack testing cluster.I
named the 3VMs as* os0* *,os1* and* os2*.
I installed* cloud controller on os0* ,and *nova-compute on both os1 and os2
*.

I set up all the live-migration preparations instructed
here:
http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html


and here:
https://help.ubuntu.com/community/SettingUpNFSHowTo

BTW:I found some error in the document ,

Update the libvirt configurations. Modify /etc/libvirt/libvirt.conf:

before : #listen_tls = 0
after : listen_tls = 0

before : #listen_tcp = 1
after : listen_tcp = 1

add: auth_tcp = "none"

here ,we should update the configuration file
/etc/libvirt/*libvirtd*.conf ,not
/etc/libvirt/libvirt.conf

After all are set ,I just use the command "*nova live-migration vm1 os2*",here
the vm1 is an instance running currently on *os1* and I wanted it to be
migrated to* os2*

the command returned without any error or infor.I guess it indicated the
migration is done correctly .But I check the vm1 with "nova show vm1",it is
still running on *os1*!
I checked the nova-compte.log file on both os1 and os2,and found the
following live_migration related* *entries :


2012-05-28 10:26:39 DEBUG nova.rpc.amqp [-] received {u'_context_roles':
[u'admin'], u'_msg_id': u'89dfda33bb7b4dd29d93867aaf1460ec',
u'_context_read_deleted':
u'no', u'_context_request_id': u'req-f1415f26-f19e-4499-aa24-22e47b586378',
u'args': {u'instance_id': 15, u'block_migration': False, u'disk': None},
u'_context_
auth_token': '<SANITIZED>', u'_context_is_admin': True,
u'_context_project_id': u'7fa3accbd2b543a2a59c5249b8d07281',
u'_context_timestamp': u'2012-05-28T02:26:3
9.026426', u'_context_user_id': u'297ad61b9a67448186c08729ab0f1971',
u'method': *u'pre_live_migration*', u'_context_remote_address':
u'192.168.0.221'} from (pid=7
290) _safe_log /usr/lib/python2.7/dist-packages/nova/rpc/common.py:160


2012-05-28 11:22:09 DEBUG nova.service [-] *live_migration_retry_count *:
30 from (pid=16393) wait
/usr/lib/python2.7/dist-packages/nova/service.py:411

2012-05-28 11:22:09 DEBUG nova.service [-] *live_migration_uri *:
qemu+tcp://%s/system from (pid=16393) wait
/usr/lib/python2.7/dist-packages/nova/service.py:411

2012-05-28 11:22:09 DEBUG nova.service [-] *live_migration_bandwidth *: 0
from (pid=16393) wait /usr/lib/python2.7/dist-packages/nova/service.py:411


I didn't see any error in these log files.

Then I checked the* /var/log/libvirt/libvirtd.log* on both os1 and os2.and
found the following.

2012-05-28 02:56:14.310+0000: 9481: error : virExecWithHook:328 : Cannot
find 'pm-is-supported' in path: No such file or directory
2012-05-28 02:56:14.310+0000: 9481: warning : qemuCapsInit:856 : Failed to
get host power management capabilities
2012-05-28 02:56:14.514+0000: 9480: error : virExecWithHook:328 : Cannot
find 'pm-is-supported' in path: No such file or directory
2012-05-28 02:56:14.515+0000: 9480: warning : qemuCapsInit:856 : Failed to
get host power management capabilities
2012-05-28 02:56:14.719+0000: 9482: error : virExecWithHook:328 : Cannot
find 'pm-is-supported' in path: No such file or directory
2012-05-28 02:56:14.719+0000: 9482: warning : qemuCapsInit:856 : Failed to
get host power management capabilities
2012-05-28 02:56:14.943+0000: 9484: error : virExecWithHook:328 : Cannot
find 'pm-is-supported' in path: No such file or directory
2012-05-28 02:56:14.944+0000: 9484: warning : qemuCapsInit:856 : Failed to
get host power management capabilities
2012-05-28 02:56:15.155+0000: 9481: error : virExecWithHook:328 : Cannot
find 'pm-is-supported' in path: No such file or directory
2012-05-28 02:56:15.155+0000: 9481: warning : qemuCapsInit:856 : Failed to
get host power management capabilities
2012-05-28 02:58:55.330+0000: 13514: info : libvirt version: 0.9.8
2012-05-28 02:58:55.330+0000: 13514: error : qemuMonitorIORead:513 : Unable
to read from monitor: Connection reset by peer
2012-05-28 03:02:15.618+0000: 13518: error : qemuMonitorOpenUnix:295 :
failed to connect to monitor socket: No such process
2012-05-28 03:02:15.618+0000: 13518: error : qemuProcessWaitForMonitor:1301
: internal error process exited while connecting to monitor: char device re
to /dev/pts/3
inet_listen_opts: bind(ipv4,192.168.0.205,5900): Cannot assign requested
address
inet_listen_opts: FAILED

Please note the above ,the last two lines I marked as red ,I googled the
error ,and do found something maybe related here :
https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=703851

I know the inet_listen_opts: bind error maybe related to my VNC
settings in*/etc/nova/nova.conf
*,So I just commented all the vnc related lines in /etc/nova/nova.conf
Then I run the
"*nova live-migration vm1 os2*" again ,and "tail
-f /var/log/nova/nova-compute.log" to both os1 and os2.I found the the
folloing:

2012-05-28 03:21:02.700+0000: 8872: error : virNetSocketReadWire:996 : End
of file while reading data: Input/output error
2012-05-28 03:21:02.700+0000: 8872: error : virNetSocketReadWire:996 : End
of file while reading data: Input/output error
2012-05-28 03:23:06.551+0000: 8879: error : virNetDevGetIndex:656 : Unable
to get index for interface vnet0: No such device
2012-05-28 03:24:36.992+0000: 8875: error :
virNetClientProgramDispatchError:174 : Unable to read from monitor:
Connection reset by peer

note that the previous inet_listen_opts:bind error is not showing this time
,but I got something new as above.


Well ,these are all problems I met.And here is my question :
1.Are these libvirt related problems showing only because I am testing
them on a "virtual "environment? I mean ,maybe I should buy some more
physical computer?
2.If not ,what should I do with this ? Did anyone else get the same problem
as I did?.
Please help me ,thank you very much in advance!



Eric Luo
2012-05-28


razique.mahroua at gmail

May 28, 2012, 12:55 AM

Post #2 of 2 (1409 views)
Permalink
Re: [OpenStack][Nova] Live-Migration ---Libvirt related problem [In reply to]

Hey Eric,
does from the two nodes a simple :
$ virsh list --all works ?

also, have you enabled started the TCP socket by enabling it into
/etc/default/libvirt-bin

Before :
libvirtd_opts="-d"

After :
libvirtd_opts="-d -l"

Restart libvirtd and check the logs, by the way, you can safely ignore
the errors about power management or install the following package :
dbus

You can also validate a manual migration by issuing the following
command from the node 1 :
$ virsh migrate --live $domain (get by virsh list)
qemu+tcp://user [at] node/system

And check the output

Best regards,
Razique

> Eric Luo <mailto:nextfool [at] gmail>
> 28 mai 2012 08:24
> Hello ,all
> first ,let me introduce my develop environment:
> I have a computer with following lshw information
>
> *-cpu
> description: CPU
> product: AMD FX(tm)-6100 Six-Core Processor
> vendor: Hynix Semiconductor (Hyundai Electronics)
> physical id: 4
> bus info: cpu@0
> version: AMD FX(tm)-6100 Six-Core Processor
> slot: Socket M2
> size: 1400MHz
> capacity: 1400MHz
> width: 64 bits
> clock: 200MHz
> *-memory
> description: System Memory
> physical id: 29
> slot: System board or motherboard
> size: 16GiB
> Then ,I installed ubuntu1204 LTS .
> Then,I started 3 VMs with virt-manager as my OpenStack testing
> cluster.I named the 3VMs as*os0* *,os1* and*os2*.
> I installed*cloud controller on os0* ,and *nova-compute on both os1
> and os2*.
>
> I set up all the live-migration preparations instructed
> here:
> http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-live-migrations.html
>
>
> and here:
> https://help.ubuntu.com/community/SettingUpNFSHowTo
>
> BTW:I found some error in the document ,
>
> Update the libvirt configurations. Modify /etc/libvirt/libvirt.conf:
>
> before : #listen_tls = 0
> after : listen_tls = 0
>
> before : #listen_tcp = 1
> after : listen_tcp = 1
>
> add: auth_tcp = "none"
>
> here ,we should update the configuration file
> /etc/libvirt/*libvirtd*.conf ,not /etc/libvirt/libvirt.conf
>
> After all are set ,I just use the command "*nova live-migration vm1
> os2*",here the vm1 is an instance running currently on *os1* and I
> wanted it to be migrated to*os2*
>
> the command returned without any error or infor.I guess
> it indicated the migration is done correctly .But I check the vm1 with
> "nova show vm1",it is still running on *os1*!
> I checked the nova-compte.log file on both os1 and os2,and found the
> following live_migration related**entries :
>
>
> 2012-05-28 10:26:39 DEBUG nova.rpc.amqp [-] received
> {u'_context_roles': [u'admin'], u'_msg_id':
> u'89dfda33bb7b4dd29d93867aaf1460ec', u'_context_read_deleted':
> u'no', u'_context_request_id':
> u'req-f1415f26-f19e-4499-aa24-22e47b586378', u'args': {u'instance_id':
> 15, u'block_migration': False, u'disk': None}, u'_context_
> auth_token': '<SANITIZED>', u'_context_is_admin': True,
> u'_context_project_id': u'7fa3accbd2b543a2a59c5249b8d07281',
> u'_context_timestamp': u'2012-05-28T02:26:3
> 9.026426', u'_context_user_id': u'297ad61b9a67448186c08729ab0f1971',
> u'method': *u'pre_live_migration*', u'_context_remote_address':
> u'192.168.0.221'} from (pid=7
> 290) _safe_log /usr/lib/python2.7/dist-packages/nova/rpc/common.py:160
>
>
> 2012-05-28 11:22:09 DEBUG nova.service [-] *live_migration_retry_count
> *: 30 from (pid=16393) wait
> /usr/lib/python2.7/dist-packages/nova/service.py:411
>
> 2012-05-28 11:22:09 DEBUG nova.service [-] *live_migration_uri *:
> qemu+tcp://%s/system from (pid=16393) wait
> /usr/lib/python2.7/dist-packages/nova/service.py:411
>
> 2012-05-28 11:22:09 DEBUG nova.service [-] *live_migration_bandwidth
> *: 0 from (pid=16393) wait
> /usr/lib/python2.7/dist-packages/nova/service.py:411
>
>
> I didn't see any error in these log files.
>
> Then I checked the* /var/log/libvirt/libvirtd.log* on both os1 and
> os2.and found the following.
>
> 2012-05-28 02:56:14.310+0000: 9481: error : virExecWithHook:328 :
> Cannot find 'pm-is-supported' in path: No such file or directory
> 2012-05-28 02:56:14.310+0000: 9481: warning : qemuCapsInit:856 :
> Failed to get host power management capabilities
> 2012-05-28 02:56:14.514+0000: 9480: error : virExecWithHook:328 :
> Cannot find 'pm-is-supported' in path: No such file or directory
> 2012-05-28 02:56:14.515+0000: 9480: warning : qemuCapsInit:856 :
> Failed to get host power management capabilities
> 2012-05-28 02:56:14.719+0000: 9482: error : virExecWithHook:328 :
> Cannot find 'pm-is-supported' in path: No such file or directory
> 2012-05-28 02:56:14.719+0000: 9482: warning : qemuCapsInit:856 :
> Failed to get host power management capabilities
> 2012-05-28 02:56:14.943+0000: 9484: error : virExecWithHook:328 :
> Cannot find 'pm-is-supported' in path: No such file or directory
> 2012-05-28 02:56:14.944+0000: 9484: warning : qemuCapsInit:856 :
> Failed to get host power management capabilities
> 2012-05-28 02:56:15.155+0000: 9481: error : virExecWithHook:328 :
> Cannot find 'pm-is-supported' in path: No such file or directory
> 2012-05-28 02:56:15.155+0000: 9481: warning : qemuCapsInit:856 :
> Failed to get host power management capabilities
> 2012-05-28 02:58:55.330+0000: 13514: info : libvirt version: 0.9.8
> 2012-05-28 02:58:55.330+0000: 13514: error : qemuMonitorIORead:513 :
> Unable to read from monitor: Connection reset by peer
> 2012-05-28 03:02:15.618+0000: 13518: error : qemuMonitorOpenUnix:295 :
> failed to connect to monitor socket: No such process
> 2012-05-28 03:02:15.618+0000: 13518: error :
> qemuProcessWaitForMonitor:1301 : internal error process exited while
> connecting to monitor: char device re
> to /dev/pts/3
> inet_listen_opts: bind(ipv4,192.168.0.205,5900): Cannot assign
> requested address
> inet_listen_opts: FAILED
>
> Please note the above ,the last two lines I marked as red ,I googled
> the error ,and do found something maybe related here :
> https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=703851
> <https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=703851>
>
> I know the inet_listen_opts: bind error maybe related to my VNC
> settings in*/etc/nova/nova.conf*,So I just commented all the vnc
> related lines in /etc/nova/nova.conf
> Then I run the
> "*nova live-migration vm1 os2*" again ,and "tail
> -f /var/log/nova/nova-compute.log" to both os1 and os2.I found the the
> folloing:
>
> 2012-05-28 03:21:02.700+0000: 8872: error : virNetSocketReadWire:996 :
> End of file while reading data: Input/output error
> 2012-05-28 03:21:02.700+0000: 8872: error : virNetSocketReadWire:996 :
> End of file while reading data: Input/output error
> 2012-05-28 03:23:06.551+0000: 8879: error : virNetDevGetIndex:656 :
> Unable to get index for interface vnet0: No such device
> 2012-05-28 03:24:36.992+0000: 8875: error :
> virNetClientProgramDispatchError:174 : Unable to read from monitor:
> Connection reset by peer
>
> note that the previous inet_listen_opts:bind error is not showing this
> time ,but I got something new as above.
>
>
> Well ,these are all problems I met.And here is my question :
> 1.Are these libvirt related problems showing only because I am
> testing them on a "virtual "environment? I mean ,maybe I should buy
> some more physical computer?
> 2.If not ,what should I do with this ? Did anyone else get the same
> problem as I did?.
> Please help me ,thank you very much in advance!
>
>
>
> Eric Luo
> 2012-05-28
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack [at] lists
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp

--
Nuage & Co - Razique Mahroua
razique.mahroua [at] gmail
Attachments: compose-unknown-contact.jpg (0.75 KB)
  image.jpg (9.88 KB)

OpenStack dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.