jhom at softlayer
Jul 30, 2012, 7:36 AM
Views: 1471
Permalink
|
|
Re: [XCP-1.1] High OVS cpu load and unresponsive host network while VMPR archive phase is running
|
|
I've seen the same type of thing when VMs on tagged networks take on a lot of traffic. In my case the root cause appears to be ovs being unable to handle the amount of flow build up/tear downs. This causes old flows to perform somewhat ok, while new flows are erratic or don't work at all. This only affects vlan networks. Any VM on networks without a vlan tag(e.g. native) don't experience the issue.
I've been able to duplicate the issue all the way up to the latest ovs 1.6.1.
When this happens can you check to see if any VM on vlan networks are taking on an increased network load ( >100k pps)?
-----Original Message-----
From: xen-api-bounces [at] lists [mailto:xen-api-bounces [at] lists] On Behalf Of Christian Fischer
Sent: Saturday, July 28, 2012 4:26 PM
To: xen-api [at] lists
Subject: [Xen-API] [XCP-1.1] High OVS cpu load and unresponsive host network while VMPR archive phase is running
We notice high vswitch cpu load while the vm protection archive phase is running, which ends up in broken network connections and unresponsive pool servers. Any help to solve this problem is welcome.
XCP build: 1.1.0-50674c
OVS build: 1.4.2
NICs: BCM5709 Gigabit TOE iSCSI Offload
OVS NIC bonding: active/active
Pool Nodes: Dell R610
Storage type: LVMoiSCSI
The archive phase starts at 03.00AM, short time after that OVS logs poll_loop events and high CPU usage, after some hours (3-4) the whole host network becomes unresponsive, except the offloaded iSCSI connections to the NetAPP guest system image LUN (bnx2i cnic). We snapshot and archive only guest system images (mostly 8GB per image), data volumes are mounted directly by guest VMs (iSCSI).
We had running an XCP-1.0 pool on Intel Servers for the last two years with a lot of VLAN trunks, active/active bonds, cheep switches, self made DRBD- replicated storage, and OVS-1.0.1 IIRC. We've never seen such behavior.
Thanks
Christian
_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
_______________________________________________
Xen-api mailing list
Xen-api [at] lists
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
|
Re: [XCP-1.1] High OVS cpu load and unresponsive host network while VMPR archive phase is running
