Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux-HA: Pacemaker

Requesting re-sync

 

 

Linux-HA pacemaker RSS feed   Index | Next | Previous | View Threaded


pieter.baele at gmail

Feb 21, 2012, 6:57 AM

Post #1 of 4 (309 views)
Permalink
Requesting re-sync

After upgrading a node (RHEL 6.1 to 6.2), my /var/log/messages grows
really really fast
because of this error, what can be wrong?

Active node:
=========
Feb 21 15:49:42 node02 cib: [3225]: info: cib_process_request:
Operation complete: op cib_sync_one for section 'all'
(origin=node01.example.porg/node01.example.org/(null),
version=0.512.22415291): ok (rc=0)


Passive node:
==========
Feb 21 15:38:34 node01 cib: [3423]: info: cib_server_process_diff:
Requesting re-sync from peer
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273989 -> 0.512.22273990 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273990 -> 0.512.22273991 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273991 -> 0.512.22273992 (sync in progress)
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-drbd_admintools:1 (10000)
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 21 15:38:34 node01 cib: [3423]: info: cib_replace_notify:
Replaced: -1.-1.-1 -> 0.512.22273992 from node02.example.org
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-drbd_zabbixdb:1 (10000)
Feb 21 15:38:34 node01 cib: [3423]: info: apply_xml_diff: Digest
mis-match: expected 4a07823e1a80867b5da40b3202ef86e8, calculated
7f6df57158440164b7803b11aa30c669
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_process_diff: Diff
0.512.22273992 -> 0.512.22273993 not applied to 0.512.22273992: Failed
application of an update diff
Feb 21 15:38:34 node01 cib: [3423]: info: cib_server_process_diff:
Requesting re-sync from peer
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273993 -> 0.512.22273994 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273994 -> 0.512.22273995 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273995 -> 0.512.22273996 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: info: cib_replace_notify:
Replaced: -1.-1.-1 -> 0.512.22273996 from node02.example.org
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-drbd_admintools:1 (10000)
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-drbd_zabbixdb:1 (10000)
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-drbd_admintools:1 (10000)
Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 21 15:38:34 node01 cib: [3423]: info: apply_xml_diff: Digest
mis-match: expected 5a0dc6a500d941b6b7011adfae430c01, calculated
c220b688f445d0e7ec13be70196387bf
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_process_diff: Diff
0.512.22273996 -> 0.512.22273997 not applied to 0.512.22273996: Failed
application of an update diff
Feb 21 15:38:34 node01 cib: [3423]: info: cib_server_process_diff:
Requesting re-sync from peer
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273997 -> 0.512.22273998 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273998 -> 0.512.22273999 (sync in progress)
Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
Not applying diff 0.512.22273999 -> 0.512.22274000 (sync in progress)

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


florian at hastexo

Feb 21, 2012, 7:05 AM

Post #2 of 4 (301 views)
Permalink
Re: Requesting re-sync [In reply to]

On Tue, Feb 21, 2012 at 3:57 PM, Pieter Baele <pieter.baele [at] gmail> wrote:
> After upgrading a node (RHEL 6.1 to 6.2), my /var/log/messages grows
> really really fast
> because of this error, what can be wrong?

So you upgraded just one node, and the other is still unchanged? Can
you give the Pacemaker and Corosync version for both?

Cheers,
Florian

--
Need help with High Availability?
http://www.hastexo.com/now

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


pieter.baele at gmail

Feb 21, 2012, 9:13 AM

Post #3 of 4 (297 views)
Permalink
Re: Requesting re-sync [In reply to]

On Tue, Feb 21, 2012 at 16:05, Florian Haas <florian [at] hastexo> wrote:
> On Tue, Feb 21, 2012 at 3:57 PM, Pieter Baele <pieter.baele [at] gmail> wrote:
>> After upgrading a node (RHEL 6.1 to 6.2), my /var/log/messages grows
>> really really fast
>> because of this error, what can be wrong?
>
> So you upgraded just one node, and the other is still unchanged? Can
> you give the Pacemaker and Corosync version for both?
>

Pacemaker 1.1.6
Corosync 1.4.1

I also updated corosync and pacemaker on the node still running.
But I've obviously not restarted this node, as everything started
again correctly on this one.

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


andrew at beekhof

Feb 21, 2012, 1:43 PM

Post #4 of 4 (304 views)
Permalink
Re: Requesting re-sync [In reply to]

On Wed, Feb 22, 2012 at 1:57 AM, Pieter Baele <pieter.baele [at] gmail> wrote:
> After upgrading a node (RHEL 6.1 to 6.2), my /var/log/messages grows
> really really fast
> because of this error, what can be wrong?

I suspect you might be hitting:

https://github.com/ClusterLabs/pacemaker/commit/10e9e579ab032bde3938d7f3e13c414e297ba3e9

6.3 will have that fix.

In the meantime, perhaps try: turn on maintenance mode; shut down
pacemaker on both nodes; edit the nodes section; start pacemaker up
again.

You could also try the latest scratch build from
http://www.clusterlabs.org/rpm-test-rhel/rhel-6/clusterlabs.repo which
has the fix too.

>
> Active node:
> =========
> Feb 21 15:49:42 node02 cib: [3225]: info: cib_process_request:
> Operation complete: op cib_sync_one for section 'all'
> (origin=node01.example.porg/node01.example.org/(null),
> version=0.512.22415291): ok (rc=0)
>
>
> Passive node:
> ==========
> Feb 21 15:38:34 node01 cib: [3423]: info: cib_server_process_diff:
> Requesting re-sync from peer
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273989 -> 0.512.22273990 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273990 -> 0.512.22273991 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273991 -> 0.512.22273992 (sync in progress)
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: master-drbd_admintools:1 (10000)
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: probe_complete (true)
> Feb 21 15:38:34 node01 cib: [3423]: info: cib_replace_notify:
> Replaced: -1.-1.-1 -> 0.512.22273992 from node02.example.org
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: master-drbd_zabbixdb:1 (10000)
> Feb 21 15:38:34 node01 cib: [3423]: info: apply_xml_diff: Digest
> mis-match: expected 4a07823e1a80867b5da40b3202ef86e8, calculated
> 7f6df57158440164b7803b11aa30c669
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_process_diff: Diff
> 0.512.22273992 -> 0.512.22273993 not applied to 0.512.22273992: Failed
> application of an update diff
> Feb 21 15:38:34 node01 cib: [3423]: info: cib_server_process_diff:
> Requesting re-sync from peer
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273993 -> 0.512.22273994 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273994 -> 0.512.22273995 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273995 -> 0.512.22273996 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: info: cib_replace_notify:
> Replaced: -1.-1.-1 -> 0.512.22273996 from node02.example.org
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: master-drbd_admintools:1 (10000)
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: probe_complete (true)
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: master-drbd_zabbixdb:1 (10000)
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: master-drbd_admintools:1 (10000)
> Feb 21 15:38:34 node01 attrd: [3425]: notice: attrd_trigger_update:
> Sending flush op to all hosts for: probe_complete (true)
> Feb 21 15:38:34 node01 cib: [3423]: info: apply_xml_diff: Digest
> mis-match: expected 5a0dc6a500d941b6b7011adfae430c01, calculated
> c220b688f445d0e7ec13be70196387bf
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_process_diff: Diff
> 0.512.22273996 -> 0.512.22273997 not applied to 0.512.22273996: Failed
> application of an update diff
> Feb 21 15:38:34 node01 cib: [3423]: info: cib_server_process_diff:
> Requesting re-sync from peer
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273997 -> 0.512.22273998 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273998 -> 0.512.22273999 (sync in progress)
> Feb 21 15:38:34 node01 cib: [3423]: notice: cib_server_process_diff:
> Not applying diff 0.512.22273999 -> 0.512.22274000 (sync in progress)
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker [at] oss
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Linux-HA pacemaker RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.