Mailing List Archive: Linux-HA: Users

need help

Index | Next | Previous | View Threaded

pmdv at comclark

Apr 16, 2008, 8:29 AM

Post #1 of 5 (911 views)
Permalink

need help

Good day to all,

I'm new here, i need some help regarding HA for a linux. Do you have any howto article for this? I'm using fedora, freebsd and ubuntu

Your help is much appreciated.

---------------------------------------------------
Linux Registered User # 413558
philip [at] comclark (philip at comclark dot com)
_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

michalko.system at a-i-p

Apr 17, 2008, 4:18 AM

Post #2 of 5 (831 views)
Permalink

Re: need help [In reply to]

Am Mittwoch, 16. April 2008 17:29 schrieb Philip Michael D Vargas:
> Good day to all,
>
> I'm new here, i need some help regarding HA for a linux. Do you have any
> howto article for this? I'm using fedora, freebsd and ubuntu
>
>
> Your help is much appreciated.
>
> ---------------------------------------------------
> Linux Registered User # 413558
> philip [at] comclark (philip at comclark dot com)
> _______________________________________________
> Linux-HA mailing list
> Linux-HA [at] lists
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
Hi, Philip,

you can start at http://wiki.linux-ha.org/ and also at
http://www.linux-ha.org//HeartbeatTutorials

HTH

Regards
--

Nikita Michalko
_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

timothy.carr at foxtrail

Oct 1, 2009, 5:52 AM

Post #3 of 5 (673 views)
Permalink

Re: NEED HELP [In reply to]

Hi Richard,

I assume that your location constraints are set correctly ? or at least
cleared if you are not making use of them.
Send us your configuration file
Tim

On Thu, Oct 1, 2009 at 2:43 PM, Marshall, Richard <
Richard.Marshall [at] arbella> wrote:

> Hello:
>
> I need help trying to debug my ha-log file. Yesterday we have an
> application configured in our HA cluster, several weeks ago it failed,
> then again yesterday it failed, but this time it would not failover to
> one of the other nodes in our (3) host cluster.
>
> If someone could take a look at the attached log, comment on why the
> resource failed, and why it also did not move to an alternate node I
> would be grateful.
>
> Thank You
>
> _________________________________________________
> Richard Marshall | Senior Technical Specialist | Arbella Insurance Group
>
>
> 1900 Crown Colony Drive | Quincy, MA 02269 | ': 617.328.2921| 7:
> 617.515.2491 | *: Richard.Marshall [at] Arbella
>
>
> This email message is intended only for the addressee(s) and contains
> information that may be confidential.
> If you are not the intended recipient please notify the sender by reply
> email and immediately delete this message.
> Use, disclosure or reproduction of this email by anyone other than the
> intended recipient(s) is strictly prohibited.
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA [at] lists
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>

--
Timothy Carr
Technical Specialist
University of Cape Town
Cell: +27834572568
Fax: +27865472190
Gtalk: timothy.carr [at] foxtrail
Skype: timothy.carr.foxtrail
Sent from Cape Town, Western Cape, South Africa
_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

dejanmm at fastmail

Oct 1, 2009, 7:48 AM

Post #4 of 5 (682 views)
Permalink

Re: NEED HELP [In reply to]

Hi,

On Thu, Oct 01, 2009 at 08:43:14AM -0400, Marshall, Richard wrote:
> Hello:
>
> I need help trying to debug my ha-log file. Yesterday we have an
> application configured in our HA cluster, several weeks ago it failed,
> then again yesterday it failed, but this time it would not failover to
> one of the other nodes in our (3) host cluster.
>
> If someone could take a look at the attached log, comment on why the
> resource failed, and why it also did not move to an alternate node I
> would be grateful.

One node log is not enough. You have to take a look at the log on
the node where you think it should've started, but it couldn't.
From this log, I can see that one oracle resource failed
repeatedly on lxp005, but since there are no logs from that host,
one can't say why. Hint: If you wonder why a resource failed
search for lrmd:.*<rsc>. The best way is to make a hb_report of
the incident, unpack it, and then take a look at the log
excerpts.

Thanks,

Dejan

> Thank You
>
> _________________________________________________
> Richard Marshall | Senior Technical Specialist | Arbella Insurance Group
>
>
> 1900 Crown Colony Drive | Quincy, MA 02269 | ': 617.328.2921| 7:
> 617.515.2491 | *: Richard.Marshall [at] Arbella
>
>
> This email message is intended only for the addressee(s) and contains information that may be confidential.
> If you are not the intended recipient please notify the sender by reply email and immediately delete this message.
> Use, disclosure or reproduction of this email by anyone other than the intended recipient(s) is strictly prohibited.

> From: richardma [richardma [at] lxpts1]
> Sent: Thursday, October 01, 2009 7:44 AM
> To: Marshall, Richard
> Subject: log
>
> Sep 30 13:27:04 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation oracle_plcyp0_status_90000 (call=506, rc=7) complete
> Sep 30 13:27:04 lxp003 tengine: [19672]: info: process_graph_event: Detected action oracle_plcyp0_status_90000 from a different transition: 3879 vs. 5371
> Sep 30 13:27:04 lxp003 tengine: [19672]: info: update_abort_priority: Abort priority upgraded to 1000000
> Sep 30 13:27:04 lxp003 tengine: [19672]: WARN: update_failcount: Updating failcount for oracle_plcyp0 on c0f812ae-091e-4e58-94a5-3fcaf189a818 after failed status: rc=7
> Sep 30 13:27:04 lxp003 crmd: [3237]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:27:04 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources.
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: determine_online_status: Node lxp005 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: extract_event: Aborting on transient_attributes changes for c0f812ae-091e-4e58-94a5-3fcaf189a818
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 tengine: [19672]: WARN: notify_crmd: Delaying completion until all CIB updates complete
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: determine_online_status: Node lxp003 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_status_90000 onlxp003: Error
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: determine_online_status: Node lxp004 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: group_print: Resource Group: grp_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: native_print: IPaddr2_lxp0plcy (ocf::heartbeat:IPaddr2): Started lxp003
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: native_print: VG_lxp0plcy (ocf::arbella:VG): Started lxp003
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: native_print: oracle_plcyp0 (ocf::arbella:oracle): Started lxp003 FAILED
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource IPaddr2_lxp0plcy (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource VG_lxp0plcy (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Recover resource oracle_plcyp0 (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003 Stop oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp003 Start oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp003 oracle_plcyp0_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource tws_lxp0plcy (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message: Transition 5372: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5162.bz2
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: determine_online_status: Node lxp003 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp003: -1000000
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_status_90000 onlxp003: Error
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource IPaddr2_lxp0plcy (lxp003 -> lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003 Stop IPaddr2_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005 Start IPaddr2_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005 IPaddr2_lxp0plcy_status_30000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource VG_lxp0plcy (lxp003 -> lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003 Stop VG_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005 Start VG_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005 VG_lxp0plcy_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Recover resource oracle_plcyp0 (lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003 Stop oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005 Start VG_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005 VG_lxp0plcy_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Recover resource oracle_plcyp0 (lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003 Stop oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005 Start oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005 oracle_plcyp0_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource tws_lxp0plcy (lxp003 -> lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003 Stop tws_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005 Start tws_lxp0plcy
> Sep 30 13:27:05 lxp003 crmd: [3237]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: unpack_graph: Unpacked transition 5373: 17 actions in 17 synapses
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 256 fired and confirmed
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 251: tws_lxp0plcy_stop_0 on lxp003
> Sep 30 13:27:05 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=tws_lxp0plcy_stop_0 key=251:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:27:05 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: stop
> Sep 30 13:27:05 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation tws_lxp0plcy_status_90000 (call=508,rc=-2) Cancelled
> Sep 30 13:27:05 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stdout) usage: /usr/lib/ocf/resource.d//arbella/tws {start|stop|monitor|validate-all|meta-data|status} Expects to have a fully populated OCF RA-compliant environment set.
> Sep 30 13:27:05 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message: Transition 5373: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5163.bz2
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stderr) TWS for UNIX/CONMAN 8.2.1 (1.36.4.5) Licensed Materials Property of IBM 5698-WKB (C) Copyright IBM Corp 1998, 2005 US Government User Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Installed for user 'lxp0plcy'.
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stderr) Locale LANG set to "en_US.UTF-8"
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stderr) Schedule (Exp) 09/30/09 (#-4687) on LXP0PLCY. Batchman LIVES. Limit: 0, Fence: 0, Audit Level: 0
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stderr) %unlink @;noask
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stderr) AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager:A0V0. This is not allowed.
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:stop:stderr) AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager:APPP40. This is not allowed.
> Sep 30 13:28:16 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation tws_lxp0plcy_stop_0 (call=549, rc=0)complete
> Sep 30 13:28:16 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_stop_0 (251) confirmed on lxp003 (rc=0)
> Sep 30 13:28:16 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 60: oracle_plcyp0_stop_0 on lxp003
> Sep 30 13:28:16 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=oracle_plcyp0_stop_0 key=60:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:28:16 lxp003 lrmd: [3234]: info: rsc:oracle_plcyp0: stop
> Sep 30 13:28:16 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation oracle_plcyp0_status_90000 (call=506, rc=-2) Cancelled
> Sep 30 13:28:58 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation oracle_plcyp0_stop_0 (call=550, rc=0) complete
> Sep 30 13:28:58 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_stop_0 (60) confirmed on lxp003 (rc=0)
> Sep 30 13:28:58 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 246: VG_lxp0plcy_stop_0 on lxp003
> Sep 30 13:28:58 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=VG_lxp0plcy_stop_0 key=246:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:28:58 lxp003 lrmd: [3234]: info: rsc:VG_lxp0plcy: stop
> Sep 30 13:28:58 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation VG_lxp0plcy_status_90000 (call=504, rc=-2) Cancelled
> Sep 30 13:29:13 lxp003 lrmd: [3234]: WARN: G_SIG_dispatch: Dispatch function for SIGCHLD was delayed 1000 ms (> 100 ms) before being called (GSource: 0x51afa8)
> Sep 30 13:29:13 lxp003 lrmd: [3234]: info: G_SIG_dispatch: started at 3314398378 should have started at 3314398278
> Sep 30 13:29:16 lxp003 kernel: hde: 709684kB, 32917/4/129 CHS, 6175 kBps, 65535 sector size, 65535 rpm
> Sep 30 13:29:16 lxp003 kernel: klogd 1.4.1, ---------- state change ----------
> Sep 30 13:29:21 lxp003 su: (to oracle) root on none
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:stop:stderr) File descriptor 4 left open Filedescriptor 5 left open File descriptor 6 left open File descriptor 7 left open File descriptor 8 left open File descriptor 9 left open
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:stop:stderr)
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:stop:stderr) Using volume group(s) on commandline
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:stop:stderr) Finding volume group "plcyp0vg"
> Sep 30 13:29:30 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_stop_0 (246) confirmed on lxp003 (rc=0)
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 243: IPaddr2_lxp0plcy_stop_0 on lxp003
> Sep 30 13:29:33 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=IPaddr2_lxp0plcy_stop_0 key=243:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:29:33 lxp003 lrmd: [3234]: info: rsc:IPaddr2_lxp0plcy: stop
> Sep 30 13:29:33 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation IPaddr2_lxp0plcy_status_30000 (call=502, rc=-2) Cancelled
> Sep 30 13:29:33 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation IPaddr2_lxp0plcy_stop_0 (call=552, rc=0) complete
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_stop_0 (243) confirmedon lxp003 (rc=0)
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 257 fired and confirmed
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 78 fired and confirmed
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 254 fired and confirmed
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 244: IPaddr2_lxp0plcy_start_0on lxp005
> Sep 30 13:29:34 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_start_0 (244) confirmed on lxp005 (rc=0)
> Sep 30 13:29:34 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 245: IPaddr2_lxp0plcy_status_30000 on lxp005
> Sep 30 13:29:34 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 247: VG_lxp0plcy_start_0 on
>
> Sep 30 13:30:09 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_start_0 (247) confirmed on lxp005 (rc=0)
> Sep 30 13:30:09 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 248: VG_lxp0plcy_status_90000on lxp005
> Sep 30 13:30:09 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 249: oracle_plcyp0_start_0 onlxp005
> Sep 30 13:30:15 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_status_90000 (248) confirmed on lxp005 (rc=0)
> Sep 30 13:31:33 lxp003 tengine: [19672]: WARN: status_from_rc: Action start on lxp005 failed (target: <null> vs.rc: 1): Error
> Sep 30 13:31:33 lxp003 tengine: [19672]: WARN: update_failcount: Updating failcount for oracle_plcyp0 on e9d8308c-2343-43a4-927c-0fcd7f438bc2 after failed start: rc=1
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: update_abort_priority: Abort priority upgraded to 1
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: update_abort_priority: Abort action 0 superceeded by 2
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_start_0 (249) confirmed on lxp005 (rc=4)
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: run_graph: ====================================================
> Sep 30 13:31:33 lxp003 tengine: [19672]: notice: run_graph: Transition 5373: (Complete=13, Pending=0, Fired=0, Skipped=4, Incomplete=0)
> Sep 30 13:31:33 lxp003 tengine: [19672]: WARN: notify_crmd: Delaying completion until all CIB updates complete
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: extract_event: Aborting on transient_attributes changes for e9d8308c-2343-43a4-927c-0fcd7f438bc2
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: update_abort_priority: Abort priority upgraded to 1000000
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: update_abort_priority: 'Event failed' abort superceeded
> Sep 30 13:31:34 lxp003 tengine: [19672]: WARN: notify_crmd: Delaying completion until all CIB updates complete
> Sep 30 13:31:34 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:31:34 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources.
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: determine_online_status: Node lxp005 is online
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp005: 727379968
> Sep 30 13:31:34 lxp003 pengine: [19673]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp003: -1000000
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: determine_online_status: Node lxp004 is online
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: group_print: Resource Group: grp_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print: IPaddr2_lxp0plcy (ocf::heartbeat:IPaddr2): Started lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print: VG_lxp0plcy (ocf::arbella:VG): Started lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print: oracle_plcyp0 (ocf::arbella:oracle): Started lxp005 FAILED
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print: tws_lxp0plcy (ocf::arbella:tws): Stopped
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource IPaddr2_lxp0plcy (lxp005 -> lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005 Stop IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 IPaddr2_lxp0plcy_status_30000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource VG_lxp0plcy (lxp005 -> lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005 Stop VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005 Stop IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 IPaddr2_lxp0plcy_status_30000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource VG_lxp0plcy (lxp005 -> lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005 Stop VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 VG_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Recover resource oracle_plcyp0 (lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005 Stop oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 oracle_plcyp0_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start tws_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 tws_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 VG_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Recover resource oracle_plcyp0 (lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005 Stop oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 oracle_plcyp0_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start tws_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 tws_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 crmd: [3237]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: unpack_graph: Unpacked transition 5374: 17 actions in 17 synapses
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 253 fired and confirmed
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 320 fired and confirmed
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 29: oracle_plcyp0_stop_0 on lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: process_pe_message: Transition 5374: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5164.bz2
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:31:40 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_stop_0 (29) confirmed on lxp005 (rc=0)
> Sep 30 13:31:40 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 244: VG_lxp0plcy_stop_0 on lxp005
> Sep 30 13:31:50 lxp003 su: (to oracle) root on none
> Sep 30 13:31:50 lxp003 su: (to oracle) root on none
> Sep 30 13:31:57 lxp003 sshd[6654]: Accepted keyboard-interactive/pam for root from 157.250.112.149 port 3071 ssh2
> Sep 30 13:31:57 lxp003 sshd[6654]: subsystem request for sftp
> Sep 30 13:32:16 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_stop_0 (244) confirmed on lxp005 (rc=0)
> Sep 30 13:32:16 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_stop_0 (244) confirmed on lxp005 (rc=0)
> Sep 30 13:32:16 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 241: IPaddr2_lxp0plcy_stop_0 on lxp005
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_stop_0 (241) confirmedon lxp005 (rc=0)
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 254 fired and confirmed
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 76 fired and confirmed
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 251 fired and confirmed
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 242: IPaddr2_lxp0plcy_start_0on lxp004
> Sep 30 13:32:19 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_start_0 (242) confirmed on lxp004 (rc=0)
> Sep 30 13:32:19 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 243: IPaddr2_lxp0plcy_status_30000 on lxp004
> Sep 30 13:32:19 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 245: VG_lxp0plcy_start_0 on lxp004
> Sep 30 13:32:20 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_status_30000 (243) confirmed on lxp004 (rc=0)
> Sep 30 13:32:49 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_start_0 (245) confirmed on lxp004 (rc=0)
> Sep 30 13:32:49 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 246: VG_lxp0plcy_status_90000on lxp004
> Sep 30 13:32:49 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 247: oracle_plcyp0_start_0 onlxp004
> Sep 30 13:32:54 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_status_90000 (246) confirmed on lxp004 (rc=0)
> Sep 30 13:33:13 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_start_0 (247) confirmed on lxp004 (rc=0)
> Sep 30 13:33:13 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 248: oracle_plcyp0_status_90000 on lxp004
> Sep 30 13:33:13 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 249: tws_lxp0plcy_start_0 on lxp004
> Sep 30 13:33:15 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_status_90000 (248) confirmed on lxp004 (rc=0)
> Sep 30 13:33:21 lxp003 su: (to oracle) root on none
> Sep 30 13:33:21 lxp003 su: (to oracle) root on none
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_start_0 (249) confirmed onlxp004 (rc=0)
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 252 fired and confirmed
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_start_0 (249) confirmed onlxp004 (rc=0)
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 252 fired and confirmed
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 250: tws_lxp0plcy_status_90000 on lxp004
> Sep 30 13:33:27 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_status_90000 (250) confirmed on lxp004 (rc=0)
> Sep 30 13:33:27 lxp003 tengine: [19672]: info: run_graph: Transition 5374: (Complete=17, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> Sep 30 13:33:27 lxp003 tengine: [19672]: info: notify_crmd: Transition 5374 status: te_complete - <null>
> Sep 30 13:33:27 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: WARN: main: here i am - 3
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cib epoch="11440" num_updates="31">
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <configuration>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <crm_config>
> Sep 30 13:37:42 lxp003 crm_resource: [15575]: info: Invoked: /usr/sbin/crm_resource -C -r IPaddr2_lxp0plcy -H lxp003
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <attributes>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <nvpair value="1250589908" id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:42 lxp003 tengine: [19672]: info: update_abort_priority: Abort priority upgraded to 1000000
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </attributes>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cluster_property_set>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </crm_config>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </configuration>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cib>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <cib epoch="11441" num_updates="1">
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <configuration>
> Sep 30 13:37:42 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources.
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <nvpair value="1254332261" id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </attributes>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </cluster_property_set>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </crm_config>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </configuration>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </cib>
> Sep 30 13:37:42 lxp003 cib: [15578]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:42 lxp003 cib: [15578]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:42 lxp003 cib: [15578]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest: /var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:42 lxp003 cib: [15578]: info: write_cib_contents: Wrote version 0.11441.1 of the CIB to disk (digest: 6f691717ebe5390792bac03a372d53fd)
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: WARN: main: here i am - 3
> Sep 30 13:37:43 lxp003 crmd: [3237]: info: do_lrm_invoke: Removing resource IPaddr2_lxp0plcy from the LRM
> Sep 30 13:37:43 lxp003 crmd: [3237]: info: send_direct_ack: ACK'ing resource op IPaddr2_lxp0plcy_delete_60000 from 0:0:crm-resource-15575: lrm_invoke-lrmd-1254332263-11029
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: determine_online_status: Node lxp005 is online
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp005: 727379968
> Sep 30 13:37:43 lxp003 pengine: [19673]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: determine_online_status: Node lxp003 is online
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp003: -1000000
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: group_print: Resource Group: grp_lxp0plcy
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print: IPaddr2_lxp0plcy (ocf::heartbeat:IPaddr2): Stopped
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print: VG_lxp0plcy (ocf::arbella:VG): Started lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print: oracle_plcyp0 (ocf::arbella:oracle): Started lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print: tws_lxp0plcy (ocf::arbella:tws): Started lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: StartRsc: lxp004 Start IPaddr2_lxp0plcy
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 IPaddr2_lxp0plcy_status_30000
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource VG_lxp0plcy (lxp004)
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource oracle_plcyp0 (lxp004)
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource tws_lxp0plcy (lxp004)
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:43 lxp003 crmd: [3237]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: unpack_graph: Unpacked transition 5375: 13 actions in 13 synapses
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 79: IPaddr2_lxp0plcy_monitor_0 on lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: process_pe_message: Transition 5375: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5165.bz2
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cib epoch="11441" num_updates="2">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <configuration>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <crm_config>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <attributes>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <nvpair value="1254332261" id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: update_abort_priority: Abort priority upgraded to 1000000
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </attributes>
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: update_abort_priority: Abort action 0 superceeded by 2
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cluster_property_set>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </crm_config>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </configuration>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cib>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <cib epoch="11442" num_updates="1">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <configuration>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <crm_config>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <attributes>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <nvpair value="1254332263" id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </attributes>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </cluster_property_set>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff: + </crm_config>
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest: /var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: write_cib_contents: Wrote version 0.11442.1 of the CIB to disk (digest: aa82dc1cc8c60bb030142196e4cbd57f)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest: /var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: info: Invoked: /usr/sbin/crm_resource -C -r IPaddr2_lxp0plcy -H lxp005
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: WARN: main: here i am - 3
> Sep 30 13:37:44 lxp003 tengine: [19672]: info: status_from_rc: Re-mapping op status to LRM_OP_ERROR for rc=0
> Sep 30 13:37:44 lxp003 tengine: [19672]: WARN: status_from_rc: Action monitor on lxp004 failed (target: 7 vs. rc: 0): Error
> Sep 30 13:37:44 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_monitor_0 (79) confirmed on lxp004 (rc=4)
> Sep 30 13:37:44 lxp003 tengine: [19672]: info: run_graph: ====================================================
> Sep 30 13:37:44 lxp003 tengine: [19672]: notice: run_graph: Transition 5375: (Complete=1, Pending=0, Fired=0, Skipped=11, Incomplete=1)
> Sep 30 13:37:44 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:44 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources.
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: determine_online_status: Node lxp005 is online
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp005: 727379968
> Sep 30 13:37:45 lxp003 pengine: [19673]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: determine_online_status: Node lxp003 is online
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp003: -1000000
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource IPaddr2_lxp0plcy (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: RecurringOp: lxp004 IPaddr2_lxp0plcy_status_30000
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource VG_lxp0plcy (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource oracle_plcyp0 (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave resource tws_lxp0plcy (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:45 lxp003 crmd: [3237]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: unpack_graph: Unpacked transition 5376: 4 actions in 4 synapses
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 80: IPaddr2_lxp0plcy_monitor_0 on lxp003
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 244: IPaddr2_lxp0plcy_status_30000 on lxp004
> Sep 30 13:37:45 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=IPaddr2_lxp0plcy_monitor_0 key=80:5376:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:37:45 lxp003 lrmd: [3234]: info: rsc:IPaddr2_lxp0plcy: monitor
> Sep 30 13:37:45 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation IPaddr2_lxp0plcy_monitor_0 (call=553, rc=7) complete
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: process_pe_message: Transition 5376: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5166.bz2
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_monitor_0 (80) confirmed on lxp003 (rc=0)
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 79: probe_complete on lxp003
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 77 fired and confirmed
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_monitor_0 (80) confirmed on lxp003 (rc=0)
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 79: probe_complete on lxp003
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 77 fired and confirmed
> Sep 30 13:37:45 lxp003 crm_failcount: [15693]: info: Invoked: /usr/sbin/crm_failcount -D -U lxp004 -r IPaddr2_lxp0plcy
> Sep 30 13:37:45 lxp003 crm_failcount: [15697]: info: Invoked: /usr/sbin/crm_failcount -D -U lxp003 -r IPaddr2_lxp0plcy
> Sep 30 13:37:46 lxp003 crm_failcount: [15706]: info: Invoked: /usr/sbin/crm_failcount -D -U lxp005 -r IPaddr2_lxp0plcy
> Sep 30 13:37:46 lxp003 crm_resource: [15719]: info: Invoked: /usr/sbin/crm_resource -C -r VG_lxp0plcy -H lxp004
> Sep 30 13:37:46 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_status_30000 (244) confirmed on lxp004 (rc=0)
> Sep 30 13:37:46 lxp003 tengine: [19672]: info: run_graph: Transition 5376: (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> Sep 30 13:37:46 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:46 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources.
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: WARN: main: here i am - 3
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: determine_online_status: Node lxp005 is online
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp005: 727379968
> Sep 30 13:37:47 lxp003 pengine: [19673]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: determine_online_status: Node lxp003 is online
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp003: -1000000
> Sep 30 13:37:47 lxp003 crmd: [3237]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:47 lxp003 tengine: [19672]: info: unpack_graph: Unpacked transition 5377: 3 actions in 3 synapses
> Sep 30 13:37:47 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 82: IPaddr2_lxp0plcy_monitor_0 on lxp005
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: process_pe_message: Transition 5377: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5167.bz2
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: info: Invoked: /usr/sbin/crm_resource -C -r VG_lxp0plcy -H lxp003
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: main: here i am - 3
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: do_lrm_invoke: Removing resource VG_lxp0plcy from the LRM
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: send_direct_ack: ACK'ing resource op VG_lxp0plcy_delete_60000 from 0:0:crm-resource-15739: lrm_invoke-lrmd-1254332268-11038
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cib epoch="11443" num_updates="3">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <configuration>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <crm_config>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <attributes>
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: ERROR: unpack_rsc_op: Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op: Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op: Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <nvpair value="1254332265" id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: update_abort_priority: Abort priority upgraded to 1000000
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </attributes>
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: update_abort_priority: Abort action 0 superceeded by 2
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cluster_property_set>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </crm_config>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </configuration>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cib>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <cib epoch="11444" num_updates="1">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <configuration>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <crm_config>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff: + <attributes>
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_monitor_0 (82) confirmed on lxp005 (rc=0)
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 81: probe_complete on lxp005
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: run_graph: ====================================================
> Sep 30 13:37:48 lxp003 tengine: [19672]: notice: run_graph: Transition 5377: (Complete=2, Pending=0, Fired=0, Skipped=1, Incomplete=0)
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources.
> Sep 30 13:37:48 lxp003 cib: [15744]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest: /var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: write_cib_contents: Wrote version 0.11444.3 of the CIB to disk (digest: 5c0bc07724759d530ff11f60fcb55234)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cib epoch="11444" num_updates="3">
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <configuration>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <crm_config>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <attributes>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - <nvpair value="1254332267" id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </attributes>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </cluster_property_set>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </crm_config>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff: - </configuration>
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest: /var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:49 lxp003 cib: [3233]: WARN: G_SIG_dispatch: Dispatch function for SIGCHLD was delayed 120 ms (> 100 ms) before being called (GSource: 0x51c8b0)
> Sep 30 13:37:49 lxp003 cib: [3233]: info: G_SIG_dispatch: started at 3314449952 should have started at 3314449940
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 crm_resource: [15747]: info: Invoked: /usr/sbin/crm_resource -C -r VG_lxp0plcy -H lxp005
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest: /var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:49 lxp003 cib: [15745]: info: write_cib_contents: Wrote version 0.11445.1 of the CIB to disk (digest: cfe718678cb51f83d45ef9383706e276)
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 pengine: [19673]: info: determine_online_status: Node lxp005 is online
> Sep 30 13:37:49 lxp003 pengine: [19673]: info: common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on lxp005: 727379968
> Sep 30 13:38:02 lxp003 crm_resource: [16095]: WARN: main: here i am - 3
> Sep 30 13:38:03 lxp003 crm_failcount: [16111]: info: Invoked: /usr/sbin/crm_failcount -D -U lxp004 -r tws_lxp0plcy
> Sep 30 13:38:03 lxp003 crm_failcount: [16115]: info: Invoked: /usr/sbin/crm_failcount -D -U lxp003 -r tws_lxp0plcy
> Sep 30 13:38:03 lxp003 crm_failcount: [16119]: info: Invoked: /usr/sbin/crm_failcount -D -U lxp005 -r tws_lxp0plcy
> Sep 30 13:38:03 lxp003 crmd: [3237]: ERROR: send_msg_via_ipc: Unknown Sub-system (16095_crm_resource)... discarding message.
> Sep 30 13:40:57 lxp003 crm_resource: [20483]: WARN: main: here i am - 3
> Sep 30 13:40:57 lxp003 tengine: [19672]: WARN: action_timer_callback: Timer popped (abort_level=1000000, complete=false)
> Sep 30 13:40:57 lxp003 tengine: [19672]: WARN: print_elem: Action missed its timeout[Action 252]: In-flight (id:tws_lxp0plcy_stop_0, loc: lxp004, priority: 0)
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: global_timer_callback: Timer popped (abort_level=1000000, complete=false)
> Sep 30 13:45:01 lxp003 tengine: [19672]: info: unconfirmed_actions: Action tws_lxp0plcy_stop_0 252 unconfirmed from lxp004
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: unconfirmed_actions: Waiting on 1 unconfirmed actions
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: global_timer_callback: Transition abort timeout reached... marking transition complete.
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: global_timer_callback: Writing 1 unconfirmed actions to the CIB
> Sep 30 13:45:01 lxp003 tengine: [19672]: info: unconfirmed_actions: Action tws_lxp0plcy_stop_0 252 unconfirmed from lxp004
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: unconfirmed_actions: Waiting on 1 unconfirmed actions
> Sep 30 13:45:01 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:45:01 lxp003 crmd: [3237]: info: do_state_transition: All 3 cluster nodes are eligible to run resources
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource IPaddr2_lxp0plcy (lxp004 -> lxp003)
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StopRsc: lxp004 Stop IPaddr2_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003 Start IPaddr2_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003 IPaddr2_lxp0plcy_status_30000
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource VG_lxp0plcy (lxp004 -> lxp003)
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StopRsc: lxp004 Stop VG_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003 Start VG_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003 VG_lxp0plcy_status_90000
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: NoRoleChange: Move resource oracle_plcyp0 (lxp004 -> lxp003)
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StopRsc: lxp004 Stop oracle_plcyp0
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003 Start oracle_plcyp0
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003 oracle_plcyp0_status_90000
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003 Start tws_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003 tws_lxp0plcy_status_90000
> Sep 30 13:45:02 lxp003 crmd: [3237]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: unpack_graph: Unpacked transition 5380: 24 actions in 24 synapses
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 255 fired and confirmed
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 77: tws_lxp0plcy_monitor_0 onlxp004
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 79: tws_lxp0plcy_monitor_0 onlxp003
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 81: tws_lxp0plcy_monitor_0 onlxp005
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 322 fired and confirmed
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 248: oracle_plcyp0_stop_0 on lxp004
> Sep 30 13:45:02 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=tws_lxp0plcy_monitor_0 key=79:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:45:02 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: monitor
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message: Transition 5380: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5170.bz2
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:45:04 lxp003 cibadmin: [26484]: info: Invoked: /usr/sbin/cibadmin -Q
> Sep 30 13:45:11 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:45:11 lxp003 kernel: hde: 289787kB, 55385/4/129 CHS, 2548 kBps, 65535 sector size, 65535 rpm
> Sep 30 13:45:12 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation tws_lxp0plcy_monitor_0 (call=556, rc=7) complete
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_monitor_0 (79) confirmed on lxp003 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 78: probe_complete on lxp003
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_monitor_0 (81) confirmed on lxp005 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 80: probe_complete on lxp005
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_monitor_0 (77) confirmed on lxp004 (rc=0)
> Sep 30 13:45:02 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=tws_lxp0plcy_monitor_0 key=79:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:45:02 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: monitor
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message: Transition 5380: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-5170.bz2
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message: Configuration WARNINGs found during PE processing. Please run "crm_verify -L" to identify issues.
> Sep 30 13:45:04 lxp003 cibadmin: [26484]: info: Invoked: /usr/sbin/cibadmin -Q
> Sep 30 13:45:11 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:45:11 lxp003 kernel: hde: 289787kB, 55385/4/129 CHS, 2548 kBps, 65535 sector size, 65535 rpm
> Sep 30 13:45:12 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation tws_lxp0plcy_monitor_0 (call=556, rc=7) complete
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_monitor_0 (79) confirmed on lxp003 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 78: probe_complete on lxp003
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_monitor_0 (81) confirmed on lxp005 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 80: probe_complete on lxp005
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_monitor_0 (77) confirmed on lxp004 (rc=0)
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 76: probe_complete on lxp004
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 75 fired and confirmed
> Sep 30 13:45:13 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 256 fired and confirmed
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 74 fired and confirmed
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 253 fired and confirmed
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 243: IPaddr2_lxp0plcy_start_0on lxp003
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=IPaddr2_lxp0plcy_start_0 key=243:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:46:20 lxp003 lrmd: [3234]: info: rsc:IPaddr2_lxp0plcy: start
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation IPaddr2_lxp0plcy_start_0 (call=557, rc=0) complete
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_start_0 (243) confirmed on lxp003 (rc=0)
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 244: IPaddr2_lxp0plcy_status_30000 on lxp003
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 246: VG_lxp0plcy_start_0 on lxp003
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=IPaddr2_lxp0plcy_status_30000 key=244:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=VG_lxp0plcy_start_0 key=246:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:46:21 lxp003 lrmd: [3234]: info: rsc:VG_lxp0plcy: start
> Sep 30 13:46:21 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation IPaddr2_lxp0plcy_status_30000 (call=558, rc=0) complete
> Sep 30 13:46:21 lxp003 tengine: [19672]: info: match_graph_event: Action IPaddr2_lxp0plcy_status_30000 (244) confirmed on lxp003 (rc=0)
> Sep 30 13:47:04 lxp003 cibadmin: [29946]: info: Invoked: /usr/sbin/cibadmin -Q
> Sep 30 13:47:05 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:start:stderr) File descriptor 4 left open File descriptor 5 left open File descriptor 6 left open File descriptor 7 left open File descriptor 8 left open Filedescriptor 9 left open File descriptor 10 left open File descriptor 12 left open File descriptor 13 left open
> Sep 30 13:47:05 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:start:stderr) Using volume group(s) on command line
> Sep 30 13:47:05 lxp003 lrmd: [3234]: info: RA output: (VG_lxp0plcy:start:stderr) Finding volume group "plcyp0vg"
> Sep 30 13:47:10 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation VG_lxp0plcy_start_0 (call=559, rc=0)complete
> Sep 30 13:47:10 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_start_0 (246) confirmed on lxp003 (rc=0)
> Sep 30 13:47:10 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 247: VG_lxp0plcy_status_90000on lxp003
> Sep 30 13:47:10 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 249: oracle_plcyp0_start_0 onlxp003
> Sep 30 13:47:10 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=VG_lxp0plcy_status_90000 key=247:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:10 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=oracle_plcyp0_start_0 key=249:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:10 lxp003 lrmd: [3234]: info: rsc:oracle_plcyp0: start
> Sep 30 13:47:10 lxp003 su: (to oracle) root on none
> Sep 30 13:47:16 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation VG_lxp0plcy_status_90000 (call=560, rc=0) complete
> Sep 30 13:47:16 lxp003 tengine: [19672]: info: match_graph_event: Action VG_lxp0plcy_status_90000 (247) confirmed on lxp003 (rc=0)
> Sep 30 13:47:17 lxp003 cibadmin: [30820]: info: Invoked: /usr/sbin/cibadmin -Q
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation oracle_plcyp0_start_0 (call=561, rc=0) complete
> Sep 30 13:47:29 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_start_0 (249) confirmed on lxp003 (rc=0)
> Sep 30 13:47:29 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 250: oracle_plcyp0_status_90000 on lxp003
> Sep 30 13:47:29 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 251: tws_lxp0plcy_start_0 on lxp003
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=oracle_plcyp0_status_90000 key=250:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=tws_lxp0plcy_start_0 key=251:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: start
> Sep 30 13:47:29 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) Program patch revision: 1.8.3.2
> Sep 30 13:47:29 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) TWS for UNIX/CONMAN 8.2.1 (1.36.4.5) Licensed Materials Property of IBM 5698-WKB (C) Copyright IBM Corp 1998, 2005 US Government User Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Installed for user 'lxp0plcy'.
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) Locale LANG set to "en_US.UTF-8"
> Sep 30 13:47:29 lxp003 su: (to oracle) root on none
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) Schedule (Exp) 09/30/09 (#-4687) on LXP0PLCY. Batchman down. Limit: 0, Fence: 0, Audit Level: 0
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stdout) start AWSBHU507I A start command was issued for LXP0PLCY.
> Sep 30 13:47:29 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation oracle_plcyp0_status_90000 (call=562, rc=0) complete
> Sep 30 13:47:30 lxp003 tengine: [19672]: info: match_graph_event: Action oracle_plcyp0_status_90000 (250) confirmed on lxp003 (rc=0)
> Sep 30 13:47:31 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) TWS for UNIX/CONMAN 8.2.1 (1.36.4.5) Licensed Materials Property of IBM 5698-WKB (C) Copyright IBM Corp 1998, 2005 US Government User Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Installed for user 'lxp0plcy'.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) Locale LANG set to "en_US.UTF-8"
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) Schedule (Exp) 09/30/09 (#-4687) on LXP0PLCY. Batchman LIVES. Limit: 0, Fence: 0, Audit Level: 0
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) %link @;noask
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stdout) link @;noask Command forwardedto batchman for UP0TWS
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: A0V0. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: APPP40. This is not allowed.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: APPP61. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: C0V1. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: HP04. This is not allowed.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: HP10. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: INFP04. This is not allowed.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output: (tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: INFP14. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: INFU07. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the following workstation within the same domain and neither is the domain manager: INTRANET01. This is not allowed. AWSBHU537E You have issued the "link" or "unlink" command to the fo-- INSERep 30 13:47:40 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation tws_lxp0plcy_start_0 (call=563, rc=0) complete
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_start_0 (251) confirmed onlxp003 (rc=0)
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: te_pseudo_action: Pseudo action 254 fired and confirmed
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: send_rsc_command: Initiating action 252: tws_lxp0plcy_status_90000 on lxp003
> Sep 30 13:47:40 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing op=tws_lxp0plcy_status_90000 key=252:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:40 lxp003 crmd: [3237]: info: process_lrm_event: LRM operation tws_lxp0plcy_status_90000 (call=564,rc=0) complete
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: match_graph_event: Action tws_lxp0plcy_status_90000 (252) confirmed on lxp003 (rc=0)
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: run_graph: Transition 5380: (Complete=24, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: notify_crmd: Transition 5380 status: te_complete - <null>
> Sep 30 13:47:40 lxp003 crmd: [3237]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
> T

> _______________________________________________
> Linux-HA mailing list
> Linux-HA [at] lists
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems

_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Richard.Marshall at Arbella

Oct 1, 2009, 8:12 AM

Post #5 of 5 (673 views)
Permalink

Re: NEED HELP [In reply to]

Thank You

_________________________________________________
Richard Marshall | Senior Technical Specialist | Arbella Insurance Group

1900 Crown Colony Drive | Quincy, MA 02269 | ': 617.328.2921| 7:
617.515.2491 | *: Richard.Marshall [at] Arbella

-----Original Message-----
From: linux-ha-bounces [at] lists
[mailto:linux-ha-bounces [at] lists] On Behalf Of Dejan
Muhamedagic
Sent: Thursday, October 01, 2009 10:48 AM
To: General Linux-HA mailing list
Subject: Re: [Linux-HA] NEED HELP

Hi,

On Thu, Oct 01, 2009 at 08:43:14AM -0400, Marshall, Richard wrote:
> Hello:
>
> I need help trying to debug my ha-log file. Yesterday we have an
> application configured in our HA cluster, several weeks ago it failed,
> then again yesterday it failed, but this time it would not failover to
> one of the other nodes in our (3) host cluster.
>
> If someone could take a look at the attached log, comment on why the
> resource failed, and why it also did not move to an alternate node I
> would be grateful.

One node log is not enough. You have to take a look at the log on
the node where you think it should've started, but it couldn't.
From this log, I can see that one oracle resource failed
repeatedly on lxp005, but since there are no logs from that host,
one can't say why. Hint: If you wonder why a resource failed
search for lrmd:.*<rsc>. The best way is to make a hb_report of
the incident, unpack it, and then take a look at the log
excerpts.

Thanks,

Dejan

> Thank You
>
> _________________________________________________
> Richard Marshall | Senior Technical Specialist | Arbella Insurance
Group
>
>
> 1900 Crown Colony Drive | Quincy, MA 02269 | ': 617.328.2921| 7:
> 617.515.2491 | *: Richard.Marshall [at] Arbella
>
>
> This email message is intended only for the addressee(s) and contains
information that may be confidential.
> If you are not the intended recipient please notify the sender by
reply email and immediately delete this message.
> Use, disclosure or reproduction of this email by anyone other than the
intended recipient(s) is strictly prohibited.

> From: richardma [richardma [at] lxpts1]
> Sent: Thursday, October 01, 2009 7:44 AM
> To: Marshall, Richard
> Subject: log
>
> Sep 30 13:27:04 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation oracle_plcyp0_status_90000 (call=506, rc=7) complete
> Sep 30 13:27:04 lxp003 tengine: [19672]: info: process_graph_event:
Detected action oracle_plcyp0_status_90000 from a different transition:
3879 vs. 5371
> Sep 30 13:27:04 lxp003 tengine: [19672]: info: update_abort_priority:
Abort priority upgraded to 1000000
> Sep 30 13:27:04 lxp003 tengine: [19672]: WARN: update_failcount:
Updating failcount for oracle_plcyp0 on
c0f812ae-091e-4e58-94a5-3fcaf189a818 after failed status: rc=7
> Sep 30 13:27:04 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:27:04 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources.
> Sep 30 13:27:05 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp005 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: extract_event: Aborting
on transient_attributes changes for c0f812ae-091e-4e58-94a5-3fcaf189a818
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 tengine: [19672]: WARN: notify_crmd: Delaying
completion until all CIB updates complete
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp003 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_status_90000 onlxp003: Error
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp004 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: group_print: Resource
Group: grp_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: native_print:
IPaddr2_lxp0plcy (ocf::heartbeat:IPaddr2): Started lxp003
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: native_print:
VG_lxp0plcy (ocf::arbella:VG): Started lxp003
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: native_print:
oracle_plcyp0 (ocf::arbella:oracle): Started lxp003 FAILED
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource IPaddr2_lxp0plcy (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource VG_lxp0plcy (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Recover
resource oracle_plcyp0 (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003
Stop oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp003
Start oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp003
oracle_plcyp0_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource tws_lxp0plcy (lxp003)
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5372: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5162.bz2
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:27:05 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp003 is online
> Sep 30 13:27:05 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp003: -1000000
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_status_90000 onlxp003: Error
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource IPaddr2_lxp0plcy (lxp003 -> lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003
Stop IPaddr2_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005
Start IPaddr2_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005
IPaddr2_lxp0plcy_status_30000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource VG_lxp0plcy (lxp003 -> lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003
Stop VG_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005
Start VG_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005
VG_lxp0plcy_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Recover
resource oracle_plcyp0 (lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003
Stop oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005
Start VG_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005
VG_lxp0plcy_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Recover
resource oracle_plcyp0 (lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003
Stop oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005
Start oracle_plcyp0
> Sep 30 13:27:05 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: RecurringOp: lxp005
oracle_plcyp0_status_90000
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource tws_lxp0plcy (lxp003 -> lxp005)
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StopRsc: lxp003
Stop tws_lxp0plcy
> Sep 30 13:27:05 lxp003 pengine: [19673]: notice: StartRsc: lxp005
Start tws_lxp0plcy
> Sep 30 13:27:05 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: unpack_graph: Unpacked
transition 5373: 17 actions in 17 synapses
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 256 fired and confirmed
> Sep 30 13:27:05 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 251: tws_lxp0plcy_stop_0 on lxp003
> Sep 30 13:27:05 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=tws_lxp0plcy_stop_0
key=251:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:27:05 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: stop
> Sep 30 13:27:05 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation tws_lxp0plcy_status_90000 (call=508,rc=-2) Cancelled
> Sep 30 13:27:05 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stdout) usage: /usr/lib/ocf/resource.d//arbella/tws
{start|stop|monitor|validate-all|meta-data|status} Expects to have a
fully populated OCF RA-compliant environment set.
> Sep 30 13:27:05 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5373: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5163.bz2
> Sep 30 13:27:05 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stderr) TWS for UNIX/CONMAN 8.2.1 (1.36.4.5)
Licensed Materials Property of IBM 5698-WKB (C) Copyright IBM Corp
1998, 2005 US Government User Restricted Rights Use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Installed for user 'lxp0plcy'.
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stderr) Locale LANG set to "en_US.UTF-8"
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stderr) Schedule (Exp) 09/30/09 (#-4687) on LXP0PLCY.
Batchman LIVES. Limit: 0, Fence: 0, Audit Level: 0
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stderr) %unlink @;noask
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stderr) AWSBHU537E You have issued the "link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager:A0V0. This is not allowed.
> Sep 30 13:27:06 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:stop:stderr) AWSBHU537E You have issued the "link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager:APPP40. This is not allowed.
> Sep 30 13:28:16 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation tws_lxp0plcy_stop_0 (call=549, rc=0)complete
> Sep 30 13:28:16 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_stop_0 (251) confirmed on lxp003 (rc=0)
> Sep 30 13:28:16 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 60: oracle_plcyp0_stop_0 on lxp003
> Sep 30 13:28:16 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=oracle_plcyp0_stop_0
key=60:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:28:16 lxp003 lrmd: [3234]: info: rsc:oracle_plcyp0: stop
> Sep 30 13:28:16 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation oracle_plcyp0_status_90000 (call=506, rc=-2) Cancelled
> Sep 30 13:28:58 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation oracle_plcyp0_stop_0 (call=550, rc=0) complete
> Sep 30 13:28:58 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_stop_0 (60) confirmed on lxp003 (rc=0)
> Sep 30 13:28:58 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 246: VG_lxp0plcy_stop_0 on lxp003
> Sep 30 13:28:58 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=VG_lxp0plcy_stop_0 key=246:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:28:58 lxp003 lrmd: [3234]: info: rsc:VG_lxp0plcy: stop
> Sep 30 13:28:58 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation VG_lxp0plcy_status_90000 (call=504, rc=-2) Cancelled
> Sep 30 13:29:13 lxp003 lrmd: [3234]: WARN: G_SIG_dispatch: Dispatch
function for SIGCHLD was delayed 1000 ms (> 100 ms) before being called
(GSource: 0x51afa8)
> Sep 30 13:29:13 lxp003 lrmd: [3234]: info: G_SIG_dispatch: started at
3314398378 should have started at 3314398278
> Sep 30 13:29:16 lxp003 kernel: hde: 709684kB, 32917/4/129 CHS, 6175
kBps, 65535 sector size, 65535 rpm
> Sep 30 13:29:16 lxp003 kernel: klogd 1.4.1, ---------- state change
----------
> Sep 30 13:29:21 lxp003 su: (to oracle) root on none
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:stop:stderr) File descriptor 4 left open Filedescriptor 5
left open File descriptor 6 left open File descriptor 7 left open File
descriptor 8 left open File descriptor 9 left open
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:stop:stderr)
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:stop:stderr) Using volume group(s) on commandline
> Sep 30 13:29:30 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:stop:stderr) Finding volume group "plcyp0vg"
> Sep 30 13:29:30 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_stop_0 (246) confirmed on lxp003 (rc=0)
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 243: IPaddr2_lxp0plcy_stop_0 on lxp003
> Sep 30 13:29:33 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=IPaddr2_lxp0plcy_stop_0
key=243:5373:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:29:33 lxp003 lrmd: [3234]: info: rsc:IPaddr2_lxp0plcy: stop
> Sep 30 13:29:33 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation IPaddr2_lxp0plcy_status_30000 (call=502, rc=-2) Cancelled
> Sep 30 13:29:33 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation IPaddr2_lxp0plcy_stop_0 (call=552, rc=0) complete
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_stop_0 (243) confirmedon lxp003 (rc=0)
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 257 fired and confirmed
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 78 fired and confirmed
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 254 fired and confirmed
> Sep 30 13:29:33 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 244: IPaddr2_lxp0plcy_start_0on lxp005
> Sep 30 13:29:34 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_start_0 (244) confirmed on lxp005 (rc=0)
> Sep 30 13:29:34 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 245: IPaddr2_lxp0plcy_status_30000 on lxp005
> Sep 30 13:29:34 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 247: VG_lxp0plcy_start_0 on
>
> Sep 30 13:30:09 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_start_0 (247) confirmed on lxp005 (rc=0)
> Sep 30 13:30:09 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 248: VG_lxp0plcy_status_90000on lxp005
> Sep 30 13:30:09 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 249: oracle_plcyp0_start_0 onlxp005
> Sep 30 13:30:15 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_status_90000 (248) confirmed on lxp005 (rc=0)
> Sep 30 13:31:33 lxp003 tengine: [19672]: WARN: status_from_rc: Action
start on lxp005 failed (target: <null> vs.rc: 1): Error
> Sep 30 13:31:33 lxp003 tengine: [19672]: WARN: update_failcount:
Updating failcount for oracle_plcyp0 on
e9d8308c-2343-43a4-927c-0fcd7f438bc2 after failed start: rc=1
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: update_abort_priority:
Abort priority upgraded to 1
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: update_abort_priority:
Abort action 0 superceeded by 2
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_start_0 (249) confirmed on lxp005 (rc=4)
> Sep 30 13:31:33 lxp003 tengine: [19672]: info: run_graph:
====================================================
> Sep 30 13:31:33 lxp003 tengine: [19672]: notice: run_graph: Transition
5373: (Complete=13, Pending=0, Fired=0, Skipped=4, Incomplete=0)
> Sep 30 13:31:33 lxp003 tengine: [19672]: WARN: notify_crmd: Delaying
completion until all CIB updates complete
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: extract_event: Aborting
on transient_attributes changes for e9d8308c-2343-43a4-927c-0fcd7f438bc2
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: update_abort_priority:
Abort priority upgraded to 1000000
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: update_abort_priority:
'Event failed' abort superceeded
> Sep 30 13:31:34 lxp003 tengine: [19672]: WARN: notify_crmd: Delaying
completion until all CIB updates complete
> Sep 30 13:31:34 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:31:34 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources.
> Sep 30 13:31:34 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp005 is online
> Sep 30 13:31:34 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp005: 727379968
> Sep 30 13:31:34 lxp003 pengine: [19673]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp003: -1000000
> Sep 30 13:31:34 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp004 is online
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: group_print: Resource
Group: grp_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print:
IPaddr2_lxp0plcy (ocf::heartbeat:IPaddr2): Started lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print:
VG_lxp0plcy (ocf::arbella:VG): Started lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print:
oracle_plcyp0 (ocf::arbella:oracle): Started lxp005 FAILED
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: native_print:
tws_lxp0plcy (ocf::arbella:tws): Stopped
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource IPaddr2_lxp0plcy (lxp005 -> lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005
Stop IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
IPaddr2_lxp0plcy_status_30000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource VG_lxp0plcy (lxp005 -> lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005
Stop VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005
Stop IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start IPaddr2_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
IPaddr2_lxp0plcy_status_30000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource VG_lxp0plcy (lxp005 -> lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005
Stop VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start VG_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
VG_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Recover
resource oracle_plcyp0 (lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005
Stop oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
oracle_plcyp0_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start tws_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
tws_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
VG_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: NoRoleChange: Recover
resource oracle_plcyp0 (lxp004)
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StopRsc: lxp005
Stop oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start oracle_plcyp0
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
oracle_plcyp0_status_90000
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start tws_lxp0plcy
> Sep 30 13:31:34 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:31:34 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
tws_lxp0plcy_status_90000
> Sep 30 13:31:34 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: unpack_graph: Unpacked
transition 5374: 17 actions in 17 synapses
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 253 fired and confirmed
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 320 fired and confirmed
> Sep 30 13:31:34 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 29: oracle_plcyp0_stop_0 on lxp005
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5374: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5164.bz2
> Sep 30 13:31:34 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:31:40 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_stop_0 (29) confirmed on lxp005 (rc=0)
> Sep 30 13:31:40 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 244: VG_lxp0plcy_stop_0 on lxp005
> Sep 30 13:31:50 lxp003 su: (to oracle) root on none
> Sep 30 13:31:50 lxp003 su: (to oracle) root on none
> Sep 30 13:31:57 lxp003 sshd[6654]: Accepted keyboard-interactive/pam
for root from 157.250.112.149 port 3071 ssh2
> Sep 30 13:31:57 lxp003 sshd[6654]: subsystem request for sftp
> Sep 30 13:32:16 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_stop_0 (244) confirmed on lxp005 (rc=0)
> Sep 30 13:32:16 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_stop_0 (244) confirmed on lxp005 (rc=0)
> Sep 30 13:32:16 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 241: IPaddr2_lxp0plcy_stop_0 on lxp005
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_stop_0 (241) confirmedon lxp005 (rc=0)
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 254 fired and confirmed
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 76 fired and confirmed
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 251 fired and confirmed
> Sep 30 13:32:18 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 242: IPaddr2_lxp0plcy_start_0on lxp004
> Sep 30 13:32:19 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_start_0 (242) confirmed on lxp004 (rc=0)
> Sep 30 13:32:19 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 243: IPaddr2_lxp0plcy_status_30000 on lxp004
> Sep 30 13:32:19 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 245: VG_lxp0plcy_start_0 on lxp004
> Sep 30 13:32:20 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_status_30000 (243) confirmed on lxp004 (rc=0)
> Sep 30 13:32:49 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_start_0 (245) confirmed on lxp004 (rc=0)
> Sep 30 13:32:49 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 246: VG_lxp0plcy_status_90000on lxp004
> Sep 30 13:32:49 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 247: oracle_plcyp0_start_0 onlxp004
> Sep 30 13:32:54 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_status_90000 (246) confirmed on lxp004 (rc=0)
> Sep 30 13:33:13 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_start_0 (247) confirmed on lxp004 (rc=0)
> Sep 30 13:33:13 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 248: oracle_plcyp0_status_90000 on lxp004
> Sep 30 13:33:13 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 249: tws_lxp0plcy_start_0 on lxp004
> Sep 30 13:33:15 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_status_90000 (248) confirmed on lxp004 (rc=0)
> Sep 30 13:33:21 lxp003 su: (to oracle) root on none
> Sep 30 13:33:21 lxp003 su: (to oracle) root on none
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_start_0 (249) confirmed onlxp004 (rc=0)
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 252 fired and confirmed
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_start_0 (249) confirmed onlxp004 (rc=0)
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 252 fired and confirmed
> Sep 30 13:33:25 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 250: tws_lxp0plcy_status_90000 on lxp004
> Sep 30 13:33:27 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_status_90000 (250) confirmed on lxp004 (rc=0)
> Sep 30 13:33:27 lxp003 tengine: [19672]: info: run_graph: Transition
5374: (Complete=17, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> Sep 30 13:33:27 lxp003 tengine: [19672]: info: notify_crmd: Transition
5374 status: te_complete - <null>
> Sep 30 13:33:27 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [input=I_TE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:41 lxp003 crm_resource: [15564]: WARN: main: here i am -
3
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cib epoch="11440" num_updates="31">
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <configuration>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <crm_config>
> Sep 30 13:37:42 lxp003 crm_resource: [15575]: info: Invoked:
/usr/sbin/crm_resource -C -r IPaddr2_lxp0plcy -H lxp003
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <attributes>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <nvpair value="1250589908"
id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:42 lxp003 tengine: [19672]: info: update_abort_priority:
Abort priority upgraded to 1000000
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </attributes>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cluster_property_set>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </crm_config>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </configuration>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cib>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <cib epoch="11441" num_updates="1">
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <configuration>
> Sep 30 13:37:42 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources.
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <nvpair value="1254332261"
id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </attributes>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </cluster_property_set>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </crm_config>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </configuration>
> Sep 30 13:37:42 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </cib>
> Sep 30 13:37:42 lxp003 cib: [15578]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:42 lxp003 cib: [15578]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:42 lxp003 cib: [15578]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest:
/var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:42 lxp003 cib: [15578]: info: write_cib_contents: Wrote
version 0.11441.1 of the CIB to disk (digest:
6f691717ebe5390792bac03a372d53fd)
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:43 lxp003 crm_resource: [15575]: WARN: main: here i am -
3
> Sep 30 13:37:43 lxp003 crmd: [3237]: info: do_lrm_invoke: Removing
resource IPaddr2_lxp0plcy from the LRM
> Sep 30 13:37:43 lxp003 crmd: [3237]: info: send_direct_ack: ACK'ing
resource op IPaddr2_lxp0plcy_delete_60000 from 0:0:crm-resource-15575:
lrm_invoke-lrmd-1254332263-11029
> Sep 30 13:37:43 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp005 is online
> Sep 30 13:37:43 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp005: 727379968
> Sep 30 13:37:43 lxp003 pengine: [19673]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp003 is online
> Sep 30 13:37:43 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp003: -1000000
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: group_print: Resource
Group: grp_lxp0plcy
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print:
IPaddr2_lxp0plcy (ocf::heartbeat:IPaddr2): Stopped
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print:
VG_lxp0plcy (ocf::arbella:VG): Started lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print:
oracle_plcyp0 (ocf::arbella:oracle): Started lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: native_print:
tws_lxp0plcy (ocf::arbella:tws): Started lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: StartRsc: lxp004
Start IPaddr2_lxp0plcy
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
IPaddr2_lxp0plcy_status_30000
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource VG_lxp0plcy (lxp004)
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource oracle_plcyp0 (lxp004)
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource tws_lxp0plcy (lxp004)
> Sep 30 13:37:43 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:43 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: unpack_graph: Unpacked
transition 5375: 13 actions in 13 synapses
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 79: IPaddr2_lxp0plcy_monitor_0 on lxp004
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5375: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5165.bz2
> Sep 30 13:37:43 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cib epoch="11441" num_updates="2">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <configuration>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <crm_config>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <attributes>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <nvpair value="1254332261"
id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: update_abort_priority:
Abort priority upgraded to 1000000
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </attributes>
> Sep 30 13:37:43 lxp003 tengine: [19672]: info: update_abort_priority:
Abort action 0 superceeded by 2
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cluster_property_set>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </crm_config>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </configuration>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cib>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <cib epoch="11442" num_updates="1">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <configuration>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <crm_config>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <attributes>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <nvpair value="1254332263"
id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </attributes>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </cluster_property_set>
> Sep 30 13:37:43 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ </crm_config>
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest:
/var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: write_cib_contents: Wrote
version 0.11442.1 of the CIB to disk (digest:
aa82dc1cc8c60bb030142196e4cbd57f)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:43 lxp003 cib: [15624]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest:
/var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: info: Invoked:
/usr/sbin/crm_resource -C -r IPaddr2_lxp0plcy -H lxp005
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:44 lxp003 crm_resource: [15627]: WARN: main: here i am -
3
> Sep 30 13:37:44 lxp003 tengine: [19672]: info: status_from_rc:
Re-mapping op status to LRM_OP_ERROR for rc=0
> Sep 30 13:37:44 lxp003 tengine: [19672]: WARN: status_from_rc: Action
monitor on lxp004 failed (target: 7 vs. rc: 0): Error
> Sep 30 13:37:44 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_monitor_0 (79) confirmed on lxp004 (rc=4)
> Sep 30 13:37:44 lxp003 tengine: [19672]: info: run_graph:
====================================================
> Sep 30 13:37:44 lxp003 tengine: [19672]: notice: run_graph: Transition
5375: (Complete=1, Pending=0, Fired=0, Skipped=11, Incomplete=1)
> Sep 30 13:37:44 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:44 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources.
> Sep 30 13:37:45 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp005 is online
> Sep 30 13:37:45 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp005: 727379968
> Sep 30 13:37:45 lxp003 pengine: [19673]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp003 is online
> Sep 30 13:37:45 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp003: -1000000
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource IPaddr2_lxp0plcy (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: RecurringOp: lxp004
IPaddr2_lxp0plcy_status_30000
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource VG_lxp0plcy (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource oracle_plcyp0 (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 pengine: [19673]: notice: NoRoleChange: Leave
resource tws_lxp0plcy (lxp004)
> Sep 30 13:37:45 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:45 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: unpack_graph: Unpacked
transition 5376: 4 actions in 4 synapses
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 80: IPaddr2_lxp0plcy_monitor_0 on lxp003
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 244: IPaddr2_lxp0plcy_status_30000 on lxp004
> Sep 30 13:37:45 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=IPaddr2_lxp0plcy_monitor_0
key=80:5376:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:37:45 lxp003 lrmd: [3234]: info: rsc:IPaddr2_lxp0plcy:
monitor
> Sep 30 13:37:45 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation IPaddr2_lxp0plcy_monitor_0 (call=553, rc=7) complete
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5376: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5166.bz2
> Sep 30 13:37:45 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_monitor_0 (80) confirmed on lxp003 (rc=0)
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 79: probe_complete on lxp003
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 77 fired and confirmed
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_monitor_0 (80) confirmed on lxp003 (rc=0)
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 79: probe_complete on lxp003
> Sep 30 13:37:45 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 77 fired and confirmed
> Sep 30 13:37:45 lxp003 crm_failcount: [15693]: info: Invoked:
/usr/sbin/crm_failcount -D -U lxp004 -r IPaddr2_lxp0plcy
> Sep 30 13:37:45 lxp003 crm_failcount: [15697]: info: Invoked:
/usr/sbin/crm_failcount -D -U lxp003 -r IPaddr2_lxp0plcy
> Sep 30 13:37:46 lxp003 crm_failcount: [15706]: info: Invoked:
/usr/sbin/crm_failcount -D -U lxp005 -r IPaddr2_lxp0plcy
> Sep 30 13:37:46 lxp003 crm_resource: [15719]: info: Invoked:
/usr/sbin/crm_resource -C -r VG_lxp0plcy -H lxp004
> Sep 30 13:37:46 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_status_30000 (244) confirmed on lxp004 (rc=0)
> Sep 30 13:37:46 lxp003 tengine: [19672]: info: run_graph: Transition
5376: (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> Sep 30 13:37:46 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:46 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources.
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:47 lxp003 crm_resource: [15719]: WARN: main: here i am -
3
> Sep 30 13:37:47 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp005 is online
> Sep 30 13:37:47 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp005: 727379968
> Sep 30 13:37:47 lxp003 pengine: [19673]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 on lxp005: Error
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:37:47 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp003 is online
> Sep 30 13:37:47 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp003: -1000000
> Sep 30 13:37:47 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:47 lxp003 tengine: [19672]: info: unpack_graph: Unpacked
transition 5377: 3 actions in 3 synapses
> Sep 30 13:37:47 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 82: IPaddr2_lxp0plcy_monitor_0 on lxp005
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5377: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5167.bz2
> Sep 30 13:37:47 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: info: Invoked:
/usr/sbin/crm_resource -C -r VG_lxp0plcy -H lxp003
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: main: here i am -
3
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: do_lrm_invoke: Removing
resource VG_lxp0plcy from the LRM
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: send_direct_ack: ACK'ing
resource op VG_lxp0plcy_delete_60000 from 0:0:crm-resource-15739:
lrm_invoke-lrmd-1254332268-11038
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cib epoch="11443" num_updates="3">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <configuration>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <crm_config>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <attributes>
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: ERROR: unpack_rsc_op:
Remapping oracle_plcyp0_start_0 (rc=1) on lxp005 to an ERROR
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op:
Processing failed op oracle_plcyp0_start_0 onlxp005: Error
> Sep 30 13:37:48 lxp003 crm_resource: [15739]: WARN: unpack_rsc_op:
Compatability handling for failed op oracle_plcyp0_start_0 on lxp005
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <nvpair value="1254332265"
id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: update_abort_priority:
Abort priority upgraded to 1000000
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </attributes>
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: update_abort_priority:
Abort action 0 superceeded by 2
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cluster_property_set>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </crm_config>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </configuration>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cib>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <cib epoch="11444" num_updates="1">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <configuration>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <crm_config>
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:48 lxp003 cib: [3233]: info: log_data_element: cib:diff:
+ <attributes>
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_monitor_0 (82) confirmed on lxp005 (rc=0)
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 81: probe_complete on lxp005
> Sep 30 13:37:48 lxp003 tengine: [19672]: info: run_graph:
====================================================
> Sep 30 13:37:48 lxp003 tengine: [19672]: notice: run_graph: Transition
5377: (Complete=2, Pending=0, Fired=0, Skipped=1, Incomplete=0)
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:37:48 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources.
> Sep 30 13:37:48 lxp003 cib: [15744]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest:
/var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: write_cib_contents: Wrote
version 0.11444.3 of the CIB to disk (digest:
5c0bc07724759d530ff11f60fcb55234)
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cib epoch="11444" num_updates="3">
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <configuration>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <crm_config>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <cluster_property_set id="cib-bootstrap-options">
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <attributes>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- <nvpair value="1254332267"
id="cib-bootstrap-options-last-lrm-refresh"/>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </attributes>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </cluster_property_set>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </crm_config>
> Sep 30 13:37:49 lxp003 cib: [3233]: info: log_data_element: cib:diff:
- </configuration>
> Sep 30 13:37:49 lxp003 cib: [15744]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest:
/var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:49 lxp003 cib: [3233]: WARN: G_SIG_dispatch: Dispatch
function for SIGCHLD was delayed 120 ms (> 100 ms) before being called
(GSource: 0x51c8b0)
> Sep 30 13:37:49 lxp003 cib: [3233]: info: G_SIG_dispatch: started at
3314449952 should have started at 3314449940
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 crm_resource: [15747]: info: Invoked:
/usr/sbin/crm_resource -C -r VG_lxp0plcy -H lxp005
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml.last (digest:
/var/lib/heartbeat/crm/cib.xml.sig.last)
> Sep 30 13:37:49 lxp003 cib: [15745]: info: write_cib_contents: Wrote
version 0.11445.1 of the CIB to disk (digest:
cfe718678cb51f83d45ef9383706e276)
> Sep 30 13:37:49 lxp003 cib: [15745]: info: retrieveCib: Reading
cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
> Sep 30 13:37:49 lxp003 pengine: [19673]: info:
determine_online_status: Node lxp005 is online
> Sep 30 13:37:49 lxp003 pengine: [19673]: info:
common_apply_stickiness: Setting failure stickiness for oracle_plcyp0 on
lxp005: 727379968
> Sep 30 13:38:02 lxp003 crm_resource: [16095]: WARN: main: here i am -
3
> Sep 30 13:38:03 lxp003 crm_failcount: [16111]: info: Invoked:
/usr/sbin/crm_failcount -D -U lxp004 -r tws_lxp0plcy
> Sep 30 13:38:03 lxp003 crm_failcount: [16115]: info: Invoked:
/usr/sbin/crm_failcount -D -U lxp003 -r tws_lxp0plcy
> Sep 30 13:38:03 lxp003 crm_failcount: [16119]: info: Invoked:
/usr/sbin/crm_failcount -D -U lxp005 -r tws_lxp0plcy
> Sep 30 13:38:03 lxp003 crmd: [3237]: ERROR: send_msg_via_ipc: Unknown
Sub-system (16095_crm_resource)... discarding message.
> Sep 30 13:40:57 lxp003 crm_resource: [20483]: WARN: main: here i am -
3
> Sep 30 13:40:57 lxp003 tengine: [19672]: WARN: action_timer_callback:
Timer popped (abort_level=1000000, complete=false)
> Sep 30 13:40:57 lxp003 tengine: [19672]: WARN: print_elem: Action
missed its timeout[Action 252]: In-flight (id:tws_lxp0plcy_stop_0, loc:
lxp004, priority: 0)
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: global_timer_callback:
Timer popped (abort_level=1000000, complete=false)
> Sep 30 13:45:01 lxp003 tengine: [19672]: info: unconfirmed_actions:
Action tws_lxp0plcy_stop_0 252 unconfirmed from lxp004
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: unconfirmed_actions:
Waiting on 1 unconfirmed actions
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: global_timer_callback:
Transition abort timeout reached... marking transition complete.
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: global_timer_callback:
Writing 1 unconfirmed actions to the CIB
> Sep 30 13:45:01 lxp003 tengine: [19672]: info: unconfirmed_actions:
Action tws_lxp0plcy_stop_0 252 unconfirmed from lxp004
> Sep 30 13:45:01 lxp003 tengine: [19672]: WARN: unconfirmed_actions:
Waiting on 1 unconfirmed actions
> Sep 30 13:45:01 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:45:01 lxp003 crmd: [3237]: info: do_state_transition: All 3
cluster nodes are eligible to run resources
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource IPaddr2_lxp0plcy (lxp004 -> lxp003)
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StopRsc: lxp004
Stop IPaddr2_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003
Start IPaddr2_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003
IPaddr2_lxp0plcy_status_30000
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource VG_lxp0plcy (lxp004 -> lxp003)
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StopRsc: lxp004
Stop VG_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003
Start VG_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003
VG_lxp0plcy_status_90000
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: NoRoleChange: Move
resource oracle_plcyp0 (lxp004 -> lxp003)
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StopRsc: lxp004
Stop oracle_plcyp0
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003
Start oracle_plcyp0
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003
oracle_plcyp0_status_90000
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: StartRsc: lxp003
Start tws_lxp0plcy
> Sep 30 13:45:02 lxp003 pengine: [19673]: WARN: text2task: Unsupported
action: status
> Sep 30 13:45:02 lxp003 pengine: [19673]: notice: RecurringOp: lxp003
tws_lxp0plcy_status_90000
> Sep 30 13:45:02 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: unpack_graph: Unpacked
transition 5380: 24 actions in 24 synapses
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 255 fired and confirmed
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 77: tws_lxp0plcy_monitor_0 onlxp004
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 79: tws_lxp0plcy_monitor_0 onlxp003
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 81: tws_lxp0plcy_monitor_0 onlxp005
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 322 fired and confirmed
> Sep 30 13:45:02 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 248: oracle_plcyp0_stop_0 on lxp004
> Sep 30 13:45:02 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=tws_lxp0plcy_monitor_0
key=79:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:45:02 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: monitor
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5380: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5170.bz2
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:45:04 lxp003 cibadmin: [26484]: info: Invoked:
/usr/sbin/cibadmin -Q
> Sep 30 13:45:11 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:45:11 lxp003 kernel: hde: 289787kB, 55385/4/129 CHS, 2548
kBps, 65535 sector size, 65535 rpm
> Sep 30 13:45:12 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation tws_lxp0plcy_monitor_0 (call=556, rc=7) complete
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_monitor_0 (79) confirmed on lxp003 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 78: probe_complete on lxp003
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_monitor_0 (81) confirmed on lxp005 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 80: probe_complete on lxp005
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_monitor_0 (77) confirmed on lxp004 (rc=0)
> Sep 30 13:45:02 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=tws_lxp0plcy_monitor_0
key=79:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:45:02 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: monitor
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message:
Transition 5380: PEngine Input stored in:
/var/lib/heartbeat/pengine/pe-input-5170.bz2
> Sep 30 13:45:02 lxp003 pengine: [19673]: info: process_pe_message:
Configuration WARNINGs found during PE processing. Please run
"crm_verify -L" to identify issues.
> Sep 30 13:45:04 lxp003 cibadmin: [26484]: info: Invoked:
/usr/sbin/cibadmin -Q
> Sep 30 13:45:11 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:45:11 lxp003 kernel: hde: 289787kB, 55385/4/129 CHS, 2548
kBps, 65535 sector size, 65535 rpm
> Sep 30 13:45:12 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation tws_lxp0plcy_monitor_0 (call=556, rc=7) complete
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_monitor_0 (79) confirmed on lxp003 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 78: probe_complete on lxp003
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_monitor_0 (81) confirmed on lxp005 (rc=0)
> Sep 30 13:45:12 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 80: probe_complete on lxp005
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_monitor_0 (77) confirmed on lxp004 (rc=0)
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 76: probe_complete on lxp004
> Sep 30 13:45:13 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 75 fired and confirmed
> Sep 30 13:45:13 lxp003 kernel: hde: Invalid capacity for disk in drive
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 256 fired and confirmed
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 74 fired and confirmed
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 253 fired and confirmed
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 243: IPaddr2_lxp0plcy_start_0on lxp003
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=IPaddr2_lxp0plcy_start_0
key=243:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:46:20 lxp003 lrmd: [3234]: info: rsc:IPaddr2_lxp0plcy: start
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation IPaddr2_lxp0plcy_start_0 (call=557, rc=0) complete
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_start_0 (243) confirmed on lxp003 (rc=0)
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 244: IPaddr2_lxp0plcy_status_30000 on lxp003
> Sep 30 13:46:20 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 246: VG_lxp0plcy_start_0 on lxp003
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=IPaddr2_lxp0plcy_status_30000
key=244:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:46:20 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=VG_lxp0plcy_start_0
key=246:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:46:21 lxp003 lrmd: [3234]: info: rsc:VG_lxp0plcy: start
> Sep 30 13:46:21 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation IPaddr2_lxp0plcy_status_30000 (call=558, rc=0) complete
> Sep 30 13:46:21 lxp003 tengine: [19672]: info: match_graph_event:
Action IPaddr2_lxp0plcy_status_30000 (244) confirmed on lxp003 (rc=0)
> Sep 30 13:47:04 lxp003 cibadmin: [29946]: info: Invoked:
/usr/sbin/cibadmin -Q
> Sep 30 13:47:05 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:start:stderr) File descriptor 4 left open File descriptor 5
left open File descriptor 6 left open File descriptor 7 left open File
descriptor 8 left open Filedescriptor 9 left open File descriptor 10
left open File descriptor 12 left open File descriptor 13 left open
> Sep 30 13:47:05 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:start:stderr) Using volume group(s) on command line
> Sep 30 13:47:05 lxp003 lrmd: [3234]: info: RA output:
(VG_lxp0plcy:start:stderr) Finding volume group "plcyp0vg"
> Sep 30 13:47:10 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation VG_lxp0plcy_start_0 (call=559, rc=0)complete
> Sep 30 13:47:10 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_start_0 (246) confirmed on lxp003 (rc=0)
> Sep 30 13:47:10 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 247: VG_lxp0plcy_status_90000on lxp003
> Sep 30 13:47:10 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 249: oracle_plcyp0_start_0 onlxp003
> Sep 30 13:47:10 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=VG_lxp0plcy_status_90000
key=247:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:10 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=oracle_plcyp0_start_0
key=249:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:10 lxp003 lrmd: [3234]: info: rsc:oracle_plcyp0: start
> Sep 30 13:47:10 lxp003 su: (to oracle) root on none
> Sep 30 13:47:16 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation VG_lxp0plcy_status_90000 (call=560, rc=0) complete
> Sep 30 13:47:16 lxp003 tengine: [19672]: info: match_graph_event:
Action VG_lxp0plcy_status_90000 (247) confirmed on lxp003 (rc=0)
> Sep 30 13:47:17 lxp003 cibadmin: [30820]: info: Invoked:
/usr/sbin/cibadmin -Q
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation oracle_plcyp0_start_0 (call=561, rc=0) complete
> Sep 30 13:47:29 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_start_0 (249) confirmed on lxp003 (rc=0)
> Sep 30 13:47:29 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 250: oracle_plcyp0_status_90000 on lxp003
> Sep 30 13:47:29 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 251: tws_lxp0plcy_start_0 on lxp003
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=oracle_plcyp0_status_90000
key=250:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=tws_lxp0plcy_start_0
key=251:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: rsc:tws_lxp0plcy: start
> Sep 30 13:47:29 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) Program patch revision: 1.8.3.2
> Sep 30 13:47:29 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) TWS for UNIX/CONMAN 8.2.1 (1.36.4.5)
Licensed Materials Property of IBM 5698-WKB (C) Copyright IBM Corp
1998, 2005 US Government User Restricted Rights Use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Installed for user 'lxp0plcy'.
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) Locale LANG set to "en_US.UTF-8"
> Sep 30 13:47:29 lxp003 su: (to oracle) root on none
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) Schedule (Exp) 09/30/09 (#-4687) on
LXP0PLCY. Batchman down. Limit: 0, Fence: 0, Audit Level: 0
> Sep 30 13:47:29 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stdout) start AWSBHU507I A start command was issued
for LXP0PLCY.
> Sep 30 13:47:29 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:29 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation oracle_plcyp0_status_90000 (call=562, rc=0) complete
> Sep 30 13:47:30 lxp003 tengine: [19672]: info: match_graph_event:
Action oracle_plcyp0_status_90000 (250) confirmed on lxp003 (rc=0)
> Sep 30 13:47:31 lxp003 su: (to lxp0plcy) root on none
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) TWS for UNIX/CONMAN 8.2.1 (1.36.4.5)
Licensed Materials Property of IBM 5698-WKB (C) Copyright IBM Corp
1998, 2005 US Government User Restricted Rights Use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Installed for user 'lxp0plcy'.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) Locale LANG set to "en_US.UTF-8"
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) Schedule (Exp) 09/30/09 (#-4687) on
LXP0PLCY. Batchman LIVES. Limit: 0, Fence: 0, Audit Level: 0
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) %link @;noask
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stdout) link @;noask Command forwardedto batchman
for UP0TWS
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager: A0V0. This is not allowed. AWSBHU537E You
have issued the "link" or "unlink" command to the following workstation
within the same domain and neither is the domain manager: APPP40. This
is not allowed.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager: APPP61. This is not allowed. AWSBHU537E
You have issued the "link" or "unlink" command to the following
workstation within the same domain and neither is the domain manager:
C0V1. This is not allowed. AWSBHU537E You have issued the "link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager: HP04. This is not allowed.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager: HP10. This is not allowed. AWSBHU537E You
have issued the "link" or "unlink" command to the following workstation
within the same domain and neither is the domain manager: INFP04. This
is not allowed.
> Sep 30 13:47:40 lxp003 lrmd: [3234]: info: RA output:
(tws_lxp0plcy:start:stderr) AWSBHU537E You have issued the"link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager: INFP14. This is not allowed. AWSBHU537E
You have issued the "link" or "unlink" command to the following
workstation within the same domain and neither is the domain manager:
INFU07. This is not allowed. AWSBHU537E You have issued the "link" or
"unlink" command to the following workstation within the same domain and
neither is the domain manager: INTRANET01. This is not allowed.
AWSBHU537E You have issued the "link" or "unlink" command to the fo--
INSERep 30 13:47:40 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation tws_lxp0plcy_start_0 (call=563, rc=0) complete
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_start_0 (251) confirmed onlxp003 (rc=0)
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: te_pseudo_action:
Pseudo action 254 fired and confirmed
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: send_rsc_command:
Initiating action 252: tws_lxp0plcy_status_90000 on lxp003
> Sep 30 13:47:40 lxp003 crmd: [3237]: info: do_lrm_rsc_op: Performing
op=tws_lxp0plcy_status_90000
key=252:5380:a70633f6-f0cb-42e4-be6f-ac28188de89c)
> Sep 30 13:47:40 lxp003 crmd: [3237]: info: process_lrm_event: LRM
operation tws_lxp0plcy_status_90000 (call=564,rc=0) complete
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: match_graph_event:
Action tws_lxp0plcy_status_90000 (252) confirmed on lxp003 (rc=0)
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: run_graph: Transition
5380: (Complete=24, Pending=0, Fired=0, Skipped=0, Incomplete=0)
> Sep 30 13:47:40 lxp003 tengine: [19672]: info: notify_crmd: Transition
5380 status: te_complete - <null>
> Sep 30 13:47:40 lxp003 crmd: [3237]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [input=I_TE_SUCCESS
cause=C_IPC_MESSAGE origin=route_message ]
> T

> _______________________________________________
> Linux-HA mailing list
> Linux-HA [at] lists
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems

_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads