Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux-HA: Pacemaker

HAproxy resource not moving

 

 

Linux-HA pacemaker RSS feed   Index | Next | Previous | View Threaded


markbusu at gmail

Apr 30, 2012, 11:06 AM

Post #1 of 6 (597 views)
Permalink
HAproxy resource not moving

Hey guys,

I recently started testing out Pacemaker + Corosync for a clustering
solution for HA proxy. My configuration is as follows

crm(live)configure# show
node node1.local
node node2.local
primitive ClusterIP ocf:heartbeat:IPaddr2 \
params ip="********" cidr_netmask="32" \
op monitor interval="30s"
primitive test lsb:haproxy \
op monitor interval="10s"
colocation haproxy-with-public-IPs inf: test ClusterIP
order haproxy-after-IP inf: ClusterIP test
property $id="cib-bootstrap-options" \
dc-version="1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false"

If both my nodes are up, the VIP responds as requested to the HA proxy
requests.. and seems to always stick to node2.

Online: [node2.local node1.local ]

ClusterIP (ocf::heartbeat:IPaddr2): Started node2.local
test (lsb:haproxy): Started node2.local

However, if i break HA proxy on node 2 or reboot that machine, the cluster
dies and doest move over to node 1 and the crm_mon displays:

Online: [node1.local ]
OFFLINE: [ node2.local ]

Not sure what im missing here, however i would assume that both resources
should move over to Node1..

Thanks in advance for any help you can provide.
--
Regards,
Mark Busuttil


jsmith at argotec

Apr 30, 2012, 11:15 AM

Post #2 of 6 (568 views)
Permalink
Re: HAproxy resource not moving [In reply to]

----- Original Message -----

> From: "Mark Busuttil" <markbusu [at] gmail>
> To: pacemaker [at] oss
> Sent: Monday, April 30, 2012 2:06:42 PM
> Subject: [Pacemaker] HAproxy resource not moving

> Hey guys,

> I recently started testing out Pacemaker + Corosync for a clustering
> solution for HA proxy. My configuration is as follows

> crm(live)configure# show
> node node1.local
> node node2.local
> primitive ClusterIP ocf:heartbeat:IPaddr2 \
> params ip="********" cidr_netmask="32" \
> op monitor interval="30s"
> primitive test lsb:haproxy \
> op monitor interval="10s"
> colocation haproxy-with-public-IPs inf: test ClusterIP
> order haproxy-after-IP inf: ClusterIP test
> property $id="cib-bootstrap-options" \
> dc-version="1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="2" \
> stonith-enabled="false"

Might be because you've lost quorum and the cluster has no directions without quorum... this might fix:

crm configure property no-quorum-policy="ignore"


HTH

Jake

> If both my nodes are up, the VIP responds as requested to the HA
> proxy requests.. and seems to always stick to node2.

> Online: [node2.local node1.local ]

> ClusterIP (ocf::heartbeat:IPaddr2): Started node2.local
> test (lsb:haproxy): Started node2.local

> However, if i break HA proxy on node 2 or reboot that machine, the
> cluster dies and doest move over to node 1 and the crm_mon displays:

> Online: [node1.local ]
> OFFLINE: [ node2.local ]

> Not sure what im missing here, however i would assume that both
> resources should move over to Node1..

> Thanks in advance for any help you can provide. --
> Regards,
> Mark Busuttil

> _______________________________________________
> Pacemaker mailing list: Pacemaker [at] oss
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker

> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


markbusu at gmail

Apr 30, 2012, 12:44 PM

Post #3 of 6 (564 views)
Permalink
Re: HAproxy resource not moving [In reply to]

Hmm unfortunately seems to be the same issue... the cluster IP seems to
remain... however the ha proxy resource just goes offline and does not
move.. thanks
On Apr 30, 2012 8:16 PM, "Jake Smith" <jsmith [at] argotec> wrote:

>
> ----- Original Message -----
>
> > From: "Mark Busuttil" <markbusu [at] gmail>
> > To: pacemaker [at] oss
> > Sent: Monday, April 30, 2012 2:06:42 PM
> > Subject: [Pacemaker] HAproxy resource not moving
>
> > Hey guys,
>
> > I recently started testing out Pacemaker + Corosync for a clustering
> > solution for HA proxy. My configuration is as follows
>
> > crm(live)configure# show
> > node node1.local
> > node node2.local
> > primitive ClusterIP ocf:heartbeat:IPaddr2 \
> > params ip="********" cidr_netmask="32" \
> > op monitor interval="30s"
> > primitive test lsb:haproxy \
> > op monitor interval="10s"
> > colocation haproxy-with-public-IPs inf: test ClusterIP
> > order haproxy-after-IP inf: ClusterIP test
> > property $id="cib-bootstrap-options" \
> > dc-version="1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558" \
> > cluster-infrastructure="openais" \
> > expected-quorum-votes="2" \
> > stonith-enabled="false"
>
> Might be because you've lost quorum and the cluster has no directions
> without quorum... this might fix:
>
> crm configure property no-quorum-policy="ignore"
>
>
> HTH
>
> Jake
>
> > If both my nodes are up, the VIP responds as requested to the HA
> > proxy requests.. and seems to always stick to node2.
>
> > Online: [node2.local node1.local ]
>
> > ClusterIP (ocf::heartbeat:IPaddr2): Started node2.local
> > test (lsb:haproxy): Started node2.local
>
> > However, if i break HA proxy on node 2 or reboot that machine, the
> > cluster dies and doest move over to node 1 and the crm_mon displays:
>
> > Online: [node1.local ]
> > OFFLINE: [ node2.local ]
>
> > Not sure what im missing here, however i would assume that both
> > resources should move over to Node1..
>
> > Thanks in advance for any help you can provide. --
> > Regards,
> > Mark Busuttil
>
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker [at] oss
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> > Project Home: http://www.clusterlabs.org
> > Getting started:
> > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > Bugs: http://bugs.clusterlabs.org
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker [at] oss
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>


jsmith at argotec

Apr 30, 2012, 12:51 PM

Post #4 of 6 (568 views)
Permalink
Re: HAproxy resource not moving [In reply to]

What do you mean the cluster IP remains?

No errors in the logs?

Is the haproxy LSB compliant? see
http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/ap-lsb.html

Output of crm_mon -fr1?

Jake

----- Original Message -----

> From: "Mark Busuttil" <markbusu [at] gmail>
> To: "Jake Smith" <jsmith [at] argotec>, pacemaker [at] oss
> Sent: Monday, April 30, 2012 3:44:55 PM
> Subject: Re: [Pacemaker] HAproxy resource not moving

> Hmm unfortunately seems to be the same issue... the cluster IP seems
> to remain... however the ha proxy resource just goes offline and
> does not move.. thanks
> On Apr 30, 2012 8:16 PM, "Jake Smith" < jsmith [at] argotec > wrote:

> > ----- Original Message -----
>

> > > From: "Mark Busuttil" < markbusu [at] gmail >
>
> > > To: pacemaker [at] oss
>
> > > Sent: Monday, April 30, 2012 2:06:42 PM
>
> > > Subject: [Pacemaker] HAproxy resource not moving
>

> > > Hey guys,
>

> > > I recently started testing out Pacemaker + Corosync for a
> > > clustering
>
> > > solution for HA proxy. My configuration is as follows
>

> > > crm(live)configure# show
>
> > > node node1.local
>
> > > node node2.local
>
> > > primitive ClusterIP ocf:heartbeat:IPaddr2 \
>
> > > params ip="********" cidr_netmask="32" \
>
> > > op monitor interval="30s"
>
> > > primitive test lsb:haproxy \
>
> > > op monitor interval="10s"
>
> > > colocation haproxy-with-public-IPs inf: test ClusterIP
>
> > > order haproxy-after-IP inf: ClusterIP test
>
> > > property $id="cib-bootstrap-options" \
>
> > > dc-version="1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558"
> > > \
>
> > > cluster-infrastructure="openais" \
>
> > > expected-quorum-votes="2" \
>
> > > stonith-enabled="false"
>

> > Might be because you've lost quorum and the cluster has no
> > directions
> > without quorum... this might fix:
>

> > crm configure property no-quorum-policy="ignore"
>

> > HTH
>

> > Jake
>

> > > If both my nodes are up, the VIP responds as requested to the HA
>
> > > proxy requests.. and seems to always stick to node2.
>

> > > Online: [node2.local node1.local ]
>

> > > ClusterIP (ocf::heartbeat:IPaddr2): Started node2.local
>
> > > test (lsb:haproxy): Started node2.local
>

> > > However, if i break HA proxy on node 2 or reboot that machine,
> > > the
>
> > > cluster dies and doest move over to node 1 and the crm_mon
> > > displays:
>

> > > Online: [node1.local ]
>
> > > OFFLINE: [ node2.local ]
>

> > > Not sure what im missing here, however i would assume that both
>
> > > resources should move over to Node1..
>

> > > Thanks in advance for any help you can provide. --
>
> > > Regards,
>
> > > Mark Busuttil
>

> > > _______________________________________________
>
> > > Pacemaker mailing list: Pacemaker [at] oss
>
> > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>

> > > Project Home: http://www.clusterlabs.org
>
> > > Getting started:
>
> > > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
> > > Bugs: http://bugs.clusterlabs.org
>

> > _______________________________________________
>
> > Pacemaker mailing list: Pacemaker [at] oss
>
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>

> > Project Home: http://www.clusterlabs.org
>
> > Getting started:
> > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
> > Bugs: http://bugs.clusterlabs.org
>


markbusu at gmail

May 1, 2012, 3:38 AM

Post #5 of 6 (559 views)
Permalink
Re: HAproxy resource not moving [In reply to]

Thank you all for your help, I have isolated the issue and it seems that
its working correctly, it was miss configuration from my end, with the help
of "crm configure property no-quorum-policy="ignore"

We are nearly working as expected, however could someone confirm if the
following assumption is correct:

Node1 is running a resource and fails
Node 2 starts running the resource
Node 2 fails, the resource would not attempt to move onto Node 1, unless i
executed:

crm resource cleanup haproxy

Thanks in advance!

On Mon, Apr 30, 2012 at 9:51 PM, Jake Smith <jsmith [at] argotec> wrote:

> What do you mean the cluster IP remains?
>
> No errors in the logs?
>
> Is the haproxy LSB compliant? see
>
> http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/ap-lsb.html
>
> Output of crm_mon -fr1?
>
> Jake****
>
> ** **
>
> ------------------------------
>
> *From: *"Mark Busuttil" <markbusu [at] gmail>
> *To: *"Jake Smith" <jsmith [at] argotec>, pacemaker [at] oss
> *Sent: *Monday, April 30, 2012 3:44:55 PM
> *Subject: *Re: [Pacemaker] HAproxy resource not moving
>
>
> Hmm unfortunately seems to be the same issue... the cluster IP seems to
> remain... however the ha proxy resource just goes offline and does not
> move.. thanks
> On Apr 30, 2012 8:16 PM, "Jake Smith" <jsmith [at] argotec> wrote:
>
>>
>> ----- Original Message -----
>>
>> > From: "Mark Busuttil" <markbusu [at] gmail>
>> > To: pacemaker [at] oss
>> > Sent: Monday, April 30, 2012 2:06:42 PM
>> > Subject: [Pacemaker] HAproxy resource not moving
>>
>> > Hey guys,
>>
>> > I recently started testing out Pacemaker + Corosync for a clustering
>> > solution for HA proxy. My configuration is as follows
>>
>> > crm(live)configure# show
>> > node node1.local
>> > node node2.local
>> > primitive ClusterIP ocf:heartbeat:IPaddr2 \
>> > params ip="********" cidr_netmask="32" \
>> > op monitor interval="30s"
>> > primitive test lsb:haproxy \
>> > op monitor interval="10s"
>> > colocation haproxy-with-public-IPs inf: test ClusterIP
>> > order haproxy-after-IP inf: ClusterIP test
>> > property $id="cib-bootstrap-options" \
>> > dc-version="1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558" \
>> > cluster-infrastructure="openais" \
>> > expected-quorum-votes="2" \
>> > stonith-enabled="false"
>>
>> Might be because you've lost quorum and the cluster has no directions
>> without quorum... this might fix:
>>
>> crm configure property no-quorum-policy="ignore"
>>
>>
>> HTH
>>
>> Jake
>>
>> > If both my nodes are up, the VIP responds as requested to the HA
>> > proxy requests.. and seems to always stick to node2.
>>
>> > Online: [node2.local node1.local ]
>>
>> > ClusterIP (ocf::heartbeat:IPaddr2): Started node2.local
>> > test (lsb:haproxy): Started node2.local
>>
>> > However, if i break HA proxy on node 2 or reboot that machine, the
>> > cluster dies and doest move over to node 1 and the crm_mon displays:
>>
>> > Online: [node1.local ]
>> > OFFLINE: [ node2.local ]
>>
>> > Not sure what im missing here, however i would assume that both
>> > resources should move over to Node1..
>>
>> > Thanks in advance for any help you can provide. --
>> > Regards,
>> > Mark Busuttil
>>
>> > _______________________________________________
>> > Pacemaker mailing list: Pacemaker [at] oss
>> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> > Project Home: http://www.clusterlabs.org
>> > Getting started:
>> > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> > Bugs: http://bugs.clusterlabs.org
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker [at] oss
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker [at] oss
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>


--
Regards,
Mark Busuttil


andrew at beekhof

May 6, 2012, 10:04 PM

Post #6 of 6 (530 views)
Permalink
Re: HAproxy resource not moving [In reply to]

On Tue, May 1, 2012 at 8:38 PM, Mark Busuttil <markbusu [at] gmail> wrote:
> Thank you all for your help, I have isolated the issue and it seems that its
> working correctly, it was miss configuration from my end, with the help of
> "crm configure property no-quorum-policy="ignore"
>
> We are nearly working as expected, however could someone confirm if the
> following assumption is correct:
>
> Node1 is running a resource and fails
> Node 2 starts running the resource
> Node 2 fails, the resource would not attempt to move onto Node 1, unless i
> executed:
>
> crm resource cleanup haproxy

Depends which action fails, how it fails (return code) and what the
operation's on-fail attribute is set to.

http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-ocf-return-codes.html
has some details on the return code aspects.

>
> Thanks in advance!
>
> On Mon, Apr 30, 2012 at 9:51 PM, Jake Smith <jsmith [at] argotec> wrote:
>>
>> What do you mean the cluster IP remains?
>>
>> No errors in the logs?
>>
>> Is the haproxy LSB compliant? see
>>
>> http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/ap-lsb.html
>>
>> Output of crm_mon -fr1?
>>
>> Jake
>>
>>
>>
>>
>> ________________________________
>>
>> From: "Mark Busuttil" <markbusu [at] gmail>
>> To: "Jake Smith" <jsmith [at] argotec>, pacemaker [at] oss
>> Sent: Monday, April 30, 2012 3:44:55 PM
>> Subject: Re: [Pacemaker] HAproxy resource not moving
>>
>>
>> Hmm unfortunately seems to be the same issue... the cluster IP seems to
>> remain... however the ha proxy resource just goes offline and does not
>> move.. thanks
>>
>> On Apr 30, 2012 8:16 PM, "Jake Smith" <jsmith [at] argotec> wrote:
>>>
>>>
>>> ----- Original Message -----
>>>
>>> > From: "Mark Busuttil" <markbusu [at] gmail>
>>> > To: pacemaker [at] oss
>>> > Sent: Monday, April 30, 2012 2:06:42 PM
>>> > Subject: [Pacemaker] HAproxy resource not moving
>>>
>>> > Hey guys,
>>>
>>> > I recently started testing out Pacemaker + Corosync for a clustering
>>> > solution for HA proxy. My configuration is as follows
>>>
>>> > crm(live)configure# show
>>> > node node1.local
>>> > node node2.local
>>> > primitive ClusterIP ocf:heartbeat:IPaddr2 \
>>> > params ip="********" cidr_netmask="32" \
>>> > op monitor interval="30s"
>>> > primitive test lsb:haproxy \
>>> > op monitor interval="10s"
>>> > colocation haproxy-with-public-IPs inf: test ClusterIP
>>> > order haproxy-after-IP inf: ClusterIP test
>>> > property $id="cib-bootstrap-options" \
>>> > dc-version="1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558" \
>>> > cluster-infrastructure="openais" \
>>> > expected-quorum-votes="2" \
>>> > stonith-enabled="false"
>>>
>>> Might be because you've lost quorum and the cluster has no directions
>>> without quorum... this might fix:
>>>
>>> crm configure property no-quorum-policy="ignore"
>>>
>>>
>>> HTH
>>>
>>> Jake
>>>
>>> > If both my nodes are up, the VIP responds as requested to the HA
>>> > proxy requests.. and seems to always stick to node2.
>>>
>>> > Online: [node2.local node1.local ]
>>>
>>> > ClusterIP (ocf::heartbeat:IPaddr2): Started node2.local
>>> > test (lsb:haproxy): Started node2.local
>>>
>>> > However, if i break HA proxy on node 2 or reboot that machine, the
>>> > cluster dies and doest move over to node 1 and the crm_mon displays:
>>>
>>> > Online: [node1.local ]
>>> > OFFLINE: [ node2.local ]
>>>
>>> > Not sure what im missing here, however i would assume that both
>>> > resources should move over to Node1..
>>>
>>> > Thanks in advance for any help you can provide. --
>>> > Regards,
>>> > Mark Busuttil
>>>
>>> > _______________________________________________
>>> > Pacemaker mailing list: Pacemaker [at] oss
>>> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> > Project Home: http://www.clusterlabs.org
>>> > Getting started:
>>> > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> > Bugs: http://bugs.clusterlabs.org
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker [at] oss
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker [at] oss
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>
>
>
> --
> Regards,
> Mark Busuttil
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker [at] oss
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Linux-HA pacemaker RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.