Mailing List Archive: Linux Virtual Server: Users

[lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Index | Next | Previous | View Threaded

mrv at tuv

Dec 3, 2010, 12:31 AM

Post #1 of 9 (774 views)
Permalink

[lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello list,

I hope you can help me.
I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.

I only use ldiretord/ipvs and an apache service on both machines.
My virtual IP address on interface lo:0 and real cluster IP on eth0.

Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
Do you agree?

I ask because if I do a tcpdump I see lots of traffic from the client browser to the server (http port) and the tcp
session will never end. ( client side has finished the transfer but on the server side tcpdump runs and runs....)

My config on both machines:
#cat ldirector.cf
autoreload = yes
checkinterval = 10
checktimeout = 3
logfile = "/var/log/ldirectord.log"
quiescent = yes
virtual = 10.10.11.60:80
fallback=127.0.0.1:80
real=10.10.11.61:80 gate
real=10.10.11.62:80 gate
service=http
request="test.html"
receive="Still alive"
scheduler=wlc
protocol=tcp
checktype=negotiate
network/ifcfg-lo:
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
IPADDR_2=127.0.0.2/8
STARTMODE=onboot
USERCONTROL=no
FIREWALL=no
IPADDR_0=10.10.11.60 #VIP
NETMASK_0=255.255.255.255
NETWORK_0=10.10.11.0
BROADCAST_0=10.10.11.255
LABEL_0=0

# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.10.11.60:80 wlc
-> 10.10.11.61:80 Local 1 0 0
-> 10.10.11.62:80 Route 1 0 177

Interesstingly on the other node it says # ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.10.11.60:80 wlc
-> 10.10.11.61:80 Route 1 0 177
-> 10.10.11.62:80 Local 1 0 0

So exactly the InActConn value is the same but the real IP addresses are changed.
I believe ldirectord shapes my requests endlessly.

After reading some documentation regarding to ARP announcements I configured the following in sysctl and did a reboot.
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.ip_forward = 1

In /var/log/ldirectord.log is nothing logged which is an error. Even if I start ldirectord in debug mode - no error.

Do anybody can explain me this story?

Thank you very much.

Best regards
Andrew
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

horms at verge

Dec 3, 2010, 1:39 AM

Post #2 of 9 (763 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> Hello list,
>
> I hope you can help me.
> I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.
>
> I only use ldiretord/ipvs and an apache service on both machines.
> My virtual IP address on interface lo:0 and real cluster IP on eth0.
>
> Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
> Do you agree?
>
> I ask because if I do a tcpdump I see lots of traffic from the client browser to the server (http port) and the tcp
> session will never end. ( client side has finished the transfer but on the server side tcpdump runs and runs....)

Hi Andreas,

if you want a pair of machines and you want one of them
to be the active linux director + a real server, and you want the other
to be the backup-linux director + a real server then yes it is
possible, but you need to take special care.

1. On the active linux director the VIP should be on lo and
On the backup-linux director the VIP should be on ethX
- That is, you need to move the VIP when a machine moves
from being active to backup and vice versa.
2. The backup-linux director must not have LVS rules in place for the
service being handled by the active linux director.
- That is, you need to have ldirectord running on only the active
linux director. Or in other words, you need to start it
when a machine becomes active and stop it when it becomes a backup.
3. You have to make sure that arp_ignore and arp_announce are set correctly.

Its a bit old, but I have some notes on this at
http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

(In relation to ARP you want the "Debian" notes unless
you have a really ancient kernel.)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

mrv at tuv

Dec 3, 2010, 2:08 AM

Post #3 of 9 (763 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Hello Simon,

thank you for your fast response! I am happy for every feedback.
Actually I thought I can have more than one active ldirector (for failover and ldirector load balancing).

Well, okay let's do ldirector with one node.
I refer to your Ultramonkey Howto.

----- On my first node I run ldirector and cluster node ---
eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on lo:0 butnot arp_ignore.
net.ipv4.conf.all.arp_announce = 1 # enable the feature
net.ipv4.conf.lo.arp_anounce = 1

----- On my second and n+1 node I run only cluster nodes ---
eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0 but not arp_announce.
net.ipv4.conf.all.arp_ignore = 1 # enable the feature
net.ipv4.conf.lo.arp_ignore = 1

Is this correct?

Thank for directing me into right direction.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Simon Horman [mailto:horms [at] verge]
Gesendet: Freitag, 3. Dezember 2010 10:39
An: Mrvka Andreas
Cc: 'lvs-users [at] linuxvirtualserver'
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> Hello list,
>
> I hope you can help me.
> I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.
>
> I only use ldiretord/ipvs and an apache service on both machines.
> My virtual IP address on interface lo:0 and real cluster IP on eth0.
>
> Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
> Do you agree?
>
> I ask because if I do a tcpdump I see lots of traffic from the client
> browser to the server (http port) and the tcp session will never end.
> ( client side has finished the transfer but on the server side tcpdump
> runs and runs....)

Hi Andreas,

if you want a pair of machines and you want one of them to be the active linux director + a real server, and you want the other to be the backup-linux director + a real server then yes it is possible, but you need to take special care.

1. On the active linux director the VIP should be on lo and
On the backup-linux director the VIP should be on ethX
- That is, you need to move the VIP when a machine moves
from being active to backup and vice versa.
2. The backup-linux director must not have LVS rules in place for the
service being handled by the active linux director.
- That is, you need to have ldirectord running on only the active
linux director. Or in other words, you need to start it
when a machine becomes active and stop it when it becomes a backup.
3. You have to make sure that arp_ignore and arp_announce are set correctly.

Its a bit old, but I have some notes on this at http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

(In relation to ARP you want the "Debian" notes unless you have a really ancient kernel.)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

mrv at tuv

Dec 3, 2010, 2:43 AM

Post #4 of 9 (759 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Hi,

I've misstped the arp_announce entries. It should be 2 instead of 1.
After a reboot it looks quite good having only one ldirector.

I am satisfied at the moment.
Novell HAE (High Availability Extension) shipps pacemaker.
I will try to configure it managing ldirector service on both machines but
only be explicitly active on one node.
And in the backup-case pacemaker should switch.

This sounds clear for me. But in fact, that the sysctl.cf file differs - how
should this be updated?

Thank you very much till now.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Mrvka Andreas
Gesendet: Freitag, 3. Dezember 2010 11:09
An: 'lvs-users [at] linuxvirtualserver'
Betreff: AW: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello Simon,

thank you for your fast response! I am happy for every feedback.
Actually I thought I can have more than one active ldirector (for failover and ldirector load balancing).

Well, okay let's do ldirector with one node.
I refer to your Ultramonkey Howto.

----- On my first node I run ldirector and cluster node ---
eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on lo:0 butnot arp_ignore.
net.ipv4.conf.all.arp_announce = 1 # enable the feature
net.ipv4.conf.lo.arp_anounce = 1

----- On my second and n+1 node I run only cluster nodes ---
eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0 but not arp_announce.
net.ipv4.conf.all.arp_ignore = 1 # enable the feature
net.ipv4.conf.lo.arp_ignore = 1

Is this correct?

Thank for directing me into right direction.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Simon Horman [mailto:horms [at] verge]
Gesendet: Freitag, 3. Dezember 2010 10:39
An: Mrvka Andreas
Cc: 'lvs-users [at] linuxvirtualserver'
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> Hello list,
>
> I hope you can help me.
> I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.
>
> I only use ldiretord/ipvs and an apache service on both machines.
> My virtual IP address on interface lo:0 and real cluster IP on eth0.
>
> Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
> Do you agree?
>
> I ask because if I do a tcpdump I see lots of traffic from the client
> browser to the server (http port) and the tcp session will never end.
> ( client side has finished the transfer but on the server side tcpdump
> runs and runs....)

Hi Andreas,

if you want a pair of machines and you want one of them to be the active linux director + a real server, and you want the other to be the backup-linux director + a real server then yes it is possible, but you need to take special care.

1. On the active linux director the VIP should be on lo and
On the backup-linux director the VIP should be on ethX
- That is, you need to move the VIP when a machine moves
from being active to backup and vice versa.
2. The backup-linux director must not have LVS rules in place for the
service being handled by the active linux director.
- That is, you need to have ldirectord running on only the active
linux director. Or in other words, you need to start it
when a machine becomes active and stop it when it becomes a backup.
3. You have to make sure that arp_ignore and arp_announce are set correctly.

Its a bit old, but I have some notes on this at http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

(In relation to ARP you want the "Debian" notes unless you have a really ancient kernel.)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jean-frederic.norguet at fnac

Dec 3, 2010, 2:55 AM

Post #5 of 9 (757 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Helloa Andew,
maybe this will help you:
http://www.novell.com/communities/node/4846/load-balancing-howto-lvs-
ldirectord-heartbeat-2
I used this howto for a cluster with 3 servers and ldirectord running in one
at a time.

--

Cordialement,
Norguet JF

Afin de contribuer au respect de l'environnement, merci de n'imprimer ce
courrier que si n�cessaire.

Le vendredi 3 d�cembre 2010, Mrvka Andreas a �crit :
> Hello Simon,
>
> thank you for your fast response! I am happy for every feedback.
> Actually I thought I can have more than one active ldirector (for failover
> and ldirector load balancing).
>
>
> Well, okay let's do ldirector with one node.
> I refer to your Ultramonkey Howto.
>
> ----- On my first node I run ldirector and cluster node ---
> eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on
> lo:0 butnot arp_ignore. net.ipv4.conf.all.arp_announce = 1 # enable the
> feature
> net.ipv4.conf.lo.arp_anounce = 1
>
> ----- On my second and n+1 node I run only cluster nodes ---
> eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0
> but not arp_announce. net.ipv4.conf.all.arp_ignore = 1 # enable the
> feature
> net.ipv4.conf.lo.arp_ignore = 1
>
>
> Is this correct?
>
> Thank for directing me into right direction.
> Andrew
>
>
> -----Urspr�ngliche Nachricht-----
> Von: Simon Horman [mailto:horms [at] verge]
> Gesendet: Freitag, 3. Dezember 2010 10:39
> An: Mrvka Andreas
> Cc: 'lvs-users [at] linuxvirtualserver'
> Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster
> server on same physical machine?
>
> On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> > Hello list,
> >
> > I hope you can help me.
> > I am using a simple 2 node cluster environment on SUSE Linux Enterprise
> > Server 11 SP1 HAE.
> >
> > I only use ldiretord/ipvs and an apache service on both machines.
> > My virtual IP address on interface lo:0 and real cluster IP on eth0.
> >
> > Regarding to the documentation of Novell it should be possible to have
> > ldirectord etc. on the same machines. Do you agree?
> >
> > I ask because if I do a tcpdump I see lots of traffic from the client
> > browser to the server (http port) and the tcp session will never end.
> > ( client side has finished the transfer but on the server side tcpdump
> > runs and runs....)
>
> Hi Andreas,
>
> if you want a pair of machines and you want one of them to be the active
> linux director + a real server, and you want the other to be the
> backup-linux director + a real server then yes it is possible, but you
> need to take special care.
>
> 1. On the active linux director the VIP should be on lo and
> On the backup-linux director the VIP should be on ethX
> - That is, you need to move the VIP when a machine moves
> from being active to backup and vice versa.
> 2. The backup-linux director must not have LVS rules in place for the
> service being handled by the active linux director.
> - That is, you need to have ldirectord running on only the active
> linux director. Or in other words, you need to start it
> when a machine becomes active and stop it when it becomes a backup.
> 3. You have to make sure that arp_ignore and arp_announce are set
> correctly.
>
> Its a bit old, but I have some notes on this at
> http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html
>
> (In relation to ARP you want the "Debian" notes unless you have a really
> ancient kernel.)
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Darren.Mansell at opengi

Dec 3, 2010, 3:33 AM

Post #6 of 9 (762 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Hello.

We have around 20 nodes of SLES 11 HAE servers running in 2 or 3 node clusters.

All of them are configured with local resources and ldirectord to load-balance to the local nodes with a highly available ldirectord resource being able to move on failover.

I can give more info if required.

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 10:43
To: 'lvs-users [at] linuxvirtualserver'
Subject: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hi,

I've misstped the arp_announce entries. It should be 2 instead of 1.
After a reboot it looks quite good having only one ldirector.

I am satisfied at the moment.
Novell HAE (High Availability Extension) shipps pacemaker.
I will try to configure it managing ldirector service on both machines but only be explicitly active on one node.
And in the backup-case pacemaker should switch.

This sounds clear for me. But in fact, that the sysctl.cf file differs - how should this be updated?

Thank you very much till now.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Mrvka Andreas
Gesendet: Freitag, 3. Dezember 2010 11:09
An: 'lvs-users [at] linuxvirtualserver'
Betreff: AW: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello Simon,

thank you for your fast response! I am happy for every feedback.
Actually I thought I can have more than one active ldirector (for failover and ldirector load balancing).

Well, okay let's do ldirector with one node.
I refer to your Ultramonkey Howto.

----- On my first node I run ldirector and cluster node ---
eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on lo:0 butnot arp_ignore.
net.ipv4.conf.all.arp_announce = 1 # enable the feature net.ipv4.conf.lo.arp_anounce = 1

----- On my second and n+1 node I run only cluster nodes ---
eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0 but not arp_announce.
net.ipv4.conf.all.arp_ignore = 1 # enable the feature net.ipv4.conf.lo.arp_ignore = 1

Is this correct?

Thank for directing me into right direction.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Simon Horman [mailto:horms [at] verge]
Gesendet: Freitag, 3. Dezember 2010 10:39
An: Mrvka Andreas
Cc: 'lvs-users [at] linuxvirtualserver'
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> Hello list,
>
> I hope you can help me.
> I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.
>
> I only use ldiretord/ipvs and an apache service on both machines.
> My virtual IP address on interface lo:0 and real cluster IP on eth0.
>
> Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
> Do you agree?
>
> I ask because if I do a tcpdump I see lots of traffic from the client
> browser to the server (http port) and the tcp session will never end.
> ( client side has finished the transfer but on the server side tcpdump
> runs and runs....)

Hi Andreas,

if you want a pair of machines and you want one of them to be the active linux director + a real server, and you want the other to be the backup-linux director + a real server then yes it is possible, but you need to take special care.

1. On the active linux director the VIP should be on lo and
On the backup-linux director the VIP should be on ethX
- That is, you need to move the VIP when a machine moves
from being active to backup and vice versa.
2. The backup-linux director must not have LVS rules in place for the
service being handled by the active linux director.
- That is, you need to have ldirectord running on only the active
linux director. Or in other words, you need to start it
when a machine becomes active and stop it when it becomes a backup.
3. You have to make sure that arp_ignore and arp_announce are set correctly.

Its a bit old, but I have some notes on this at http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

(In relation to ARP you want the "Debian" notes unless you have a really ancient kernel.)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

mrv at tuv

Dec 3, 2010, 3:35 AM

Post #7 of 9 (759 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Hi Jeff,

I know this page.
The picture in there shows that the ldirector have seperate IPs.
But I have ldirector directly on the real servers.
Furthermore I've bought the package SLES11 HAE and it shipps pacemaker instead of heartbeart.
So some parameters are not available anymore etc. etc.
And if I refer to this page and set the arp settings like described, the cluster does not respond anymore.

These are the facts I do not use the package but only ldirector at the moment.

As Simon explained me before - arp entries should look different on ldirector server and on real server.

Novell itself has not enought knowledge in their product to support me - this means quite a lot.

Best regards
Andrew

________________________________
Von: Jeff Norguet [mailto:jean-frederic.norguet [at] fnac]
Gesendet: Freitag, 3. Dezember 2010 11:56
An: lvs-users [at] linuxvirtualserver
Cc: Mrvka Andreas
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Helloa Andew,

maybe this will help you:

http://www.novell.com/communities/node/4846/load-balancing-howto-lvs-ldirectord-heartbeat-2

I used this howto for a cluster with 3 servers and ldirectord running in one at a time.

--

Cordialement,

Norguet JF

Afin de contribuer au respect de l'environnement, merci de n'imprimer ce courrier que si n�cessaire.

Le vendredi 3 d�cembre 2010, Mrvka Andreas a �crit :

> Hello Simon,

>

> thank you for your fast response! I am happy for every feedback.

> Actually I thought I can have more than one active ldirector (for failover

> and ldirector load balancing).

>

>

> Well, okay let's do ldirector with one node.

> I refer to your Ultramonkey Howto.

>

> ----- On my first node I run ldirector and cluster node ---

> eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on

> lo:0 butnot arp_ignore. net.ipv4.conf.all.arp_announce = 1 # enable the

> feature

> net.ipv4.conf.lo.arp_anounce = 1

>

> ----- On my second and n+1 node I run only cluster nodes ---

> eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0

> but not arp_announce. net.ipv4.conf.all.arp_ignore = 1 # enable the

> feature

> net.ipv4.conf.lo.arp_ignore = 1

>

>

> Is this correct?

>

> Thank for directing me into right direction.

> Andrew

>

>

> -----Urspr�ngliche Nachricht-----

> Von: Simon Horman [mailto:horms [at] verge]

> Gesendet: Freitag, 3. Dezember 2010 10:39

> An: Mrvka Andreas

> Cc: 'lvs-users [at] linuxvirtualserver'

> Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster

> server on same physical machine?

>

> On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:

> > Hello list,

> >

> > I hope you can help me.

> > I am using a simple 2 node cluster environment on SUSE Linux Enterprise

> > Server 11 SP1 HAE.

> >

> > I only use ldiretord/ipvs and an apache service on both machines.

> > My virtual IP address on interface lo:0 and real cluster IP on eth0.

> >

> > Regarding to the documentation of Novell it should be possible to have

> > ldirectord etc. on the same machines. Do you agree?

> >

> > I ask because if I do a tcpdump I see lots of traffic from the client

> > browser to the server (http port) and the tcp session will never end.

> > ( client side has finished the transfer but on the server side tcpdump

> > runs and runs....)

>

> Hi Andreas,

>

> if you want a pair of machines and you want one of them to be the active

> linux director + a real server, and you want the other to be the

> backup-linux director + a real server then yes it is possible, but you

> need to take special care.

>

> 1. On the active linux director the VIP should be on lo and

> On the backup-linux director the VIP should be on ethX

> - That is, you need to move the VIP when a machine moves

> from being active to backup and vice versa.

> 2. The backup-linux director must not have LVS rules in place for the

> service being handled by the active linux director.

> - That is, you need to have ldirectord running on only the active

> linux director. Or in other words, you need to start it

> when a machine becomes active and stop it when it becomes a backup.

> 3. You have to make sure that arp_ignore and arp_announce are set

> correctly.

>

> Its a bit old, but I have some notes on this at

> http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

>

> (In relation to ARP you want the "Debian" notes unless you have a really

> ancient kernel.)

>

> _______________________________________________

> Please read the documentation before posting - it's available at:

> http://www.linuxvirtualserver.org/

>

> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer

> Send requests to lvs-users-request [at] LinuxVirtualServer

> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

mrv at tuv

Dec 6, 2010, 12:43 AM

Post #8 of 9 (696 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Hello list,

sorrily I didn't succeed last week in deploying the cluster.
Please can anybody show me the error? It has to be somewhere very deep inside.

I only want to have a two node cluster with apache load balanced and fail-overing.
It shouldn't be that complex - but where is the error?

Maby the solution or this configs will help others.

Here my ldirectord.cf (with TABs)
autoreload = yes
checkinterval = 10
checktimeout = 3
logfile = "/var/log/ldirectord.log"
quiescent = yes
virtual = 10.10.11.60:80
checktype = negotiate
fallback = 127.0.0.1:80
protocol = tcp
real = 10.10.11.61:80 gate
real = 10.10.11.62:80 gate
receive = "Still alive"
request = "test.html"
scheduler = wlc
service = http

My crm configure:

node linlbtemp01
node linlbtemp02
primitive ClusterIP ocf:heartbeat:IPaddr2 \
operations $id="ClusterIP-operations" \
op monitor interval="5s" timeout="20s" \
params ip="10.10.11.60" nic="lo" cidr_netmask="16" lvs_support="true"
primitive Virtual-IP-Apache ocf:heartbeat:IPaddr2 \
params lvs_support="true" ip="10.10.11.60" cidr_netmask="16" broadcast="255.255.255.255" \
op monitor interval="1m" timeout="10s" \
meta migration-threshold="10"
primitive apache ocf:heartbeat:apache \
op monitor interval="30s" timeout="10s" \
meta migration-threshold="10" target-role="Started" \
params configfile="/etc/apache2/httpd.conf" httpd="/usr/sbin/httpd" testurl="/test.html"
primitive ldirectord ocf:heartbeat:ldirectord \
params configfile="/etc/ha.d/ldirectord.cf" \
op monitor interval="2m" timeout="20s" \
meta migration-threshold="10" target-role="Started"
group Load-Balancing Virtual-IP-Apache ldirectord
clone cl-apache apache
location Prefer-Node1 ldirectord \
rule $id="prefer-node1-rule" 100: #uname eq linlbtemp01
property $id="cib-bootstrap-options" \
dc-version="1.1.2-ecb1e2ea172ba2551f0bd763e557fccde68c849b" \
cluster-infrastructure="openais" \
expected-quorum-votes="2"

My /etc/sysctl:
# Disable response to broadcasts.
# You don't want yourself becoming a Smurf amplifier.
net.ipv4.icmp_echo_ignore_broadcasts = 1
# enable route verification on all interfaces
net.ipv4.conf.all.rp_filter = 1
# enable ipV6 forwarding
#net.ipv6.conf.all.forwarding = 1
# increase the number of possible inotify(7) watches
fs.inotify.max_user_watches = 65536
# avoid deleting secondary IPs on deleting the primary IP
#net.ipv4.conf.default.promote_secondaries = 1
#net.ipv4.conf.all.promote_secondaries = 1
#net.ipv4.conf.lo.arp_ignore = 1
#net.ipv4.conf.lo.arp_announce = 2
#net.ipv4.conf.all.arp_ignore = 1
#net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.ip_forward = 1

My ifcfg-lo:

IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
IPADDR_2=127.0.0.2/8
STARTMODE=onboot
USERCONTROL=no
FIREWALL=no
IPADDR_0=10.10.11.60 #VIP
NETMASK_0=255.255.255.255
NETWORK_0=10.10.11.0
BROADCAST_0=10.10.11.255
LABEL_0=0

Actually it seems, that my ldirectord out of openais does not start.

Can anybody point me to the error?

Thanks a lot in advance.
Andrew

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Darren.Mansell [at] opengi
Sent: Freitag, 3. Dezember 2010 14:53
To: lvs-users [at] linuxvirtualserver
Subject: Re: [lvs-users] is it possible to have ldirectorand realcluster server on same physical machine?

Glad it helped. This is my original howto for this kind of setup:

http://www.clusterlabs.org/wiki/Load_Balanced_MySQL_Replicated_Cluster

darren

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 13:46
To: 'LinuxVirtualServer.org users mailing list.'
Subject: Re: [lvs-users] is it possible to have ldirectorand realcluster server on same physical machine?

Hi Darren,

thank you for the detailed infos.
I've read out of your messages that in sysctl.conf is nothing special for the lo-interface.
This is a very important information! Thank you.

I know that there is a parameter no-quorum=ignore but I've already run into it something.
It would be nice to know how to restore quorum if it happens again (what I don't believe after your HowTo) because 2-node it just for testing and I will also add more nodes.

Thank you at this time.
I am motivated to finish building the cluster.

Regards
Andrew

-----Urspr�ngliche Nachricht-----
Von: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] Im Auftrag von Darren.Mansell [at] opengi
Gesendet: Freitag, 3. Dezember 2010 14:20
An: lvs-users [at] linuxvirtualserver
Betreff: Re: [lvs-users] is it possible to have ldirector and realcluster server on same physical machine?

1. Yes, ldirectord, cluster, real server, resources all on the same machine.

2. /etc/sysctl.conf is (appended to existing default) :
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_announce = 2

You will need to put non-arping interfaces for VIPs on all nodes:
MYSQL-01:~ # cat /etc/sysconfig/network/ifcfg-lo # Loopback (lo) configuration
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
IPADDR_2=127.0.0.2/8
STARTMODE=onboot
USERCONTROL=no
FIREWALL=no
IPADDR_VIP=10.167.20.100
NETMASK_VIP=255.255.255.255
NETWORK_VIP=10.167.20.0
BROADCAST_VIP=10.167.20.255
LABEL_VIP='VIP'

The cluster config for the VIP resource requires the lvs_support parameter set:
primitive Virtual-IP ocf:heartbeat:IPaddr2 \
params lvs_support="true" ip="10.167.20.100" cidr_netmask="24" broadcast="10.167.20.255" \
op monitor interval="1m" timeout="10s" \
meta migration-threshold="10"

3. STONITH is there for fencing, but failover happens when the node is unavailabe, or the resource hits the failcount limit depending on your own settings. The Pacemaker documentation at Clusterlabs is excellent for this.

4. Not sure I understand the question. Quorum is taken care of by OpenAIS. If you only have a 2-node cluster you need to set the CRM property 'no-quorum-policy' to 'ignore'.

Regards,
Darren

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 11:53
To: 'LinuxVirtualServer.org users mailing list.'
Subject: Re: [lvs-users] is it possible to have ldirector and realcluster server on same physical machine?

Hi Darren,

pleased to get notice of someone who uses the same product.
I have lots of questions ;-)

1) Do do you arrange ldirector and real cluster on the same machine?
Novell documentation do never go into detail so I believe it works.

2) How do you arrange sysctl configs (arp settings) in detail on all servers?

3) How do you arrange fail-over? via stonith ressource? how?

4) I often got the price to run into quorum (at pacemaker). How do you prevent it?
And if the cluster crashes (unexpected shutdown) how do you restore its sync?

Best regards
Andrew
GoogleTalk: andreas.mrvka [at] gmail

-----Urspr�ngliche Nachricht-----
Von: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] Im Auftrag von Darren.Mansell [at] opengi
Gesendet: Freitag, 3. Dezember 2010 12:34
An: lvs-users [at] linuxvirtualserver
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello.

We have around 20 nodes of SLES 11 HAE servers running in 2 or 3 node clusters.

All of them are configured with local resources and ldirectord to load-balance to the local nodes with a highly available ldirectord resource being able to move on failover.

I can give more info if required.

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 10:43
To: 'lvs-users [at] linuxvirtualserver'
Subject: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hi,

I've misstped the arp_announce entries. It should be 2 instead of 1.
After a reboot it looks quite good having only one ldirector.

I am satisfied at the moment.
Novell HAE (High Availability Extension) shipps pacemaker.
I will try to configure it managing ldirector service on both machines but only be explicitly active on one node.
And in the backup-case pacemaker should switch.

This sounds clear for me. But in fact, that the sysctl.cf file differs - how should this be updated?

Thank you very much till now.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Mrvka Andreas
Gesendet: Freitag, 3. Dezember 2010 11:09
An: 'lvs-users [at] linuxvirtualserver'
Betreff: AW: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello Simon,

thank you for your fast response! I am happy for every feedback.
Actually I thought I can have more than one active ldirector (for failover and ldirector load balancing).

Well, okay let's do ldirector with one node.
I refer to your Ultramonkey Howto.

----- On my first node I run ldirector and cluster node --- eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on lo:0 butnot arp_ignore.
net.ipv4.conf.all.arp_announce = 1 # enable the feature net.ipv4.conf.lo.arp_anounce = 1

----- On my second and n+1 node I run only cluster nodes --- eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0 but not arp_announce.
net.ipv4.conf.all.arp_ignore = 1 # enable the feature net.ipv4.conf.lo.arp_ignore = 1

Is this correct?

Thank for directing me into right direction.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Simon Horman [mailto:horms [at] verge]
Gesendet: Freitag, 3. Dezember 2010 10:39
An: Mrvka Andreas
Cc: 'lvs-users [at] linuxvirtualserver'
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> Hello list,
>
> I hope you can help me.
> I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.
>
> I only use ldiretord/ipvs and an apache service on both machines.
> My virtual IP address on interface lo:0 and real cluster IP on eth0.
>
> Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
> Do you agree?
>
> I ask because if I do a tcpdump I see lots of traffic from the client
> browser to the server (http port) and the tcp session will never end.
> ( client side has finished the transfer but on the server side tcpdump
> runs and runs....)

Hi Andreas,

if you want a pair of machines and you want one of them to be the active linux director + a real server, and you want the other to be the backup-linux director + a real server then yes it is possible, but you need to take special care.

1. On the active linux director the VIP should be on lo and
On the backup-linux director the VIP should be on ethX
- That is, you need to move the VIP when a machine moves
from being active to backup and vice versa.
2. The backup-linux director must not have LVS rules in place for the
service being handled by the active linux director.
- That is, you need to have ldirectord running on only the active
linux director. Or in other words, you need to start it
when a machine becomes active and stop it when it becomes a backup.
3. You have to make sure that arp_ignore and arp_announce are set correctly.

Its a bit old, but I have some notes on this at http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

(In relation to ARP you want the "Debian" notes unless you have a really ancient kernel.)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Darren.Mansell at opengi

Dec 6, 2010, 4:00 AM

Post #9 of 9 (688 views)
Permalink

Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine? [In reply to]

Check the /var/log/ldirectord.log file for errors and check you can manually start it yourself:

rcldirectord restart

I've had to compile a Perl module myself for ldirector in SLES 11 HAE: http://www.clusterlabs.org/wiki/Load_Balanced_MySQL_Replicated_Cluster#Missing_Perl_Socket6

You also need lvs_support="true" in your ldirectord resource.

I've added this to the pacemaker list as it may be more suited for support there.

Darren Mansell

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 06 December 2010 08:44
To: LinuxVirtualServer.org users mailing list.
Subject: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello list,

sorrily I didn't succeed last week in deploying the cluster.
Please can anybody show me the error? It has to be somewhere very deep inside.

I only want to have a two node cluster with apache load balanced and fail-overing.
It shouldn't be that complex - but where is the error?

Maby the solution or this configs will help others.

Here my ldirectord.cf (with TABs)
autoreload = yes
checkinterval = 10
checktimeout = 3
logfile = "/var/log/ldirectord.log"
quiescent = yes
virtual = 10.10.11.60:80
checktype = negotiate
fallback = 127.0.0.1:80
protocol = tcp
real = 10.10.11.61:80 gate
real = 10.10.11.62:80 gate
receive = "Still alive"
request = "test.html"
scheduler = wlc
service = http

My crm configure:

node linlbtemp01
node linlbtemp02
primitive ClusterIP ocf:heartbeat:IPaddr2 \
operations $id="ClusterIP-operations" \
op monitor interval="5s" timeout="20s" \
params ip="10.10.11.60" nic="lo" cidr_netmask="16" lvs_support="true"
primitive Virtual-IP-Apache ocf:heartbeat:IPaddr2 \
params lvs_support="true" ip="10.10.11.60" cidr_netmask="16" broadcast="255.255.255.255" \
op monitor interval="1m" timeout="10s" \
meta migration-threshold="10"
primitive apache ocf:heartbeat:apache \
op monitor interval="30s" timeout="10s" \
meta migration-threshold="10" target-role="Started" \
params configfile="/etc/apache2/httpd.conf" httpd="/usr/sbin/httpd" testurl="/test.html"
primitive ldirectord ocf:heartbeat:ldirectord \
params configfile="/etc/ha.d/ldirectord.cf" \
op monitor interval="2m" timeout="20s" \
meta migration-threshold="10" target-role="Started"
group Load-Balancing Virtual-IP-Apache ldirectord clone cl-apache apache location Prefer-Node1 ldirectord \
rule $id="prefer-node1-rule" 100: #uname eq linlbtemp01 property $id="cib-bootstrap-options" \
dc-version="1.1.2-ecb1e2ea172ba2551f0bd763e557fccde68c849b" \
cluster-infrastructure="openais" \
expected-quorum-votes="2"

My /etc/sysctl:
# Disable response to broadcasts.
# You don't want yourself becoming a Smurf amplifier.
net.ipv4.icmp_echo_ignore_broadcasts = 1 # enable route verification on all interfaces net.ipv4.conf.all.rp_filter = 1 # enable ipV6 forwarding #net.ipv6.conf.all.forwarding = 1 # increase the number of possible inotify(7) watches fs.inotify.max_user_watches = 65536 # avoid deleting secondary IPs on deleting the primary IP #net.ipv4.conf.default.promote_secondaries = 1 #net.ipv4.conf.all.promote_secondaries = 1 #net.ipv4.conf.lo.arp_ignore = 1 #net.ipv4.conf.lo.arp_announce = 2 #net.ipv4.conf.all.arp_ignore = 1 #net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.eth0.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.eth0.arp_announce = 2 net.ipv4.ip_forward = 1

My ifcfg-lo:

IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
IPADDR_2=127.0.0.2/8
STARTMODE=onboot
USERCONTROL=no
FIREWALL=no
IPADDR_0=10.10.11.60 #VIP
NETMASK_0=255.255.255.255
NETWORK_0=10.10.11.0
BROADCAST_0=10.10.11.255
LABEL_0=0

Actually it seems, that my ldirectord out of openais does not start.

Can anybody point me to the error?

Thanks a lot in advance.
Andrew

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Darren.Mansell [at] opengi
Sent: Freitag, 3. Dezember 2010 14:53
To: lvs-users [at] linuxvirtualserver
Subject: Re: [lvs-users] is it possible to have ldirectorand realcluster server on same physical machine?

Glad it helped. This is my original howto for this kind of setup:

http://www.clusterlabs.org/wiki/Load_Balanced_MySQL_Replicated_Cluster

darren

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 13:46
To: 'LinuxVirtualServer.org users mailing list.'
Subject: Re: [lvs-users] is it possible to have ldirectorand realcluster server on same physical machine?

Hi Darren,

thank you for the detailed infos.
I've read out of your messages that in sysctl.conf is nothing special for the lo-interface.
This is a very important information! Thank you.

I know that there is a parameter no-quorum=ignore but I've already run into it something.
It would be nice to know how to restore quorum if it happens again (what I don't believe after your HowTo) because 2-node it just for testing and I will also add more nodes.

Thank you at this time.
I am motivated to finish building the cluster.

Regards
Andrew

-----Urspr�ngliche Nachricht-----
Von: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] Im Auftrag von Darren.Mansell [at] opengi
Gesendet: Freitag, 3. Dezember 2010 14:20
An: lvs-users [at] linuxvirtualserver
Betreff: Re: [lvs-users] is it possible to have ldirector and realcluster server on same physical machine?

1. Yes, ldirectord, cluster, real server, resources all on the same machine.

2. /etc/sysctl.conf is (appended to existing default) :
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_announce = 2

You will need to put non-arping interfaces for VIPs on all nodes:
MYSQL-01:~ # cat /etc/sysconfig/network/ifcfg-lo # Loopback (lo) configuration
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
IPADDR_2=127.0.0.2/8
STARTMODE=onboot
USERCONTROL=no
FIREWALL=no
IPADDR_VIP=10.167.20.100
NETMASK_VIP=255.255.255.255
NETWORK_VIP=10.167.20.0
BROADCAST_VIP=10.167.20.255
LABEL_VIP='VIP'

The cluster config for the VIP resource requires the lvs_support parameter set:
primitive Virtual-IP ocf:heartbeat:IPaddr2 \
params lvs_support="true" ip="10.167.20.100" cidr_netmask="24" broadcast="10.167.20.255" \
op monitor interval="1m" timeout="10s" \
meta migration-threshold="10"

3. STONITH is there for fencing, but failover happens when the node is unavailabe, or the resource hits the failcount limit depending on your own settings. The Pacemaker documentation at Clusterlabs is excellent for this.

4. Not sure I understand the question. Quorum is taken care of by OpenAIS. If you only have a 2-node cluster you need to set the CRM property 'no-quorum-policy' to 'ignore'.

Regards,
Darren

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 11:53
To: 'LinuxVirtualServer.org users mailing list.'
Subject: Re: [lvs-users] is it possible to have ldirector and realcluster server on same physical machine?

Hi Darren,

pleased to get notice of someone who uses the same product.
I have lots of questions ;-)

1) Do do you arrange ldirector and real cluster on the same machine?
Novell documentation do never go into detail so I believe it works.

2) How do you arrange sysctl configs (arp settings) in detail on all servers?

3) How do you arrange fail-over? via stonith ressource? how?

4) I often got the price to run into quorum (at pacemaker). How do you prevent it?
And if the cluster crashes (unexpected shutdown) how do you restore its sync?

Best regards
Andrew
GoogleTalk: andreas.mrvka [at] gmail

-----Urspr�ngliche Nachricht-----
Von: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] Im Auftrag von Darren.Mansell [at] opengi
Gesendet: Freitag, 3. Dezember 2010 12:34
An: lvs-users [at] linuxvirtualserver
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello.

We have around 20 nodes of SLES 11 HAE servers running in 2 or 3 node clusters.

All of them are configured with local resources and ldirectord to load-balance to the local nodes with a highly available ldirectord resource being able to move on failover.

I can give more info if required.

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Mrvka Andreas
Sent: 03 December 2010 10:43
To: 'lvs-users [at] linuxvirtualserver'
Subject: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hi,

I've misstped the arp_announce entries. It should be 2 instead of 1.
After a reboot it looks quite good having only one ldirector.

I am satisfied at the moment.
Novell HAE (High Availability Extension) shipps pacemaker.
I will try to configure it managing ldirector service on both machines but only be explicitly active on one node.
And in the backup-case pacemaker should switch.

This sounds clear for me. But in fact, that the sysctl.cf file differs - how should this be updated?

Thank you very much till now.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Mrvka Andreas
Gesendet: Freitag, 3. Dezember 2010 11:09
An: 'lvs-users [at] linuxvirtualserver'
Betreff: AW: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

Hello Simon,

thank you for your fast response! I am happy for every feedback.
Actually I thought I can have more than one active ldirector (for failover and ldirector load balancing).

Well, okay let's do ldirector with one node.
I refer to your Ultramonkey Howto.

----- On my first node I run ldirector and cluster node --- eth0 is real IP and lo:0 is virtual IP. So I do have to arp_announce on lo:0 butnot arp_ignore.
net.ipv4.conf.all.arp_announce = 1 # enable the feature net.ipv4.conf.lo.arp_anounce = 1

----- On my second and n+1 node I run only cluster nodes --- eth0 is real IP and lo:0 is virtual IP. So I do have to arp_ignore on lo:0 but not arp_announce.
net.ipv4.conf.all.arp_ignore = 1 # enable the feature net.ipv4.conf.lo.arp_ignore = 1

Is this correct?

Thank for directing me into right direction.
Andrew

-----Urspr�ngliche Nachricht-----
Von: Simon Horman [mailto:horms [at] verge]
Gesendet: Freitag, 3. Dezember 2010 10:39
An: Mrvka Andreas
Cc: 'lvs-users [at] linuxvirtualserver'
Betreff: Re: [lvs-users] is it possible to have ldirector and real cluster server on same physical machine?

On Fri, Dec 03, 2010 at 09:31:19AM +0100, Mrvka Andreas wrote:
> Hello list,
>
> I hope you can help me.
> I am using a simple 2 node cluster environment on SUSE Linux Enterprise Server 11 SP1 HAE.
>
> I only use ldiretord/ipvs and an apache service on both machines.
> My virtual IP address on interface lo:0 and real cluster IP on eth0.
>
> Regarding to the documentation of Novell it should be possible to have ldirectord etc. on the same machines.
> Do you agree?
>
> I ask because if I do a tcpdump I see lots of traffic from the client
> browser to the server (http port) and the tcp session will never end.
> ( client side has finished the transfer but on the server side tcpdump
> runs and runs....)

Hi Andreas,

if you want a pair of machines and you want one of them to be the active linux director + a real server, and you want the other to be the backup-linux director + a real server then yes it is possible, but you need to take special care.

1. On the active linux director the VIP should be on lo and
On the backup-linux director the VIP should be on ethX
- That is, you need to move the VIP when a machine moves
from being active to backup and vice versa.
2. The backup-linux director must not have LVS rules in place for the
service being handled by the active linux director.
- That is, you need to have ldirectord running on only the active
linux director. Or in other words, you need to start it
when a machine becomes active and stop it when it becomes a backup.
3. You have to make sure that arp_ignore and arp_announce are set correctly.

Its a bit old, but I have some notes on this at http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html

(In relation to ARP you want the "Debian" notes unless you have a really ancient kernel.)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads