Mailing List Archive: Linux Virtual Server: Users

[lvs-users] Mysterious documentation

Index | Next | Previous | View Threaded

steiny at infopoint

May 19, 2009, 11:27 PM

Post #1 of 8 (1167 views)
Permalink

[lvs-users] Mysterious documentation

I found this recent and comprehensive documentation that is missing one
important thing. It tells that we need add a VIP, but to what? I have
machines that have 2 ethernet cards. On are on an internal subnet
172.21.4.32 and so on, and the others go though the router to the
outside: 66.124.8.1 and so on. When it says "set up a VIP" and that it
can be "pinged from the outside" I am totally lost. Do I do an ifconfig
on eth0:1 or something like that? What ip address should I use? If I
make it part of the internal network, then I can't get to it from the
outside, if I make it part of the external, then the machines inside
can't see it. HELP!!

Don (steiny [at] infopoint)

Adding the VIP to the load balancer requires no special configuration
apart from adding a virtual address. In part two of this article series,
the heartbeat program will be adding and removing this address as a
configured “resource,” but at present you will configure it manually. It
is important that you set up the VIP in such a way that the default
route out of the machine is still via the primary address (the RIP).
This is done by defining the subnet mask to be 255.255.255.255 (32 in
CIDR notation). Set it up as an additional address on |eth0|.

When adding the VIP to the nodes, it is essential that the IP address is
unresolvable to the network via ARP. If it were, the load balancer would
become unreachable. In order to hide the address, you need to set some
kernel “sysctl” parameters by editing |/etc/sysctl.conf|. Look in your
distribution’s documentation to confirm this file is not auto-generated
from other files or by a configuration utility. Set the following
parameters: [3
<http://tag1consulting.com/Scalable_Linux_Clusters_with_LVS_Part_I#3>]

|net.ipv4.conf.all.arp_ignore = 1|
|net.ipv4.conf.all.arp_announce = 2|

This ensures that interfaces will only answer ARP requests for IP
addresses that belong to them, as opposed to all IP addresses on the
machine. For example, if the VIP is a virtual address on the loopback
device (|lo|), then the RIP (|eth0|) will not advertise it. Run |sysctl
-p| as root, or, if you are familiar with it, use the |/proc/sys/|
interface to set these values.

Now that you have set these parameters, you may add the VIP to |lo|.
This will be similar to configuring the VIP on the load balancer, except
that the addiional address is for |lo|, not |eth0|. Again, ensure that
the netmask of the address is 255.255.255.255.

Time to test. The service you are running on the nodes must be
configured to listen on both the RIP and VIP addresses. Assuming your
firewall policy allows pings, you should still be able to ping the RIP
of each node from a third-party machine unrelated to the load balancer
setup. Next, try pinging the RIP of each node from the load balancer;
connectivity to the node from the load balancer will be necessary once
you configure the load balancers to check the nodes for availability.

Lastly, pinging the VIP from off-network should result in a response
from the load balancer.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

keijser at stone-it

May 20, 2009, 5:02 AM

Post #2 of 8 (1121 views)
Permalink

Re: [lvs-users] Mysterious documentation [In reply to]

Hi,

On Tue, 2009-05-19 at 23:27 -0700, Don Steiny wrote:
> I found this recent and comprehensive documentation that is missing one
> important thing. It tells that we need add a VIP, but to what? I have
> machines that have 2 ethernet cards. On are on an internal subnet
> 172.21.4.32 and so on, and the others go though the router to the
> outside: 66.124.8.1 and so on. When it says "set up a VIP" and that it
> can be "pinged from the outside" I am totally lost. Do I do an ifconfig
> on eth0:1 or something like that? What ip address should I use? If I
> make it part of the internal network, then I can't get to it from the
> outside, if I make it part of the external, then the machines inside
> can't see it. HELP!!

Think logically: if you need a VIP that can be reached from 'outside',
it must be a public address, right? Also you don't mention which method
of LVS you're going to use.

Anyway, you'll need a public VIP and an internal VIP in the case of NAT:

[client] -> [director] -> [realserver] -> [director] -> [client]

And set the default gateway of your realserver to point to the internal
VIP on the director to guarantee the packets travels back through the
director to the client.

If you use DR you'll need only a public VIP:

[client] -> [director] -> [realserver] -> [client]

The default gateway on the realserver can be set to its 'normal' value.

Btw, all this is documented in the (mini-)HOWTO. There are even
ready-to-run examples of DR/NAT that you can modify. I suggest you
really read the HOWTO again because if this boggles you, you're in for a
treat when it comes to the ARP problem ;)

Good luck

--
LÃ©on

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

May 20, 2009, 5:28 AM

Post #3 of 8 (1111 views)
Permalink

Re: [lvs-users] Mysterious documentation [In reply to]

On Tue, 19 May 2009, Don Steiny wrote:

> I found this recent and comprehensive documentation that is missing one
> important thing. It tells that we need add a VIP, but to what?

to a NIC on the director that advertises the VIP to the
router.

> Do I do an ifconfig on eth0:1 or something like that?

read about iproute2 tools in the HOWTO

> What ip address should I use?

the same IP you used to connect to the single machine that
preceded the LVS.

> If I make it part of the internal network, then I can't
> get to it from the outside, if I make it part of the
> external, then the machines inside can't see it. HELP!!

the realservers don't ever need to know about the VIP on the
director

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

steiny at infopoint

May 20, 2009, 6:53 AM

Post #4 of 8 (1109 views)
Permalink

Re: [lvs-users] Mysterious documentation [In reply to]

I am doing DR and the ARP problem makes sense. The part I still don't
understand is "I need a public and private VIP." The ISP I am using has
assigned me some IP addresses. I know how to use ifconfig to put
multiple addresses on an interface, that is no big deal, but what I
can't figure out is how to decide what IP address to use for the VIP. I
have looked up VIP and found many explanations, including ones that tie
it to lo, ones that say that it does not need to be tied to an interface
(and I don't know how to do that). Unless it is one of the ones my ISP
lets me use, then no one can get to me from the outside. I need to both
configure the machine and use it from outside the data center. I have
an IP address that I go into with ssh. If that is my RIP, then what
address should my VIP be? Should I request an address from my ISP that
I can use for the VIP and configure it on eth0? Do I get an address
from the ISP and have them configure the router so people can get to it?

-Don
> Hi,
>
> On Tue, 2009-05-19 at 23:27 -0700, Don Steiny wrote:
>
>> I found this recent and comprehensive documentation that is missing one
>> important thing. It tells that we need add a VIP, but to what? I have
>> machines that have 2 ethernet cards. On are on an internal subnet
>> 172.21.4.32 and so on, and the others go though the router to the
>> outside: 66.124.8.1 and so on. When it says "set up a VIP" and that it
>> can be "pinged from the outside" I am totally lost. Do I do an ifconfig
>> on eth0:1 or something like that? What ip address should I use? If I
>> make it part of the internal network, then I can't get to it from the
>> outside, if I make it part of the external, then the machines inside
>> can't see it. HELP!!
>>
>
> Think logically: if you need a VIP that can be reached from 'outside',
> it must be a public address, right? Also you don't mention which method
> of LVS you're going to use.
>
> Anyway, you'll need a public VIP and an internal VIP in the case of NAT:
>
> [client] -> [director] -> [realserver] -> [director] -> [client]
>
> And set the default gateway of your realserver to point to the internal
> VIP on the director to guarantee the packets travels back through the
> director to the client.
>
> If you use DR you'll need only a public VIP:
>
> [client] -> [director] -> [realserver] -> [client]
>
> The default gateway on the realserver can be set to its 'normal' value.
>
> Btw, all this is documented in the (mini-)HOWTO. There are even
> ready-to-run examples of DR/NAT that you can modify. I suggest you
> really read the HOWTO again because if this boggles you, you're in for a
> treat when it comes to the ARP problem ;)
>
> Good luck
>
>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

keijser at stone-it

May 20, 2009, 7:06 AM

Post #5 of 8 (1116 views)
Permalink

Re: [lvs-users] Mysterious documentation [In reply to]

Hi,

On Wed, 2009-05-20 at 06:53 -0700, Don Steiny wrote:
> I am doing DR and the ARP problem makes sense. The part I still don't
> understand is "I need a public and private VIP." The ISP I am using has
> assigned me some IP addresses. I know how to use ifconfig to put
> multiple addresses on an interface, that is no big deal, but what I
> can't figure out is how to decide what IP address to use for the VIP.

Pick any public IP you have that's not being used for anything yet. You
say you have multiple IP addresses assigned to you by your ISP so that's
great. Use one for the IP that isn't already used.

> I
> have looked up VIP and found many explanations, including ones that tie
> it to lo, ones that say that it does not need to be tied to an interface
> (and I don't know how to do that). Unless it is one of the ones my ISP
> lets me use, then no one can get to me from the outside. I need to both
> configure the machine and use it from outside the data center. I have
> an IP address that I go into with ssh. If that is my RIP, then what
> address should my VIP be? Should I request an address from my ISP that
> I can use for the VIP and configure it on eth0? Do I get an address
> from the ISP and have them configure the router so people can get to it?

Since you're going to use DR, you'll need to configure a loopback device
on your realservers (provided you're using a fairly recent kernel - see
the HOWTO for other solutions if this is not the case) that has the VIP
(the public yes, you're not using anything else on the director)
configured with a netmask of /32 and solve the arp problem (again look
in the HOWTO for methods per kernel) if they are running Linux.

This way the packets destined for the VIP get handled by the director
and sent to one of the realservers configured in the LVS table. The
realserver gets the Link Layer packet, sees its destination (the VIP),
recognizes that it has that ip configured (the loopback device), handles
the request (eg. a http request) and sends the reply back to the
src_addr (the client).

--
LÃ©on

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

steiny at infopoint

May 20, 2009, 9:06 AM

Post #6 of 8 (1106 views)
Permalink

Re: [lvs-users] Mysterious documentation [In reply to]

THANKS! That is just what I needed to know!

-Don
> Hi,
>
> On Wed, 2009-05-20 at 06:53 -0700, Don Steiny wrote:
>
>> I am doing DR and the ARP problem makes sense. The part I still don't
>> understand is "I need a public and private VIP." The ISP I am using has
>> assigned me some IP addresses. I know how to use ifconfig to put
>> multiple addresses on an interface, that is no big deal, but what I
>> can't figure out is how to decide what IP address to use for the VIP.
>>
>
> Pick any public IP you have that's not being used for anything yet. You
> say you have multiple IP addresses assigned to you by your ISP so that's
> great. Use one for the IP that isn't already used.
>
>
>> I
>> have looked up VIP and found many explanations, including ones that tie
>> it to lo, ones that say that it does not need to be tied to an interface
>> (and I don't know how to do that). Unless it is one of the ones my ISP
>> lets me use, then no one can get to me from the outside. I need to both
>> configure the machine and use it from outside the data center. I have
>> an IP address that I go into with ssh. If that is my RIP, then what
>> address should my VIP be? Should I request an address from my ISP that
>> I can use for the VIP and configure it on eth0? Do I get an address
>> from the ISP and have them configure the router so people can get to it?
>>
>
> Since you're going to use DR, you'll need to configure a loopback device
> on your realservers (provided you're using a fairly recent kernel - see
> the HOWTO for other solutions if this is not the case) that has the VIP
> (the public yes, you're not using anything else on the director)
> configured with a netmask of /32 and solve the arp problem (again look
> in the HOWTO for methods per kernel) if they are running Linux.
>
> This way the packets destined for the VIP get handled by the director
> and sent to one of the realservers configured in the LVS table. The
> realserver gets the Link Layer packet, sees its destination (the VIP),
> recognizes that it has that ip configured (the loopback device), handles
> the request (eg. a http request) and sends the reply back to the
> src_addr (the client).
>
>
>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

steiny at infopoint

May 20, 2009, 9:15 AM

Post #7 of 8 (1121 views)
Permalink

Re: [lvs-users] Mysterious documentation - what if the real servers are Windows? I know I can add a loopback device to them too. Any how-toos you would recommend? [In reply to]

Thanks so much. The realservers are on both a private and public net.
Should I use their public IPs? I guess I will have to with DR because
DR sends the stuff right back to them, correct? Also the realservers
are Windows 2003. I know I can add a loopback device to them. Are
there any good how tos about doing this?

-Don
> Hi,
>
> On Wed, 2009-05-20 at 06:53 -0700, Don Steiny wrote:
>
>> I am doing DR and the ARP problem makes sense. The part I still don't
>> understand is "I need a public and private VIP." The ISP I am using has
>> assigned me some IP addresses. I know how to use ifconfig to put
>> multiple addresses on an interface, that is no big deal, but what I
>> can't figure out is how to decide what IP address to use for the VIP.
>>
>
> Pick any public IP you have that's not being used for anything yet. You
> say you have multiple IP addresses assigned to you by your ISP so that's
> great. Use one for the IP that isn't already used.
>
>
>> I
>> have looked up VIP and found many explanations, including ones that tie
>> it to lo, ones that say that it does not need to be tied to an interface
>> (and I don't know how to do that). Unless it is one of the ones my ISP
>> lets me use, then no one can get to me from the outside. I need to both
>> configure the machine and use it from outside the data center. I have
>> an IP address that I go into with ssh. If that is my RIP, then what
>> address should my VIP be? Should I request an address from my ISP that
>> I can use for the VIP and configure it on eth0? Do I get an address
>> from the ISP and have them configure the router so people can get to it?
>>
>
> Since you're going to use DR, you'll need to configure a loopback device
> on your realservers (provided you're using a fairly recent kernel - see
> the HOWTO for other solutions if this is not the case) that has the VIP
> (the public yes, you're not using anything else on the director)
> configured with a netmask of /32 and solve the arp problem (again look
> in the HOWTO for methods per kernel) if they are running Linux.
>
> This way the packets destined for the VIP get handled by the director
> and sent to one of the realservers configured in the LVS table. The
> realserver gets the Link Layer packet, sees its destination (the VIP),
> recognizes that it has that ip configured (the loopback device), handles
> the request (eg. a http request) and sends the reply back to the
> src_addr (the client).
>
>
>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

keijser at stone-it

May 20, 2009, 11:14 PM

Post #8 of 8 (1099 views)
Permalink

Re: [lvs-users] Mysterious documentation - what if the real servers are Windows? I know I can add a loopback device to them too. Any how-toos you would recommend? [In reply to]

On Wed, 2009-05-20 at 09:15 -0700, Don Steiny wrote:
> Thanks so much. The realservers are on both a private and public net.
> Should I use their public IPs? I guess I will have to with DR because
> DR sends the stuff right back to them, correct?

correct

> Also the realservers
> are Windows 2003. I know I can add a loopback device to them. Are
> there any good how tos about doing this?

I guess if you know how to set the loopback device, it's all you'll need
to know to configure the realserver for use with LVS-DR. Just keep in
mind to set the metric of the loopback device to 254.

--
LÃ©on

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads