Mailing List Archive: Linux Virtual Server: Users

[lvs-users] LVS-DR problem.

Index | Next | Previous | View Threaded

vasco.postfix at gmail

Feb 19, 2009, 10:35 PM

Post #1 of 6 (975 views)
Permalink

[lvs-users] LVS-DR problem.

Hi,
I am having following configuration of LVS Cluster.
All servers in the following configuration have Public IP and can access
internet.

I want to setup Linux-DR

Master/ Director
IP address = 192.168.0.1
Gateway = 192.168.0.254
VIP = 192.168.2.1

Realserver 1
IP Address = 192.168.2.2
Gateway = 192.168.2.254
Netmask = 255.255.255.224

Realserver 2
IP Address = 192.168.2.3
Gateway = 192.168.2.254
Netmask = 255.255.255.224

All the systems are CENTOS 5.2

I enabled packet forwarding on Director using
also disabled arp request

My sysctrl -p output on realservers is as follows
[root [at] localhos ~]# sysctl -p
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

I configured VIP on Director using following command
ifconfig eth1 192.168.2.1 broadcast 192.168.2.255 netmask 255.255.0.0

I am running smtp service on both the realservers,

Did all stuff with ipvsadm

but still not able to telnet 192.168.2.1 25

it gives me

Trying 216.185.60.11...
telnet: Unable to connect to remote host: No route to host

but I am able to ping 216.185.60.1

Not able to find out the prob,

Thanx in adv.

Meghanand Acharekar
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

xufengnju at sina

Feb 19, 2009, 11:20 PM

Post #2 of 6 (924 views)
Permalink

Re: [lvs-users] LVS-DR problem. [In reply to]

Hi

In LVS with Direct Routing(DR mode),you must configure the VIP on
loadbalancer and realservers with 32 bits netmask.
You may refer to my picture of LVS/DR structure on
http://blog.chinaunix.net/u2/74751/showart_1834194.html
I think you can figure out what should your configuration would be in your
case.
On real servers put the following lines in your /etc/sysctl.conf

net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

To resole arp issues with LVS/DR.

Yours
XU FENG
From China.

> -----Original Message-----
> From: lvs-users-bounces [at] linuxvirtualserver
> [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Meghanand
> Acharekar
> Sent: 2009年2月20日 14:36
> To: lvs-users [at] linuxvirtualserver
> Subject: [lvs-users] LVS-DR problem.
>
> Hi,
> I am having following configuration of LVS Cluster.
> All servers in the following configuration have Public IP and can access
> internet.
>
> I want to setup Linux-DR
>
> Master/ Director
> IP address = 192.168.0.1
> Gateway = 192.168.0.254
> VIP = 192.168.2.1
>
> Realserver 1
> IP Address = 192.168.2.2
> Gateway = 192.168.2.254
> Netmask = 255.255.255.224
>
> Realserver 2
> IP Address = 192.168.2.3
> Gateway = 192.168.2.254
> Netmask = 255.255.255.224
>
>
> All the systems are CENTOS 5.2
>
> I enabled packet forwarding on Director using
> also disabled arp request
>
> My sysctrl -p output on realservers is as follows
> [root [at] localhos ~]# sysctl -p
> net.ipv4.ip_forward = 1
> net.ipv4.conf.default.rp_filter = 1
> net.ipv4.conf.default.accept_source_route = 0
> kernel.sysrq = 0
> kernel.core_uses_pid = 1
> net.ipv4.tcp_syncookies = 1
> kernel.msgmnb = 65536
> kernel.msgmax = 65536
> kernel.shmmax = 4294967295
> kernel.shmall = 268435456
> net.ipv4.conf.eth0.arp_ignore = 1
> net.ipv4.conf.eth0.arp_announce = 2
> net.ipv4.conf.all.arp_ignore = 1
> net.ipv4.conf.all.arp_announce = 2
>
> I configured VIP on Director using following command
> ifconfig eth1 192.168.2.1 broadcast 192.168.2.255 netmask 255.255.0.0
>
> I am running smtp service on both the realservers,
>
> Did all stuff with ipvsadm
>
> but still not able to telnet 192.168.2.1 25
>
> it gives me
>
> Trying 216.185.60.11...
> telnet: Unable to connect to remote host: No route to host
>
> but I am able to ping 216.185.60.1
>
> Not able to find out the prob,
>
> Thanx in adv.
>
> Meghanand Acharekar
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

toanilsharma1 at gmail

Feb 19, 2009, 11:52 PM

Post #3 of 6 (928 views)
Permalink

Re: [lvs-users] LVS-DR problem. [In reply to]

Have you check your firewall configuration.? direct routing is sometimes not
permitted by the network setup, it assumes DR like IP spoofing.
I tried DR but didn't worked. but sure TUNULING would work. remember all the
servers under same Router. otherwise your packet would be droped at the
forigen router.

Configure your Ldirector this way

negotiatetimeout=10
checktimeout=10
checkinterval=30
checkcount=10
autoreload=no
logfile="/var/log/ldirectord.log"

virtual=yourVIP:443
real=RS1IP:443 ipip ## this is IPIP incapsulation
real=RS2IP:443 ipip
service=https
request="/www/test.html" # this is the Test file for Ldirector to
test the health of RS
receive="300"
scheduler=sh # this schedular is for source hashing
protocol=tcp
checktype=negotiate
virtual=yourVIP:1369 # you may add more than 1 virtual service in
Ldirector
real=RS1IP:1369 ipip
real=RS2IP:1369 ipip
checkport=369
service=ldap
request="dc=yourSuffix"
receive="dc=yourSuffix"
scheduler=rr # this is for round robin
protocol=tcp
checktype=negotiate

Configure the tunul on your Real Servers like this.

ifconfig tunl0 0.0.0.0 up;
ifconfig tunl0 yourVIP netmask 255.255.255.255 broadcast yourVIP;
echo "1" > /proc/sys/net/ipv4/conf/all/hidden;
echo "1" > /proc/sys/net/ipv4/conf/tunl0/hidden;
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore;
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce;
echo "1" > /proc/sys/net/ipv4/conf/tunl0/arp_ignore;
echo "2" > /proc/sys/net/ipv4/conf/tunl0/arp_announce;
echo "0" > /proc/sys/net/ipv4/conf/tunl0/rp_filter;
echo "1" > /proc/sys/net/ipv4/ip_forward;
/etc/init.d/arno-iptables-firewall stop;

Now start you Ldirector ::: ldirectord start

Good Luck

Regards
Anil Sharma

On Fri, Feb 20, 2009 at 12:05 PM, Meghanand Acharekar <
vasco.postfix [at] gmail> wrote:

> Hi,
> I am having following configuration of LVS Cluster.
> All servers in the following configuration have Public IP and can access
> internet.
>
> I want to setup Linux-DR
>
> Master/ Director
> IP address = 192.168.0.1
> Gateway = 192.168.0.254
> VIP = 192.168.2.1
>
> Realserver 1
> IP Address = 192.168.2.2
> Gateway = 192.168.2.254
> Netmask = 255.255.255.224
>
> Realserver 2
> IP Address = 192.168.2.3
> Gateway = 192.168.2.254
> Netmask = 255.255.255.224
>
>
> All the systems are CENTOS 5.2
>
> I enabled packet forwarding on Director using
> also disabled arp request
>
> My sysctrl -p output on realservers is as follows
> [root [at] localhos ~]# sysctl -p
> net.ipv4.ip_forward = 1
> net.ipv4.conf.default.rp_filter = 1
> net.ipv4.conf.default.accept_source_route = 0
> kernel.sysrq = 0
> kernel.core_uses_pid = 1
> net.ipv4.tcp_syncookies = 1
> kernel.msgmnb = 65536
> kernel.msgmax = 65536
> kernel.shmmax = 4294967295
> kernel.shmall = 268435456
> net.ipv4.conf.eth0.arp_ignore = 1
> net.ipv4.conf.eth0.arp_announce = 2
> net.ipv4.conf.all.arp_ignore = 1
> net.ipv4.conf.all.arp_announce = 2
>
> I configured VIP on Director using following command
> ifconfig eth1 192.168.2.1 broadcast 192.168.2.255 netmask 255.255.0.0
>
> I am running smtp service on both the realservers,
>
> Did all stuff with ipvsadm
>
> but still not able to telnet 192.168.2.1 25
>
> it gives me
>
> Trying 216.185.60.11...
> telnet: Unable to connect to remote host: No route to host
>
> but I am able to ping 216.185.60.1
>
> Not able to find out the prob,
>
> Thanx in adv.
>
> Meghanand Acharekar
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

graeme at graemef

Feb 20, 2009, 12:40 AM

Post #4 of 6 (945 views)
Permalink

Re: [lvs-users] LVS-DR problem. [In reply to]

On Fri, 2009-02-20 at 12:05 +0530, Meghanand Acharekar wrote:
> I am having following configuration of LVS Cluster.
<snip>
> Trying 216.185.60.11...
> telnet: Unable to connect to remote host: No route to host
>
> but I am able to ping 216.185.60.1

It would help enormously if you didn't mix and match IP addressing
schemes in your post - you say all devices have public addresses, then
use RFC1918 space, and then switch to public at the end. That's
confusing.

All that said, the way you've configured the VIP on your realservers is
wrong.

> ifconfig eth1 192.168.2.1 broadcast 192.168.2.255 netmask 255.255.0.0

Should be:

ifconfig lo:0 192.168.2.1 broadcast 192.168.2.1 netmask 255.255.255.255

You should, in theory, always use a /32 netmask for the VIP *and* put it
on a loopback device alias (or dummy, or something like that) so the
realserver doesn't add a route to a netblock that it thinks it owns.

Graeme

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

vasco.postfix at gmail

Feb 24, 2009, 1:31 AM

Post #5 of 6 (894 views)
Permalink

Re: [lvs-users] LVS-DR problem. [In reply to]

Revisiting my setup

Hi,

I having following setup for LVS-DR cluster.
Having 3 machine ( 1 director & 2 real servers) with following settings

Director

IP eth0 : 192.168.0.10 ( Netmask : 255.255.255.224 )
VIP eth0:1 : 192.168.0.110 ( Netmask : 255.255.255.224 )

Realserver 1

IP eth0 : 192.168.0.11 ( Netmask : 255.255.255.224 )
lo:0 : 192.168.0.110 ( Netmask : 255.255.255.255 )

Realserver 2
IP eth0 : 192.168.0.12 ( Netmask : 255.255.255.224 )
lo:0 : 192.168.0.110 ( Netmask : 255.255.255.255 )

* Enabled packet forwarding on Director

net.ipv4.ip_forward = 1

on both the real server I have disabled arp by making following entries in
/etc/sysctl.conf
( Kindly let me know if anything is wrong here )

net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

When I do a telnet to port 25 to VIP.
It did not get any response

[root [at] client ~]# telnet 192.168.0.110 25
Trying 192.168.0.110...

But I observed that request is reaching Director but not getting passed to
realservers from there.
Checked following output

[root [at] localhos ~]# ipvsadm -L -n --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes
OutBytes
-> RemoteAddress:Port
TCP 192.168.0.110:25 36 56 0 3360
0
-> 192.168.0.11:25 22 33 0
1980 0
-> 192.168.0.12:25 14 23 0
1380 0

Not able to find out whats wrong :-(

I did LVS setup several time using LVS-NAT, but now facing lots of issues
with LVS-DR

Regards,

Meghanand N. Acharekar
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

thomas at pedoussaut

Feb 24, 2009, 7:00 AM

Post #6 of 6 (881 views)
Permalink

Re: [lvs-users] LVS-DR problem. [In reply to]

Meghanand Acharekar wrote:
> Revisiting my setup
>
> Hi,
>
> I having following setup for LVS-DR cluster.
> Having 3 machine ( 1 director & 2 real servers) with following settings
>
> Director
>
> IP eth0 : 192.168.0.10 ( Netmask : 255.255.255.224 )
> VIP eth0:1 : 192.168.0.110 ( Netmask : 255.255.255.224 )
>
I have put a /32 for the VIP, not a /27.
And for the real IP, a /27 isn't big enough to reach .110 from .10
> Realserver 1
>
> IP eth0 : 192.168.0.11 ( Netmask : 255.255.255.224 )
> lo:0 : 192.168.0.110 ( Netmask : 255.255.255.255 )
>
> Realserver 2
> IP eth0 : 192.168.0.12 ( Netmask : 255.255.255.224 )
> lo:0 : 192.168.0.110 ( Netmask : 255.255.255.255 )
>
> * Enabled packet forwarding on Director
>
> net.ipv4.ip_forward = 1
>
>
> on both the real server I have disabled arp by making following entries in
> /etc/sysctl.conf
> ( Kindly let me know if anything is wrong here )
>
>
> net.ipv4.conf.eth0.arp_ignore = 1
> net.ipv4.conf.eth0.arp_announce = 2
> net.ipv4.conf.all.arp_ignore = 1
> net.ipv4.conf.all.arp_announce = 2
>

Did you run sysctl -p (or rebooted the real servers since) ?
> When I do a telnet to port 25 to VIP.
> It did not get any response
>
> [root [at] client ~]# telnet 192.168.0.110 25
> Trying 192.168.0.110...
>
> But I observed that request is reaching Director but not getting passed to
> realservers from there.
> Checked following output
>
> [root [at] localhos ~]# ipvsadm -L -n --stats
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Conns InPkts OutPkts InBytes
> OutBytes
> -> RemoteAddress:Port
> TCP 192.168.0.110:25 36 56 0 3360
> 0
> -> 192.168.0.11:25 22 33 0
> 1980 0
> -> 192.168.0.12:25 14 23 0
> 1380 0
>
> Not able to find out whats wrong :-(
>
> I did LVS setup several time using LVS-NAT, but now facing lots of issues
> with LVS-DR
I'm pretty sure that you problem is linked with the network mask mismatch.

You can also tcpdump on the various machines to find out if packet
arrive on the real servers or not.
--
Thomas

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads