Mailing List Archive: Linux Virtual Server: Users

[lvs-users] Help with LVS NAT and RHEL5.8

Index | Next | Previous | View Threaded

wliu at nds

Jul 26, 2012, 9:40 AM

Post #1 of 4 (457 views)
Permalink

[lvs-users] Help with LVS NAT and RHEL5.8

Hi,

I am a problem with LVS NAT configuration where the packets do not look like they are being masqueraded by LVS. Here's my setup:

LVS server has 3 interfaces: primary, nat_router, virtual IP
172.5.111.74 -primary
172.25.117.4 - nat router
172.25.117.5 - virtual IP, port 80
|---- 172.28.12.56 (Real server)

A client (172.25.111.8) connects to 172.25.117.5 on port 80 never gets a response back. What I see on Real sever (172.28.12.56) on tcpdump is :
16:35:08.103968 IP 172.25.111.8.34271 > 172.28.12.56.http: S 1718115488:1718115488(0) win 5840 <mss 1460,sackOK,timestamp 500867550 0,nop,wscale 7>

This shows source IP of the client and NOT from LVS. I presume in NAT mode, the source IP should be of the "nat router?" From my understanding LVS should have done the header masquerading? I shouldn't have to use IPtables? Please let me know what I have to do for this function to work?

# ipvsadm -l -n --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 172.25.117.5:80 4 15 0 900 0
-> 172.28.12.56:80 4 15 0 900 0

I would like LVS server to be the gateway for both traffic to the real server and on the way back to the client.

This LVS server is a RHEL5.8 with 2.6.18 kernel. Here is my lvs.cf config:
serial_no = 16
primary = 172.25.111.74
service = lvs
backup = 0.0.0.0
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 18
network = nat
nat_router = 172.25.117.4 eth1:1
nat_nmask = 255.255.255.0
debug_level = NONE
virtual 172.28.12.56 {
active = 1
address = 172.25.117.5 eth1:2
vip_nmask = 255.255.255.255
port = 80
expect = "OK"
use_regex = 0
send_program = "/etc/sysconfig/ha/check_tcp80.sh %h"
load_monitor = none
scheduler = rr
protocol = tcp
timeout = 60
reentry = 15
quiesce_server = 0
server diadm1cm {
address = 172.28.12.56
active = 1
weight = 1
}
}

________________________________
This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster [at] nds and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
To protect the environment please do not print this e-mail unless necessary.

An NDS Group Limited company. www.nds.com
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

david at davidcoulson

Jul 26, 2012, 10:23 AM

Post #2 of 4 (439 views)
Permalink

Re: [lvs-users] Help with LVS NAT and RHEL5.8 [In reply to]

On 7/26/12 12:40 PM, Liu, William wrote:
> Hi,
>
> I am a problem with LVS NAT configuration where the packets do not look like they are being masqueraded by LVS. Here's my setup:
>
> LVS server has 3 interfaces: primary, nat_router, virtual IP
> 172.5.111.74 -primary
> 172.25.117.4 - nat router
> 172.25.117.5 - virtual IP, port 80
> |---- 172.28.12.56 (Real server)
>
> A client (172.25.111.8) connects to 172.25.117.5 on port 80 never gets a response back. What I see on Real sever (172.28.12.56) on tcpdump is :
> 16:35:08.103968 IP 172.25.111.8.34271 > 172.28.12.56.http: S 1718115488:1718115488(0) win 5840 <mss 1460,sackOK,timestamp 500867550 0,nop,wscale 7>
>
> This shows source IP of the client and NOT from LVS. I presume in NAT mode, the source IP should be of the "nat router?" From my understanding LVS should have done the header masquerading? I shouldn't have to use IPtables? Please let me know what I have to do for this function to work?

There is a SNAT patch for LVS out on the Internet somewhere, but it is
not supported by RedHat. With RHEL, none of the three (DR,NAT, TUN)
mechanisms modify the source IP of the packets.

If you use LVS-NAT, you need to make sure the real server routes the
packet back through the LVS director so the 'un-NAT' can happen
correctly before the request goes back to the client.

David

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

malcolm at loadbalancer

Jul 26, 2012, 10:43 AM

Post #3 of 4 (444 views)
Permalink

Re: [lvs-users] Help with LVS NAT and RHEL5.8 [In reply to]

Liu,

Yes, David is right it is working as expected Full-NAT i.e. source IP
transparent.
If you want the real servers/backend servers to have Internet access
as well then you will need an iptables masquerade rule or something
similar for the outgoing traffic.

LVS Half-Nat or SNAT is in mainline kernel, the old way of patching is
described here:
http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/

But to be honest if you want SNAT/proxy you'd be better off using
HAProxy which is well tested for that purpose...

On 26 July 2012 18:23, David Coulson <david [at] davidcoulson> wrote:
>
>
> On 7/26/12 12:40 PM, Liu, William wrote:
> > Hi,
> >
> > I am a problem with LVS NAT configuration where the packets do not look like they are being masqueraded by LVS. Here's my setup:
> >
> > LVS server has 3 interfaces: primary, nat_router, virtual IP
> > 172.5.111.74 -primary
> > 172.25.117.4 - nat router
> > 172.25.117.5 - virtual IP, port 80
> > |---- 172.28.12.56 (Real server)
> >
> > A client (172.25.111.8) connects to 172.25.117.5 on port 80 never gets a response back. What I see on Real sever (172.28.12.56) on tcpdump is :
> > 16:35:08.103968 IP 172.25.111.8.34271 > 172.28.12.56.http: S 1718115488:1718115488(0) win 5840 <mss 1460,sackOK,timestamp 500867550 0,nop,wscale 7>
> >
> > This shows source IP of the client and NOT from LVS. I presume in NAT mode, the source IP should be of the "nat router?" From my understanding LVS should have done the header masquerading? I shouldn't have to use IPtables? Please let me know what I have to do for this function to work?
>
> There is a SNAT patch for LVS out on the Internet somewhere, but it is
> not supported by RedHat. With RHEL, none of the three (DR,NAT, TUN)
> mechanisms modify the source IP of the packets.
>
> If you use LVS-NAT, you need to make sure the real server routes the
> packet back through the LVS director so the 'un-NAT' can happen
> correctly before the request goes back to the client.
>
> David
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

--
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

dennisml at conversis

Jul 26, 2012, 10:49 AM

Post #4 of 4 (444 views)
Permalink

Re: [lvs-users] Help with LVS NAT and RHEL5.8 [In reply to]

On 07/26/2012 06:40 PM, Liu, William wrote:
> Hi,
>
> I am a problem with LVS NAT configuration where the packets do not look like they are being masqueraded by LVS. Here's my setup:
>
> LVS server has 3 interfaces: primary, nat_router, virtual IP
> 172.5.111.74 -primary
> 172.25.117.4 - nat router
> 172.25.117.5 - virtual IP, port 80
> |---- 172.28.12.56 (Real server)
>
> A client (172.25.111.8) connects to 172.25.117.5 on port 80 never gets a response back. What I see on Real sever (172.28.12.56) on tcpdump is :
> 16:35:08.103968 IP 172.25.111.8.34271 > 172.28.12.56.http: S 1718115488:1718115488(0) win 5840 <mss 1460,sackOK,timestamp 500867550 0,nop,wscale 7>
>
> This shows source IP of the client and NOT from LVS. I presume in NAT mode, the source IP should be of the "nat router?" From my understanding LVS should have done the header masquerading? I shouldn't have to use IPtables? Please let me know what I have to do for this function to work?
>
> # ipvsadm -l -n --stats
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
> -> RemoteAddress:Port
> TCP 172.25.117.5:80 4 15 0 900 0
> -> 172.28.12.56:80 4 15 0 900 0
>
> I would like LVS server to be the gateway for both traffic to the real server and on the way back to the client.
>
> This LVS server is a RHEL5.8 with 2.6.18 kernel. Here is my lvs.cf config:
> serial_no = 16
> primary = 172.25.111.74
> service = lvs
> backup = 0.0.0.0
> heartbeat = 1
> heartbeat_port = 539
> keepalive = 6
> deadtime = 18
> network = nat
> nat_router = 172.25.117.4 eth1:1
> nat_nmask = 255.255.255.0
> debug_level = NONE
> virtual 172.28.12.56 {
> active = 1
> address = 172.25.117.5 eth1:2
> vip_nmask = 255.255.255.255
> port = 80
> expect = "OK"
> use_regex = 0
> send_program = "/etc/sysconfig/ha/check_tcp80.sh %h"
> load_monitor = none
> scheduler = rr
> protocol = tcp
> timeout = 60
> reentry = 15
> quiesce_server = 0
> server diadm1cm {
> address = 172.28.12.56
> active = 1
> weight = 1
> }
> }

As far as I know the NAT stands for DNAT not SNAT so the source address
doesn't change but the destination address is changed to the ip of the
realserver. The realserver has to have the director as it's default gateway
so that when the return packet is sent the DNAT gets reverted on the director.

Regards,
Dennis

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads