Mailing List Archive: Linux Virtual Server: Users

LVS/NAT - UM3 Issue

Index | Next | Previous | View Threaded

james at alchemy

May 9, 2007, 10:56 AM

Post #1 of 12 (817 views)
Permalink

LVS/NAT - UM3 Issue

Ok, I may just be overlooking something small but here is my setup:

---------------- ----------------
: ld1 :--HB--: ld2 : <-----
directors and ld1 is a gateway
---------------- ----------------
| |
<---- Virtual IP: 1.2.3.4 and ld1 GWip: 192.168.0.1
-------------------------------------------------------
| | | | |
RS1 RS2 RS3 RS4 RS5...

Failover is working properly for the VIP and the GWip but here is the
problem. When I failover from ld1 to ld2 I can no longer get a response
from the RSs. I can login to each RS and get out just fine to anything
so I know the GWip took fine and they can still route through it but for
some reason the HTTP requests trying to come through once failed-over
are not being sent to the RS. I have verified this through doing
"tcpdump -n -i eth0 port 80" on the RSs while trying to access the test
page I have setup. As soon as I fail back over to ld1 the connections
continue. I was thinking that it was some kind of ARP related issue but
to test that theory I went to each server after failover to ld2 and
cleared the ARP entry for the GWip. Anybody have any ideas? This is
the only thing keeping me from having a working setup. Thanks in advance!

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 11:08 AM

Post #2 of 12 (799 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

Yes. I have both in the haresources file and have verified that it was
carried over via ip addr sh eth1 on ld2 and the same on ld1. After
failover, ld1 no longer has it and ld2 shows it.

Malcolm wrote:
> James,
>
> Are you sure that your gateway ip is failing over as well as your VIP?
> You need two entries in haresources one for the external VIP and one
> for the internal gateway VIP.
>
>
> James Bowling wrote:
>> Ok, I may just be overlooking something small but here is my setup:
>>
>> ---------------- ----------------
>> : ld1 :--HB--: ld2 : <-----
>> directors and ld1 is a gateway
>> ---------------- ----------------
>> | |
>> <---- Virtual IP: 1.2.3.4 and ld1 GWip: 192.168.0.1
>> -------------------------------------------------------
>> | | | | |
>> RS1 RS2 RS3 RS4 RS5...
>>
>> Failover is working properly for the VIP and the GWip but here is the
>> problem. When I failover from ld1 to ld2 I can no longer get a
>> response from the RSs. I can login to each RS and get out just fine
>> to anything so I know the GWip took fine and they can still route
>> through it but for some reason the HTTP requests trying to come
>> through once failed-over are not being sent to the RS. I have
>> verified this through doing "tcpdump -n -i eth0 port 80" on the RSs
>> while trying to access the test page I have setup. As soon as I fail
>> back over to ld1 the connections continue. I was thinking that it
>> was some kind of ARP related issue but to test that theory I went to
>> each server after failover to ld2 and cleared the ARP entry for the
>> GWip. Anybody have any ideas? This is the only thing keeping me
>> from having a working setup. Thanks in advance!
>>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

jmack at wm7d

May 9, 2007, 11:16 AM

Post #3 of 12 (802 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

On Wed, 9 May 2007, James Bowling wrote:

> Yes. I have both in the haresources file and have verified that it was
> carried over via ip addr sh eth1 on ld2 and the same on ld1. After failover,
> ld1 no longer has it and ld2 shows it.

do these files run send-arp (or it's equiv)? If not it will
take 90secs or so to update the arp table on the
realservers. This would be the explanation if the
connections continue after a pause of 90secs.

is the connection table being transferred to the backup
director?

Joe
--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!
_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

lists at loadbalancer

May 9, 2007, 11:17 AM

Post #4 of 12 (800 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

James,

Are you sure that your gateway ip is failing over as well as your VIP?
You need two entries in haresources one for the external VIP and one for
the internal gateway VIP.

James Bowling wrote:
> Ok, I may just be overlooking something small but here is my setup:
>
> ---------------- ----------------
> : ld1 :--HB--: ld2 : <-----
> directors and ld1 is a gateway
> ---------------- ----------------
> | |
> <---- Virtual IP: 1.2.3.4 and ld1 GWip: 192.168.0.1
> -------------------------------------------------------
> | | | | |
> RS1 RS2 RS3 RS4 RS5...
>
> Failover is working properly for the VIP and the GWip but here is the
> problem. When I failover from ld1 to ld2 I can no longer get a
> response from the RSs. I can login to each RS and get out just fine
> to anything so I know the GWip took fine and they can still route
> through it but for some reason the HTTP requests trying to come
> through once failed-over are not being sent to the RS. I have
> verified this through doing "tcpdump -n -i eth0 port 80" on the RSs
> while trying to access the test page I have setup. As soon as I fail
> back over to ld1 the connections continue. I was thinking that it was
> some kind of ARP related issue but to test that theory I went to each
> server after failover to ld2 and cleared the ARP entry for the GWip.
> Anybody have any ideas? This is the only thing keeping me from having
> a working setup. Thanks in advance!
>

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

lists at loadbalancer

May 9, 2007, 11:25 AM

Post #5 of 12 (801 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

James Bowling wrote:
> Yes. I have both in the haresources file and have verified that it
> was carried over via ip addr sh eth1 on ld2 and the same on ld1.
> After failover, ld1 no longer has it and ld2 shows it.
>
OK, when you fail over what does ipvsadm -Ln show?
And when you try to connect what does ipvsadm -Lnc show?

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 11:25 AM

Post #6 of 12 (803 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

ipvsadm -Ln on the backup director shows all the RSs there, output shown
here:

TCP video:http wlc
-> video10:http Masq 1 0 0
-> video9:http Masq 1 0 0
-> video8:http Masq 1 0 0
-> video7:http Masq 1 0 0
-> video6:http Masq 1 0 0
-> video5:http Masq 1 0 0
-> video3:http Masq 1 0 0
-> video2:http Masq 1 0 0
-> video1:http Masq 1 0 0

ipvsadm -Lnc on the backup director shows no connections, output shown here:

IPVS connection entries
pro expire state source virtual destination

Malcolm wrote:
> James Bowling wrote:
>> Yes. I have both in the haresources file and have verified that it
>> was carried over via ip addr sh eth1 on ld2 and the same on ld1.
>> After failover, ld1 no longer has it and ld2 shows it.
>>
> OK, when you fail over what does ipvsadm -Ln show?
> And when you try to connect what does ipvsadm -Lnc show?
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 11:26 AM

Post #7 of 12 (794 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

I have definitely let it sit for longer than 90secs.

Joseph Mack NA3T wrote:
> On Wed, 9 May 2007, James Bowling wrote:
>
>> Yes. I have both in the haresources file and have verified that it
>> was carried over via ip addr sh eth1 on ld2 and the same on ld1.
>> After failover, ld1 no longer has it and ld2 shows it.
>
> do these files run send-arp (or it's equiv)? If not it will take
> 90secs or so to update the arp table on the realservers. This would be
> the explanation if the connections continue after a pause of 90secs.
>
> is the connection table being transferred to the backup director?
>
> Joe

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 11:55 AM

Post #8 of 12 (796 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

Also, the connection IS NOT being transferred over.

Joseph Mack NA3T wrote:
> On Wed, 9 May 2007, James Bowling wrote:
>
>> Yes. I have both in the haresources file and have verified that it
>> was carried over via ip addr sh eth1 on ld2 and the same on ld1.
>> After failover, ld1 no longer has it and ld2 shows it.
>
> do these files run send-arp (or it's equiv)? If not it will take
> 90secs or so to update the arp table on the realservers. This would be
> the explanation if the connections continue after a pause of 90secs.
>
> is the connection table being transferred to the backup director?
>
> Joe

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 11:56 AM

Post #9 of 12 (793 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

And yes, I have SendArp::192.168.0.1/eth1 in my haresources and have
verified that it is sending gratuitous arp through tcpdump -n -i eth1
arp on both lds.

Joseph Mack NA3T wrote:
> On Wed, 9 May 2007, James Bowling wrote:
>
>> Yes. I have both in the haresources file and have verified that it
>> was carried over via ip addr sh eth1 on ld2 and the same on ld1.
>> After failover, ld1 no longer has it and ld2 shows it.
>
> do these files run send-arp (or it's equiv)? If not it will take
> 90secs or so to update the arp table on the realservers. This would be
> the explanation if the connections continue after a pause of 90secs.
>
> is the connection table being transferred to the backup director?
>
> Joe

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 12:29 PM

Post #10 of 12 (797 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

So does anyone have any ideas?

James Bowling wrote:
> And yes, I have SendArp::192.168.0.1/eth1 in my haresources and have
> verified that it is sending gratuitous arp through tcpdump -n -i eth1
> arp on both lds.
>
> Joseph Mack NA3T wrote:
>> On Wed, 9 May 2007, James Bowling wrote:
>>
>>> Yes. I have both in the haresources file and have verified that it
>>> was carried over via ip addr sh eth1 on ld2 and the same on ld1.
>>> After failover, ld1 no longer has it and ld2 shows it.
>>
>> do these files run send-arp (or it's equiv)? If not it will take
>> 90secs or so to update the arp table on the realservers. This would
>> be the explanation if the connections continue after a pause of 90secs.
>>
>> is the connection table being transferred to the backup director?
>>
>> Joe
>

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

lists at loadbalancer

May 9, 2007, 12:55 PM

Post #11 of 12 (799 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

James,

If ipvsadm -Lnc doesn't show any connections then its either local
firewall, or routing to the VIP thats the problem.
Does arp -a on the client show the correct MAC address for the slave?

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

james at alchemy

May 9, 2007, 1:20 PM

Post #12 of 12 (799 views)
Permalink

Re: LVS/NAT - UM3 Issue [In reply to]

And what do you know...I overlooked something as small as not allowing
HTTP connects through iptables on ld2!!! Thanks guys!

Malcolm wrote:
> James,
>
> If ipvsadm -Lnc doesn't show any connections then its either local
> firewall, or routing to the VIP thats the problem.
> Does arp -a on the client show the correct MAC address for the slave?
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users

--
Regards,
James Bowling <james [at] alchemy>
Director of Network Operations
Alchemy Communications, Inc.
818-206-9260 x4043

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads