Mailing List Archive: Linux Virtual Server: Users

[lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP?

Index | Next | Previous | View Threaded

partysoft at gmail

Oct 28, 2009, 9:13 AM

Post #1 of 8 (1146 views)
Permalink

[lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP?

Hy All!
i'm trying to setup an LVS system for http with 3 servers 1 being the
director..(i have a previous post for this that went sucessfull)
the subjects are the servers:
10.1.228.234 (public ip of the director) -has configured the 10.1.228.236
as the VIP (virtual ip) - so the http service will be accesible through
this ip and directed to the real servers (RS)

10.1.228.235 RS1 (real server 1 - we used the method VS/DR as they are
cable linked and same C class, no router between them ) - and it works
great...without problems
10.1.239.163 RS2 (real server 2 - VS/TUN, by encapsulating the packages in
a tunnel since it passes through a router..and this is the way to do it )

we don't want to use the VS/NAT method since is's serving us because of the
performance

we didn't succeed to make it work with VS/NAT , and sincerly we kind of have
a hunch why... and it's because we can't solve the arp problem ..
of course we folowed some tutorials on the net..that helped us make it work
on the VS/DR method.

here's what we've found wierd..very wierd.. we've tried to arping the ip's
from each server:

[root [at] linu 10.1.228.234]# arping 10.1.228.235 < from the director
ARPING 10.1.228.235 from 10.1.228.234 eth0
Unicast reply from 10.1.228.235 [00:24:5D:24:61:AB] 0.550ms

root [at] linu 10.1.228.235]# ifconfig eth0 <-as you can see the mac was the
same..so that's why it's probably working between those
eth0 Link encap:Ethernet HWaddr 00:24:5D:24:61:AB
inet addr:10.1.228.235 Bcast:10.1.228.239 Mask:255.255.255.248

from the external ip
-bash-10.1.239.163 # arping 10.1.228.234
ARPING 10.1.228.234 from 10.1.239.163 eth0 < doesn't match the real mac of
the 10.1.228.234
Unicast reply from 10.1.228.234 [00:0A:F3:82:3A:00] 1.179ms

-bash-10.1.239.163 # arping 10.1.228.235
ARPING 10.1.228.234 from 10.1.239.163 eth0
Unicast reply from 10.1.228.234 [00:0A:F3:82:3A:00] 1.179ms < how can the
reply be from the same MAC as the previous ping??? and it's not even the
real one..

[root [at] linu 10.1.228.234]# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:24:2D:67:30:29
inet addr:10.1.228.234 Bcast:10.1.228.239 Mask:255.255.255.248

so it's an arp problem

we've tried to solve the arp problem with arptables, and with the

here's a tcpdump from the director...

22:30:10.598330 arp who-has 10.1.228.235 tell 10.1.228.234
22:30:10.598355 arp reply 10.1.228.235 is-at 00:24:5D:24:61:ab (oui Unknown)

22:33:34.569821 arp who-has 10.1.239.163 tell 10.1.228.236
22:33:34.570389 arp reply 10.1.239.163 is-at 00:0a:f3:82:3A:00 (oui Unknown)

---------------
here's what we configured

LVS:
ifconfig eth0:0 down
ipvsadm -C
ifconfig eth0:0 10.1.228.236 netmask 255.255.255.255 broadcast 10.1.228.236
up
echo 0 > /proc/sys/net/ipv4/ip_forward
ipvsadm -A -t 10.1.228.236:80 -s wlc
ipvsadm -a -t 10.1.228.236:80 -r 10.1.239.163 -i -w 1

Real Server 1

arptables -F
ifconfig tunl0 down
echo 0 > /proc/sys/net/ipv4/ip_forward
modprobe ipip
ifconfig tunl0 0.0.0.0 up
echo 1 > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
arptables -A IN -d 10.1.228.236 -j DROP
arptables -A OUT -s 10.1.228.236 -o eth0 -j mangle --mangle-ip-s
10.1.239.163
ifconfig tunl0 10.1.228.236 netmask 255.255.255.255 broadcast 10.1.228.236
up
route add -host 10.1.228.236 dev tunl0
--
View this message in context: http://www.nabble.com/LVS-TUN-wierd-conf..bad-arp-reply-form-the-same-MAC-on-any-IP--tp26097535p26097535.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

partysoft at gmail

Oct 28, 2009, 3:14 PM

Post #2 of 8 (1079 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

Just a quick update

when i try to load in the server browser the..
http://10.1.228.234
it doesn't give a timeout...right away it stays there for like 30 seconds
,..connecting...
that's when i've made the tcp dump in the previous post

if i do a list
[root [at] linu ~]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.1.228.236:http wlc
-> 10.1.239.163:http Tunnel 1 0 2

so probably the answer of what i'm doing wrong is..pretty simlpe, but i'm
not a 100% networking guru, so i\m clearly missing it

so it sais clearly InActConn 2,

partysoft wrote:
>
> Hy All!
> i'm trying to setup an LVS system for http with 3 servers 1 being the
> director..(i have a previous post for this that went sucessfull)
> the subjects are the servers:
> 10.1.228.234 (public ip of the director) -has configured the 10.1.228.236
> as the VIP (virtual ip) - so the http service will be accesible through
> this ip and directed to the real servers (RS)
>
> 10.1.228.235 RS1 (real server 1 - we used the method VS/DR as they are
> cable linked and same C class, no router between them ) - and it works
> great...without problems
> 10.1.239.163 RS2 (real server 2 - VS/TUN, by encapsulating the packages
> in a tunnel since it passes through a router..and this is the way to do it
> )
> ----------------------------------------
>

--
View this message in context: http://www.nabble.com/LVS-TUN-wierd-conf..bad-arp-reply-form-the-same-MAC-on-any-IP--tp26097535p26103118.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

malte at snapscouts

Oct 28, 2009, 7:08 PM

Post #3 of 8 (1083 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

Please take a look at the bottom of the howto
where there's explained howto set tcp-mss size ;)

> Just a quick update
>
> when i try to load in the server browser the..
> http://10.1.228.234
> it doesn't give a timeout...right away it stays there for like 30 seconds
> ,..connecting...
> that's when i've made the tcp dump in the previous post
>
> if i do a list
> [root [at] linu ~]# ipvsadm -L
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 10.1.228.236:http wlc
> -> 10.1.239.163:http Tunnel 1 0 2
>
>
> so probably the answer of what i'm doing wrong is..pretty simlpe, but i'm
> not a 100% networking guru, so i\m clearly missing it
>
> so it sais clearly InActConn 2,
>
>
> partysoft wrote:
>
>> Hy All!
>> i'm trying to setup an LVS system for http with 3 servers 1 being the
>> director..(i have a previous post for this that went sucessfull)
>> the subjects are the servers:
>> 10.1.228.234 (public ip of the director) -has configured the 10.1.228.236
>> as the VIP (virtual ip) - so the http service will be accesible through
>> this ip and directed to the real servers (RS)
>>
>> 10.1.228.235 RS1 (real server 1 - we used the method VS/DR as they are
>> cable linked and same C class, no router between them ) - and it works
>> great...without problems
>> 10.1.239.163 RS2 (real server 2 - VS/TUN, by encapsulating the packages
>> in a tunnel since it passes through a router..and this is the way to do it
>> )
>> ----------------------------------------
>>
>>
>

--
Malte Geierhos
Serverbetreuung / Administration

SnapScouts.de
Snapscouts Limited Zweigniederlassung G�rlitz
Krischelstra�e 14
02826 G�rlitz

Fax: +49 (0) 35 81 / 66 72 53
Mobil: +49 (0) 1 51 / 27 00 22 70
E-Mail: malte [at] snapscouts

Web: http://www.snapscouts.de

Gesch�ftsf�hrer: Thomas von Skrbensky
Registergericht: Dresden HRB 25880
USt-IdNr.: DE254464823

Zweigniederlassung der Snapscouts Limited
69 Great Hampton Street
Birmingham, B18 6EW
Handelsreegister f�r England& Wales
Unternehmen Nr.: 6198239

Diese Information ist ausschlie�lich f�r den Adressaten bestimmt und
kann vertrauliche oder gesetzlich gesch�tzte Informationen enthalten.
Wenn Sie nicht der bestimmungsgem��e Adressat sind, unterrichten Sie
bitte den Absender und vernichten Sie diese E-Mail. Anderen als dem
bestimmungsgem��en Adressaten ist es untersagt, diese E-Mail zu lesen,
zu speichern, weiterzuleiten oder ihren Inhalt, auf welche Weise auch
immer, zu verwenden. Wir verwenden aktuelle Virenschutzprogramme.
F�r Sch�den, die dem Empf�nger, gleichwohl durch von uns zugesandte
mit Viren befallene E-Mails entstehen, schlie�en wir jede Haftung aus.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

partysoft at gmail

Oct 28, 2009, 8:01 PM

Post #4 of 8 (1085 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

so it isn't because of the arp and the router ? i thought that it had
something to do with it, but of course the TUN is supposed to work
regardless of the ARP, because the packages are encapsulates , and the main
arp is ignored from the kernel params..right?

On Wed, Oct 28, 2009 at 8:08 PM, Malte Geierhos <malte [at] snapscouts> wrote:

> Please take a look at the bottom of the howto
> where there's explained howto set tcp-mss size ;)
>
> > Just a quick update
> >
> > when i try to load in the server browser the..
> > http://10.1.228.234
> > it doesn't give a timeout...right away it stays there for like 30 seconds
> > ,..connecting...
> > that's when i've made the tcp dump in the previous post
> >
> > if i do a list
> > [root [at] linu ~]# ipvsadm -L
> > IP Virtual Server version 1.2.1 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > TCP 10.1.228.236:http wlc
> > -> 10.1.239.163:http Tunnel 1 0 2
> >
> >
> > so probably the answer of what i'm doing wrong is..pretty simlpe, but i'm
> > not a 100% networking guru, so i\m clearly missing it
> >
> > so it sais clearly InActConn 2,
> >
> >
> > partysoft wrote:
> >
> >> Hy All!
> >> i'm trying to setup an LVS system for http with 3 servers 1 being the
> >> director..(i have a previous post for this that went sucessfull)
> >> the subjects are the servers:
> >> 10.1.228.234 (public ip of the director) -has configured the
> 10.1.228.236
> >> as the VIP (virtual ip) - so the http service will be accesible through
> >> this ip and directed to the real servers (RS)
> >>
> >> 10.1.228.235 RS1 (real server 1 - we used the method VS/DR as they are
> >> cable linked and same C class, no router between them ) - and it works
> >> great...without problems
> >> 10.1.239.163 RS2 (real server 2 - VS/TUN, by encapsulating the packages
> >> in a tunnel since it passes through a router..and this is the way to do
> it
> >> )
> >> ----------------------------------------
> >>
> >>
> >
>
>
> --
> Malte Geierhos
> Serverbetreuung / Administration
>
>
> SnapScouts.de
> Snapscouts Limited Zweigniederlassung G�rlitz
> Krischelstra�e 14
> 02826 G�rlitz
>
> Fax: +49 (0) 35 81 / 66 72 53
> Mobil: +49 (0) 1 51 / 27 00 22 70
> E-Mail: malte [at] snapscouts
>
> Web: http://www.snapscouts.de
>
>
> Gesch�ftsf�hrer: Thomas von Skrbensky
> Registergericht: Dresden HRB 25880
> USt-IdNr.: DE254464823
>
> Zweigniederlassung der Snapscouts Limited
> 69 Great Hampton Street
> Birmingham, B18 6EW
> Handelsreegister f�r England& Wales
> Unternehmen Nr.: 6198239
>
> Diese Information ist ausschlie�lich f�r den Adressaten bestimmt und
> kann vertrauliche oder gesetzlich gesch�tzte Informationen enthalten.
> Wenn Sie nicht der bestimmungsgem��e Adressat sind, unterrichten Sie
> bitte den Absender und vernichten Sie diese E-Mail. Anderen als dem
> bestimmungsgem��en Adressaten ist es untersagt, diese E-Mail zu lesen,
> zu speichern, weiterzuleiten oder ihren Inhalt, auf welche Weise auch
> immer, zu verwenden. Wir verwenden aktuelle Virenschutzprogramme.
> F�r Sch�den, die dem Empf�nger, gleichwohl durch von uns zugesandte
> mit Viren befallene E-Mails entstehen, schlie�en wir jede Haftung aus.
>
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

graeme at graemef

Oct 29, 2009, 2:13 AM

Post #5 of 8 (1079 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

On Wed, 2009-10-28 at 09:13 -0700, partysoft wrote:
> [root [at] linu 10.1.228.234]# arping 10.1.228.235 < from the director
> ARPING 10.1.228.235 from 10.1.228.234 eth0
> Unicast reply from 10.1.228.235 [00:24:5D:24:61:AB] 0.550ms

These are on the same network; the reply will come direct.

> from the external ip
> -bash-10.1.239.163 # arping 10.1.228.234
> ARPING 10.1.228.234 from 10.1.239.163 eth0 < doesn't match the real mac of
> the 10.1.228.234
> Unicast reply from 10.1.228.234 [00:0A:F3:82:3A:00] 1.179ms
>
> -bash-10.1.239.163 # arping 10.1.228.235
> ARPING 10.1.228.234 from 10.1.239.163 eth0
> Unicast reply from 10.1.228.234 [00:0A:F3:82:3A:00] 1.179ms < how can the
> reply be from the same MAC as the previous ping??? and it's not even the
> real one..

These are not on the same network; the reply will come via a router. The
source MAC in the reply packet will always be that of the router in this
context.

MAC addresses do not propagate beyond a device which separates switching
domains - this is a fundamental networking property.

I'm having some trouble understanding what you're trying to achieve, and
it appears that you're flailing around trying to get an LVS working
without really waiting for people to help. It's difficult for anyone to
help if you keep moving your setup around.

Very simple question: have you tried to setup a basic LVS with one
director and one realserver yet? If you do that, using ipvsadm, then you
will get a better feel for what's required and a better understanding of
the underlying principles.

Graeme

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

graeme at graemef

Oct 29, 2009, 2:15 AM

Post #6 of 8 (1079 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

On Wed, 2009-10-28 at 21:01 -0600, Partica Cristian wrote:
> so it isn't because of the arp and the router ? i thought that it had
> something to do with it, but of course the TUN is supposed to work
> regardless of the ARP, because the packages are encapsulates , and the main
> arp is ignored from the kernel params..right?

Please take a step back and try NAT again. I'm quite sure, once you
understand how it works, that it will fit your requirements exactly. TUN
is far more complex and subject to a lot of other inputs which can stop
it from working.

Graeme

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

partysoft at gmail

Oct 29, 2009, 4:39 PM

Post #7 of 8 (1067 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

Isn't the NAT method lame, like it doesn't support as many connections ?

anyway i've moved the external ip to the LAN , in the same C class , and now
it works of course..
i will try this with other server in the future because i was very pressured
by time..

Thank you for the info i will investigate in this order, NAT then the TUN
(tunnel)

One question OFF the topic....
I want to make a NFS between the 2 real servers, so when one HTTP stops
serving the other can..
as the files won't be the same on both servers, but both can serve that way
through NFS

the question: When the HTTP service can't serve because ..too many
connections... will the NFS stop working as well? i mean they are 2 separate
daemons.. and HTTP will serve to the outside connections and NFS on LAN...
is there any other way to make hard drive syncro or sharing ?

Thank you

Bugzilla from graeme [at] graemef wrote:
>
> On Wed, 2009-10-28 at 21:01 -0600, Partica Cristian wrote:
>> so it isn't because of the arp and the router ? i thought that it had
>> something to do with it, but of course the TUN is supposed to work
>> regardless of the ARP, because the packages are encapsulates , and the
>> main
>> arp is ignored from the kernel params..right?
>
> Please take a step back and try NAT again. I'm quite sure, once you
> understand how it works, that it will fit your requirements exactly. TUN
> is far more complex and subject to a lot of other inputs which can stop
> it from working.
>
> Graeme
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
>

--
View this message in context: http://www.nabble.com/LVS-TUN-wierd-conf..bad-arp-reply-form-the-same-MAC-on-any-IP--tp26097535p26122711.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Oct 29, 2009, 6:29 PM

Post #8 of 8 (1072 views)
Permalink

Re: [lvs-users] LVS TUN wierd conf..bad arp reply form the same MAC on any IP? [In reply to]

On Thu, 29 Oct 2009, partysoft wrote:

>
> Isn't the NAT method lame,

not according to the HOWTO

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads