Mailing List Archive: Linux Virtual Server: Users

[lvs-users] ipvs or apache/mod_proxy/mod_balancer

1 2

View All

Index | Next | Previous | View Threaded

public at ecopatz

Jun 24, 2009, 6:37 AM

Post #1 of 34 (2931 views)
Permalink

[lvs-users] ipvs or apache/mod_proxy/mod_balancer

Hello,

i want to put a loadbalancer in front of several squids, which cache output
of content generators.

The first option is to use the apache/mod_proxy/mod_balancer combination.
Thats easy to configure, you even have a little backend to disable
BalancerMembers manually and also i dont need any special network
configuration. We are quite happy with it. Its simple

But there is this overhead of interpreting the content and to manage the
connections.

So the second option would involve IPVS, which just rewrites the packes and
forwards them to a chosen BalancerMember.

There are two ways:

A) IPVS behaves like mod_proxy, the chosen Member will see a request coming
from the LoadBalancer's LAN-IP.

Bad: I lose the client IP on the Member/real server (in mod_proxy i have at
least "X-Forwarded-For")

B) IPVS forwards the packet to the chosen Member. The source address is
unchanged.

Bad: I need to change the default route, so that answers always go back
through the LoadBalancer. This complicates the otherwise so simple network
setup. People need to know about IPVS. Its not so obvious and self
explanatory.

So the question: Is IPVS worth the trouble?

Or is this an exaggeration, when i say apache/mod_proxy uses way more
resources?

What would you say?
--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24184926.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jun 24, 2009, 7:41 AM

Post #2 of 34 (2864 views)
Permalink

[lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

leon at linux

Jun 29, 2009, 12:16 PM

Post #3 of 34 (2870 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

On Wed, 2009-06-24 at 07:41 -0700, Olaf Krische wrote:
> A) IPVS behaves like mod_proxy, the chosen Member will see a request coming
> from the LoadBalancer's LAN-IP.
>
> Bad: I lose the client IP on the Member/real server (in mod_proxy i have at
> least "X-Forwarded-For")

No you don't. LVS-DR (i assume this is what you mean, judging by your
option B's 'Bad' description) just forwards the packet to a realserver.
This sees the request as if the packet would come straight from the
client (eg. not through LVS; it's completely transparent). It will send
the reply back straight to the client.

regards,

--
Léon

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

xufengnju at sina

Jun 29, 2009, 6:19 PM

Post #4 of 34 (2872 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Hi Olaf Krische,

What you described in your B option is the LVS-NAT mode ��you may call it D-NAT��You may try to use LVS-DR (Direct Routing) .On IBM developer's website,you can search articals from the author of LVS (Wensong Zhang) ,and I am sure you will get a clear picture of LVS.

------------------
XUFENG
2009-06-30

-------------------------------------------------------------
��ˣ�Olaf Krische
��ڣ�2009-06-24 21:37:53
�ռ��ˣ�lvs-users
��ͣ�
��⣺[lvs-users] ipvs or apache/mod_proxy/mod_balancer

Hello,

i want to put a loadbalancer in front of several squids, which cache output
of content generators.

The first option is to use the apache/mod_proxy/mod_balancer combination.
Thats easy to configure, you even have a little backend to disable
BalancerMembers manually and also i dont need any special network
configuration. We are quite happy with it. Its simple

But there is this overhead of interpreting the content and to manage the
connections.

So the second option would involve IPVS, which just rewrites the packes and
forwards them to a chosen BalancerMember.

There are two ways:

A) IPVS behaves like mod_proxy, the chosen Member will see a request coming
from the LoadBalancer's LAN-IP.

Bad: I lose the client IP on the Member/real server (in mod_proxy i have at
least "X-Forwarded-For")

B) IPVS forwards the packet to the chosen Member. The source address is
unchanged.

Bad: I need to change the default route, so that answers always go back
through the LoadBalancer. This complicates the otherwise so simple network
setup. People need to know about IPVS. Its not so obvious and self
explanatory.

So the question: Is IPVS worth the trouble?

Or is this an exaggeration, when i say apache/mod_proxy uses way more
resources?

What would you say?
--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24184926.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

malcolm at loadbalancer

Jun 30, 2009, 2:34 AM

Post #5 of 34 (2866 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Olaf,

Usually we use Direct Routing mode with proxies(squids) because it is:

a) source ip transparent
b) return traffic doesn't need to go via the load balancer
c) its fast

2009/6/24 Olaf Krische <public [at] ecopatz>
>
> Hello,
>
> i want to put a loadbalancer in front of several squids, which cache output
> of content generators.
>
> The first option is to use the apache/mod_proxy/mod_balancer combination.
> Thats easy to configure, you even have a little backend to disable
> BalancerMembers manually and also i dont need any special network
> configuration. We are quite happy with it. Its simple
>
> But there is this overhead of interpreting the content and to manage the
> connections.
>
> So the second option would involve IPVS, which just rewrites the packes and
> forwards them to a chosen BalancerMember.
>
> There are two ways:
>
> A) IPVS behaves like mod_proxy, the chosen Member will see a request coming
> from the LoadBalancer's LAN-IP.
>
> Bad: I lose the client IP on the Member/real server (in mod_proxy i have at
> least "X-Forwarded-For")
>
> B) IPVS forwards the packet to the chosen Member. The source address is
> unchanged.
>
> Bad: I need to change the default route, so that answers always go back
> through the LoadBalancer. This complicates the otherwise so simple network
> setup. People need to know about IPVS. Its not so obvious and self
> explanatory.
>
> So the question: �Is IPVS worth the trouble?
>
> Or is this an exaggeration, when i say apache/mod_proxy uses way more
> resources?
>
> What would you say?
> --
> View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24184926.html
> Sent from the LVS mailing list archive at Nabble.com.
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

--
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jun 30, 2009, 5:39 AM

Post #6 of 34 (2861 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Thanks for answering to everyone,

Malcolm Turnbull wrote:
>
> Usually we use Direct Routing mode with proxies(squids) because it is:
>
> a) source ip transparent
> b) return traffic doesn't need to go via the load balancer
> c) its fast
>

Okay, i will try this out. Tho, i am not a big fan of giving each squid an
IP. I do not own a big block.

I used to have a hardware loadbalancer, which owns the VIP. It would rewrite
the incoming packet by replacing the VIP with the IP of the chosen real
server in the LAN. And the real server then had to answer back to the
loadbalancer, so he can replace the IP of the real server with the VIP
again. It felt more...self describing and did not need a special
configuration on the real servers except the routing.

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24271155.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

misch at multinet

Jun 30, 2009, 5:51 AM

Post #7 of 34 (2865 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Am Dienstag, 30. Juni 2009 14:39:49 schrieb Olaf Krische:
> Thanks for answering to everyone,
>
> Malcolm Turnbull wrote:
> > Usually we use Direct Routing mode with proxies(squids) because it is:
> >
> > a) source ip transparent
> > b) return traffic doesn't need to go via the load balancer
> > c) its fast
>
> Okay, i will try this out. Tho, i am not a big fan of giving each squid an
> IP. I do not own a big block.
>
> I used to have a hardware loadbalancer, which owns the VIP. It would
> rewrite the incoming packet by replacing the VIP with the IP of the chosen
> real server in the LAN. And the real server then had to answer back to the
> loadbalancer, so he can replace the IP of the real server with the VIP
> again. It felt more...self describing and did not need a special
> configuration on the real servers except the routing.

Hi,

you can do the same config with LVS. Just use NAT mode. The disadvantage: it
does not scale that good because every traffix has to be routed back through the
gateway. As a rule of the thumb you can tell: About 10 to 20 real servers in
NAT mode.

--
Dr. Michael Schwartzkopff
MultiNET Services GmbH
Addresse: Bretonischer Ring 7; 85630 Grasbrunn; Germany
Tel: +49 - 89 - 45 69 11 0
Fax: +49 - 89 - 45 69 11 21
mob: +49 - 174 - 343 28 75

mail: misch [at] multinet
web: www.multinet.de

Sitz der Gesellschaft: 85630 Grasbrunn
Registergericht: Amtsgericht M�nchen HRB 114375
Gesch�ftsf�hrer: G�nter Jurgeneit, Hubert Martens

---

PGP Fingerprint: F919 3919 FF12 ED5A 2801 DEA6 AA77 57A4 EDD8 979B
Skype: misch42

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Jun 30, 2009, 7:15 AM

Post #8 of 34 (2859 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

On Tue, 30 Jun 2009, Olaf Krische wrote:

> Okay, i will try this out. Tho, i am not a big fan of giving each squid an
> IP. I do not own a big block.

LVS-DR realservers have only private IPs

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jay.faulkner at mailtrust

Jun 30, 2009, 7:28 AM

Post #9 of 34 (2857 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

[snip]

>
> you can do the same config with LVS. Just use NAT mode. The
> disadvantage: it
> does not scale that good because every traffix has to be routed back
> through the
> gateway. As a rule of the thumb you can tell: About 10 to 20 real
> servers in
> NAT mode.
>

We use LVS-NAT, and load balance, in some of our clusters, 100 servers. The amount of traffic that LVS-NAT can handle is fully dependent on the amount of network bandwidth available, quality of NICs in the server, and CPU power.

--
Jason Faulkner
Linux Engineer, Rackspace Email & Apps
jason.faulkner [at] rackspace
o: (540) 443-2101 (ex. 505-2101)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 1, 2009, 6:13 AM

Post #10 of 34 (2832 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Hello,

Joseph Mack wrote:
>
> On Tue, 30 Jun 2009, Olaf Krische wrote:
>> Okay, i will try this out. Tho, i am not a big fan of giving each squid
>> an
>> IP. I do not own a big block.
> LVS-DR realservers have only private IPs
>

Okay, hereby i out myself as fresh and being totally stunned about the
magic:

This is my config that works so far:

load balancer:

echo 1 > /proc/sys/net/ipv4/ip_forward
ipvsadm -A -t $VIP:80 -s wlc
ipvsadm -a -t $VIP:80 -r $RIP1 -g
ipvsadm -a -t $VIP:80 -r $RIP2 -g

real server:

iptables -t nat -A PREROUTING -p tcp -d $VIP --dport 80 -j REDIRECT
--to-port 80

So, when i do netstat, i see something like that:

tcp 0 0 192.168.2.66:80 92.79.30.10:3769

But when i see the response coming back to my station, i see $VIP as the
source address.

Is this iptables, which rewrites from "192.168.2.66" to "$VIP" again?

How is this possible?

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24289572.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Jul 1, 2009, 6:51 AM

Post #11 of 34 (2837 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

On Wed, 1 Jul 2009, Olaf Krische wrote:

> So, when i do netstat, i see something like that:
>
> tcp 0 0 192.168.2.66:80 92.79.30.10:3769
>
> But when i see the response coming back to my station, i see $VIP as the
> source address.
>
> Is this iptables, which rewrites from "192.168.2.66" to "$VIP" again?

read how LVS-DR works. The HOWTO is one place

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 2, 2009, 11:35 AM

Post #12 of 34 (2828 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Hello again,

Joseph Mack wrote:
>
> On Tue, 30 Jun 2009, Olaf Krische wrote:
>
>> Okay, i will try this out. Tho, i am not a big fan of giving each squid
>> an
>> IP. I do not own a big block.
> LVS-DR realservers have only private IPs
>

Your statement kept me busy, because the target is not to waste IPs from the
pool. :-)

Director:

(eth0): 192.168.2.10
(eth1) 72.14.147.10
(eth1:0) 72.14.147.200 (VIP)

ipvsadm -A -t 72.14.147.200:80 -s rr
ipvsadm -a -t 72.14.147.200:80 -r 192.168.2.11 -g

Real:

(eth0): 192.168.2.11 (RIP)
(eth1) 72.14.147.11 (from ip pool)
default gw 72.14.147.1
iptables -t nat -A PREROUTING -p tcp -d 72.14.147.200 --dport 80 -j REDIRECT
--to-port 80

Works perfect.

Now i reduce.

1)

I remove the IP from eth1 on the real server. As expected i lose the access
to the default gateway. OK.

2)

Something like that does not work either:

ifconfig eth1 0 up
route add -host 72.14.147.1 dev eth1
route add default gw 72.14.147.1

3)

I change the default gateway to the one responsible for the private network.
Since it does NAT and does not allow to pass traffic otherwise, it does not
work either.

4)

I guess i have either to change the rules on that gateway, so it does pass
traffic "from VIP to internet" without NAT.

5)

Or i create a new private network, which works on eth1 and having a gateway
there, which passes the traffic.

I dont give up yet!

(and iam not google, just used their IP as an example)
--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24311944.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Jul 2, 2009, 11:58 AM

Post #13 of 34 (2824 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

On Thu, 2 Jul 2009, Olaf Krische wrote:

> Your statement kept me busy, because the target is not to waste IPs from the
> pool. :-)

how about you setup LVS-DR, loadbalancing telnet as in the
mini-HOWTO, and then move to squid

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 2, 2009, 2:16 PM

Post #14 of 34 (2819 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Hello Joseph,

Joseph Mack wrote:
>
> On Thu, 2 Jul 2009, Olaf Krische wrote:
>
>> Your statement kept me busy, because the target is not to waste IPs from
>> the
>> pool. :-)
> how about you setup LVS-DR, loadbalancing telnet as in the
> mini-HOWTO, and then move to squid
>

Your Mini-Howto, "5. Example: Setup LVS using LVS-DR forwarding":

I have it working exactly like this, only that i use non private IPs.

But i can not afford to use "non private IPs" for each realserver.

The realservers should be in a private network.

Are you still saying, i can find this in the howto? Then i will continue to
look. Aaaaah.
--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24314140.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 2, 2009, 2:19 PM

Post #15 of 34 (2822 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Hello Joseph,

Joseph Mack wrote:
>
> On Thu, 2 Jul 2009, Olaf Krische wrote:
>
>> Your statement kept me busy, because the target is not to waste IPs from
>> the
>> pool. :-)
> how about you setup LVS-DR, loadbalancing telnet as in the
> mini-HOWTO, and then move to squid
>

Your Mini-Howto, "5. Example: Setup LVS using LVS-DR forwarding":

I have it working exactly like this, only that i use non private IPs.

But i can not afford to use "non private IPs" for each realserver.

The realservers should be in a private network (even if they are connected
already with the net on an interface, without having an ip set)

Are you still saying, i can find this in the howto? Then i will continue to
look. Aaaaah.
--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24314140.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jjohndu at gmail

Jul 2, 2009, 3:32 PM

Post #16 of 34 (2827 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Olaf Krische wrote:
> Hello Joseph,
>
>
> Joseph Mack wrote:
>
>> On Thu, 2 Jul 2009, Olaf Krische wrote:
>>
>>
>>> Your statement kept me busy, because the target is not to waste IPs from
>>> the
>>> pool. :-)
>>>
>> how about you setup LVS-DR, loadbalancing telnet as in the
>> mini-HOWTO, and then move to squid
>>
>>
>
> Your Mini-Howto, "5. Example: Setup LVS using LVS-DR forwarding":
>
> I have it working exactly like this, only that i use non private IPs.
>
> But i can not afford to use "non private IPs" for each realserver.
>
> The realservers should be in a private network.
>
> Are you still saying, i can find this in the howto? Then i will continue to
> look. Aaaaah.
>

Why do you have to use public IPs for the real servers in a LVS-DR
setup? LVS-DR does not require the real servers to have public IPs. It
only requires that the load balancer and the real servers are on the
same sub-net.

I have a LVS-DR setup that does not use any public IPs on the load
balancer or on the real servers. You must have a reason that is not
obvious to us from what you have told us so far.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 2, 2009, 5:59 PM

Post #17 of 34 (2816 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

John Du-4 wrote:
>
> Why do you have to use public IPs for the real servers in a LVS-DR
> setup? LVS-DR does not require the real servers to have public IPs. It
> only requires that the load balancer and the real servers are on the
> same sub-net.
>

You guys really must think, i try to tease you, mh?

I would offer free beer, if i could, by receiving the patience
to follow me (yet)

If not using the public IPs, i could not route to the client.

This is how my brain imagines the routing:

1)

Without public IP address on real server:

Router's IP and directors IP are in the same network.

a) client traffic from Internet to VIP arrives at router A.

b) traffic is passed to the director.

c) director rewrites and passes the client traffic to the chosen realserver
in a LAN.

d) realserver answers to the client by using its default route, which points
to router B.

It can not use router A for the answer, router A is not reachable from
LAN.

Router B does NAT only. It translates LAN-IP to router B's IP and then
forwards to router A.

e) realserver answers to the client by sending traffic to router B

f) Router B receives traffic, which looks like traffic coming from VIP to a
client in the internet.

Router B does not know what to do. No traffic arrives to client.

Lost.

2)

With public IP address on real server:

router's IP, director's IP and realserver's IP are in the same network.

a) client traffic from Internet to VIP arrives at router A.

b) traffic is passed to the director.

c) director rewrites and passes the client traffic to the chosen realserver.

d) realserver answers to the client by using its default route, which points
to router A.

It can use router A for the answer, because router A is reachable.

e) realserver answers to the client by sending traffic to router A

f) router A receives traffic, which looks like traffic coming from VIP to a
client in the internet.

Router A knows what to do with it.

g) router A sends to its next hop, etc etc.

Conclusion:

2) is like the example in the mini-howto and it runs perfectly here.

1) is broken in my head. Probably i miss something big. I have no idea, what
i dont get,
especially when all of you are say: "it works here"

G'night

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24316166.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 2, 2009, 6:01 PM

Post #18 of 34 (2821 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

John Du-4 wrote:
>
> Why do you have to use public IPs for the real servers in a LVS-DR
> setup? LVS-DR does not require the real servers to have public IPs. It
> only requires that the load balancer and the real servers are on the
> same sub-net.
>

You guys really must think, i try to tease you, mh?

I would offer free beer, if i could, by receiving the patience
to follow me (yet)

If not using the public IPs, i could not route to the client.

This is how my brain imagines the routing:

1)

Without public IP address on real server:

Router's IP and directors IP are in the same network; director and
realserver are in the same LAN.

a) client traffic from Internet to VIP arrives at router A.

b) traffic is passed to the director.

c) director rewrites and passes the client traffic to the chosen realserver
in the LAN.

d) realserver answers to the client by using its default route, which points
to router B.

It can not use router A for the answer, router A is not reachable from
LAN.

Router B does NAT only. It translates LAN-IP to router B's IP and then
forwards to router A.

e) realserver answers to the client by sending traffic to router B

f) Router B receives traffic, which looks like traffic coming from VIP to a
client in the internet.

Router B does not know what to do. No traffic arrives to client.

Lost.

2)

With public IP address on real server:

router's IP, director's IP and realserver's IP are in the same network.

a) client traffic from Internet to VIP arrives at router A.

b) traffic is passed to the director.

c) director rewrites and passes the client traffic to the chosen realserver.

d) realserver answers to the client by using its default route, which points
to router A.

It can use router A for the answer, because router A is reachable.

e) realserver answers to the client by sending traffic to router A

f) router A receives traffic, which looks like traffic coming from VIP to a
client in the internet.

Router A knows what to do with it.

g) router A sends to its next hop, etc etc.

Conclusion:

2) is like the example in the mini-howto and it runs perfectly here.

1) is broken in my head. Probably i miss something big. I have no idea, what
i dont get,
especially when all of you are say: "it works here"

G'night

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24316166.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 2, 2009, 6:02 PM

Post #19 of 34 (2821 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

John Du-4 wrote:
>
> Why do you have to use public IPs for the real servers in a LVS-DR
> setup? LVS-DR does not require the real servers to have public IPs. It
> only requires that the load balancer and the real servers are on the
> same sub-net.
>

You guys really must think, i try to tease you, mh?

I would offer free beer, if i could, by receiving the patience
to follow me (yet)

If not using the public IPs, i could not route to the client.

This is how my brain imagines the routing:

1)

Without public IP address on real server:

Router's IP and directors IP are in the same network;
director has a second interface, and so being in the same LAN with the
realserver

a) client traffic from Internet to VIP arrives at router A.

b) traffic is passed to the director.

c) director rewrites and passes the client traffic to the chosen realserver
in the LAN.

d) realserver answers to the client by using its default route, which points
to router B.

It can not use router A for the answer, router A is not reachable from
LAN.

Router B does NAT only. It translates LAN-IP to router B's IP and then
forwards to router A.

e) realserver answers to the client by sending traffic to router B

f) Router B receives traffic, which looks like traffic coming from VIP to a
client in the internet.

Router B does not know what to do. No traffic arrives to client.

Lost.

2)

With public IP address on real server:

router's IP, director's IP and realserver's IP are in the same network.

a) client traffic from Internet to VIP arrives at router A.

b) traffic is passed to the director.

c) director rewrites and passes the client traffic to the chosen realserver.

d) realserver answers to the client by using its default route, which points
to router A.

It can use router A for the answer, because router A is reachable.

e) realserver answers to the client by sending traffic to router A

f) router A receives traffic, which looks like traffic coming from VIP to a
client in the internet.

Router A knows what to do with it.

g) router A sends to its next hop, etc etc.

Conclusion:

2) is like the example in the mini-howto and it runs perfectly here.

1) is broken in my head. Probably i miss something big. I have no idea, what
i dont get,
especially when all of you are say: "it works here"

G'night

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24316166.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Jul 2, 2009, 6:46 PM

Post #20 of 34 (2816 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

On Thu, 2 Jul 2009, Olaf Krische wrote:

> If not using the public IPs, i could not route to the client.

on the realserver the service is listening on the VIP (as it
must, because that's where the client sent the packet). The
realserver replies from the VIP. The packet goes back to the
client. The RIP is needed so that the director can find its
MAC address and send a layer 2 packet from MAC_DIP->MAC_RIP.

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 3, 2009, 6:18 AM

Post #21 of 34 (2796 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

Hello Joseph,

Joseph Mack wrote:
>
> On Thu, 2 Jul 2009, Olaf Krische wrote:
>> If not using the public IPs, i could not route to the client.
> on the realserver the service is listening on the VIP (as it
> must, because that's where the client sent the packet). The
> realserver replies from the VIP. The packet goes back to the
> client. The RIP is needed so that the director can find its
> MAC address and send a layer 2 packet from MAC_DIP->MAC_RIP.
>

And my problem is "how the packet goes back to the client".

I guess, i have become totally offtopic. Since it is a routing problem,
which has come up with using LVS-DR:

- CIP is "1.2.3.4"
- RIP on realserver is "192.168.1.2", default gw is "192.168.1.1"
- realservers answers to client by sending packet "VIP -> CIP" to gateway
- gateway must be enabled to forward "VIP -> CIP" to the next
gateway, which is responsible for "CIP".

Right now this gateway just does NAT. I have no new rule for this above yet.

The rule for the gateway must be something like:

- When traffic on eth0 and ip-src is "VIP", then forward to "internet
gateway" on eth1

I think thats the result of all the lines. :-)
--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24323032.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Jul 3, 2009, 9:51 AM

Post #22 of 34 (2778 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

On Fri, 3 Jul 2009, Olaf Krische wrote:

> And my problem is "how the packet goes back to the client".

you could read the HOWTO. It's explained in great detail

Joe
--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 3, 2009, 11:32 AM

Post #23 of 34 (2780 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

(Seriously, i read the howto. Sorry if it seems i dont.)

I am here:

7.7.2. Realserver's default gw, route to realserver from router

I have the realserver (192.168.2.xx) in the private network.
My default gateway for this network does NAT only. I can not use it
to route back to the client. It does not pass "VIP -> CIP" to the
next "internet" router.

So an alternative is what i need, which is not covered in the howto:

I can use the second interface (eth1) on the realserver,
setting on it an "internet" IP and changing the default route to the
"internet" router. Then i have a default route to the client,
which works perfectly.

$ ip route add default via $router dev eth1

But i dont want to do this, since i waste this exact IP plus the realserver
is reachable from the internet.

So i bring up eth1 without IP:

$ ifconfig eth1 0 up

And putting the VIP on lo:0:

$ ifconfig lo:0 $VIP netmask 255.255.255.255 up

And say, that all traffic shall go over eth1 in general:

$ ip route add default dev eth1

This is not enough, since it doesnt know yet,
that the traffic on eth1 shall go to $router.

So i try this by "ip route", which does not work:

$ ip route add default via $router dev eth1

But with the route command it works:

$ route add default gw $router dev eth1

Now i have all what i want! And it works. I can even ping.

But i am not sure, if that is alright. Not that the router thinks now,
that the VIP belongs to the realserver and does not use the
LoadBalancer anymore. Mh.

And what is weird now as well, when i watch the routing table,
that there it is written, what the "ip route" actually denied me to do:

$ ip route
...
default via $router dev eth1
default dev eth1 scope link

But I can remove the rule:

$ ip route del default via $router dev eth1

But i can not add it again with "ip route add", just with "route add" as
before.

# not working: ip route add default via $router dev eth1
# working: route add default gw $router dev eth1

Weird, weird, weird.

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24327104.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 3, 2009, 11:33 AM

Post #24 of 34 (2788 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

(Seriously, i read the howto. Sorry if it seems i dont.)

I am here:

7.7.2. Realserver's default gw, route to realserver from router

I have the realserver (192.168.2.xx) in the private network.
My default gateway for this network does NAT only. I can not use it
to route back to the client. It does not pass "VIP -> CIP" to the
next "internet" router.

So an alternative is what i need, which is not covered in the howto:

I can use the second interface (eth1) on the realserver,
setting on it an "internet" IP and changing the default route to the
"internet" router. Then i have a default route to the client,
which works perfectly.

$ ip route add default via $router dev eth1

But i dont want to do this, since i waste this exact IP plus the realserver
is reachable from the internet.

So i bring up eth1 without IP:

$ ifconfig eth1 0 up

And putting the VIP on lo:0:

$ ifconfig lo:0 $VIP netmask 255.255.255.255 up

And say, that all traffic shall go over eth1 in general:

$ ip route add default dev eth1

This is not enough, since it doesnt know yet,
that the traffic on eth1 shall go to $router.

So i try this by "ip route", which does not work:

$ ip route add default via $router dev eth1

But with the route command it works:

$ route add default gw $router dev eth1

Now i have all what i want! And it works. I can even ping.

But i am not sure, if that is alright. Not that the router has learned now,
that the VIP belongs to the realserver and does not use the
LoadBalancer anymore. Mh.

And what is weird now as well, when i watch the routing table,
that there it is written, what the "ip route" actually denied me to do:

$ ip route
...
default via $router dev eth1
default dev eth1 scope link

But I can remove the rule:

$ ip route del default via $router dev eth1

But i can not add it again with "ip route add", just with "route add" as
before.

# not working: ip route add default via $router dev eth1
# working: route add default gw $router dev eth1

Weird, weird, weird.

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24327104.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

public at ecopatz

Jul 3, 2009, 11:41 AM

Post #25 of 34 (2777 views)
Permalink

Re: [lvs-users] ipvs or apache/mod_proxy/mod_balancer [In reply to]

(Seriously, i read the howto. Sorry if it seems i dont.)

I am here:

7.7.2. Realserver's default gw, route to realserver from router

I have the realserver (192.168.2.xx) in the private network.
My default gateway for this network does NAT only. I can not use it
to route back to the client. It does not pass "VIP -> CIP" to the
next "internet" router.

So an alternative is what i need, which is not covered in the howto:

I can use the second interface (eth1) on the realserver,
setting on it an "internet" IP and changing the default route to the
"internet" router. Then i have a default route to the client,
which works perfectly.

$ ip route add default via $router dev eth1

But i dont want to do this, since i waste this exact IP plus the realserver
is reachable from the internet.

So i bring up eth1 without IP:

$ ifconfig eth1 0 up

And putting the VIP on lo:0:

$ ifconfig lo:0 $VIP netmask 255.255.255.255 up

And say, that all traffic shall go over eth1 in general:

$ ip route add default dev eth1

This is not enough, since it doesnt know yet,
that the traffic on eth1 shall go to $router.

So i try this by "ip route", which does not work:

$ ip route add default via $router dev eth1

But with the route command it works:

$ route add default gw $router dev eth1

Now i have all what i want! And it works. I can even ping.

But i am not sure, if that is alright. Not that the router has learned now,
that the VIP belongs to the realserver and does not use the
LoadBalancer anymore. Mh.

And what is weird now as well, when i watch the routing table,
that there it is written, what the "ip route" actually denied me to do:

$ ip route
...
default via $router dev eth1
default dev eth1 scope link

But I can remove the rule:

$ ip route del default via $router dev eth1

But i can not add it again with "ip route add", just with "route add" as
before.

# not working: ip route add default via $router dev eth1
# working: route add default gw $router dev eth1

Weird, weird, weird.

--
View this message in context: http://www.nabble.com/ipvs-or-apache-mod_proxy-mod_balancer-tp24184926p24327104.html
Sent from the LVS mailing list archive at Nabble.com.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads