Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux-HA: Users

Built in Affinity with LVS and LinuxHA?

 

 

Linux-HA users RSS feed   Index | Next | Previous | View Threaded


mgbutler at nbnet

Sep 7, 2010, 11:57 AM

Post #1 of 5 (284 views)
Permalink
Built in Affinity with LVS and LinuxHA?

Hello all,

I've implemented a LVS cluster using ldirectord and LinuxHA. Here is a
snippet from my ldirectord.cf file:

virtual=172.28.185.54:8080
protocol=tcp
scheduler=wrr
checktype=connect
checkport=8080
#service=ldap
real=172.28.185.57:8080 ipip
real=172.28.185.58:8080 ipip
#service=http
checktimeout=10
checkinterval=10

While running tests that hit the index.html page on the VIP above I
noticed that we seem to be hitting the same backend server repeatedly.
When I run the test from another server, we start hitting the other back
end server repeatedly. I'm curious if anyone can tell me if there is a
built in affinity here that I'm missing. I'm thinking that ldirecotor
seems to be smart enough (or dumb enough depending on your point of
view) to notice that a connection attempt from ip address guest will be
sent to the same backend server for productivity reasons.

Anyone know if thee is a natural built in affinity here?

_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


eschoeller at users

Sep 7, 2010, 2:07 PM

Post #2 of 5 (251 views)
Permalink
Re: Built in Affinity with LVS and LinuxHA? [In reply to]

Sounds to me like it might be using the iptables clusterip module.

Mike wrote:
> Hello all,
>
> I've implemented a LVS cluster using ldirectord and LinuxHA. Here is a
> snippet from my ldirectord.cf file:
>
> virtual=172.28.185.54:8080
> protocol=tcp
> scheduler=wrr
> checktype=connect
> checkport=8080
> #service=ldap
> real=172.28.185.57:8080 ipip
> real=172.28.185.58:8080 ipip
> #service=http
> checktimeout=10
> checkinterval=10
>
> While running tests that hit the index.html page on the VIP above I
> noticed that we seem to be hitting the same backend server repeatedly.
> When I run the test from another server, we start hitting the other back
> end server repeatedly. I'm curious if anyone can tell me if there is a
> built in affinity here that I'm missing. I'm thinking that ldirecotor
> seems to be smart enough (or dumb enough depending on your point of
> view) to notice that a connection attempt from ip address guest will be
> sent to the same backend server for productivity reasons.
>
> Anyone know if thee is a natural built in affinity here?
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA [at] lists
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


horms at verge

Sep 9, 2010, 2:15 AM

Post #3 of 5 (260 views)
Permalink
Re: Built in Affinity with LVS and LinuxHA? [In reply to]

On Tue, Sep 07, 2010 at 03:57:52PM -0300, Mike wrote:
> Hello all,
>
> I've implemented a LVS cluster using ldirectord and LinuxHA. Here is a
> snippet from my ldirectord.cf file:
>
> virtual=172.28.185.54:8080
> protocol=tcp
> scheduler=wrr
> checktype=connect
> checkport=8080
> #service=ldap
> real=172.28.185.57:8080 ipip
> real=172.28.185.58:8080 ipip
> #service=http
> checktimeout=10
> checkinterval=10
>
> While running tests that hit the index.html page on the VIP above I
> noticed that we seem to be hitting the same backend server repeatedly.
> When I run the test from another server, we start hitting the other back
> end server repeatedly. I'm curious if anyone can tell me if there is a
> built in affinity here that I'm missing. I'm thinking that ldirecotor
> seems to be smart enough (or dumb enough depending on your point of
> view) to notice that a connection attempt from ip address guest will be
> sent to the same backend server for productivity reasons.
>
> Anyone know if thee is a natural built in affinity here?

If you have persistence enabled, then yes that is the expected behaviour.
Otherwise its a bit odd.
_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


mgbutler at nbnet

Sep 10, 2010, 12:53 AM

Post #4 of 5 (257 views)
Permalink
Re: Built in Affinity with LVS and LinuxHA? [In reply to]

Thank you for the response Simon. Helpful as always.

I did some testing on this issue tonight and I've discovered an
interesting issue.

First of all I am using LVS-Tun and I have created a tunnel interface on
each of my back end servers. This tunnel interface is of course the VIP
you see below - 172.28.185.54.

When we started our tests this evening all requests were going to one
back end server, 172.28.185.57. As a test I decided to completely
shutdown the LVS servers by shutting down LinuxHA which shuts down
ldirectord.

To my surprise, I was still able to send connections to the VIP. When I
ssh'ed into the VIP I ended up on the back end server that was receiving
the connections. Of course the VIP should be down since LVS was down. I
rebooted both back end servers, started up LVS and like magic everything
worked. Load balancing was working perfectly.

So the problem was the backend server somehow took ownership of the VIP
and as a result was grabbing all requests. A reboot resolved it. Can you
tell me why this may have happened? What could be wrong on my backend
servers that they would grab the VIP like this?

Any help would be appreciated greatly.

Mike
On Thu, 2010-09-09 at 18:15 +0900, Simon Horman wrote:
> On Tue, Sep 07, 2010 at 03:57:52PM -0300, Mike wrote:
> > Hello all,
> >
> > I've implemented a LVS cluster using ldirectord and LinuxHA. Here is a
> > snippet from my ldirectord.cf file:
> >
> > virtual=172.28.185.54:8080
> > protocol=tcp
> > scheduler=wrr
> > checktype=connect
> > checkport=8080
> > #service=ldap
> > real=172.28.185.57:8080 ipip
> > real=172.28.185.58:8080 ipip
> > #service=http
> > checktimeout=10
> > checkinterval=10
> >
> > While running tests that hit the index.html page on the VIP above I
> > noticed that we seem to be hitting the same backend server repeatedly.
> > When I run the test from another server, we start hitting the other back
> > end server repeatedly. I'm curious if anyone can tell me if there is a
> > built in affinity here that I'm missing. I'm thinking that ldirecotor
> > seems to be smart enough (or dumb enough depending on your point of
> > view) to notice that a connection attempt from ip address guest will be
> > sent to the same backend server for productivity reasons.
> >
> > Anyone know if thee is a natural built in affinity here?
>
> If you have persistence enabled, then yes that is the expected behaviour.
> Otherwise its a bit odd.


_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


mgbutler at nbnet

Sep 10, 2010, 3:02 AM

Post #5 of 5 (261 views)
Permalink
Re: Built in Affinity with LVS and LinuxHA? [In reply to]

Think I got it - ARP problem.


On 10-09-10 04:53 AM, Mike wrote:
> Thank you for the response Simon. Helpful as always.
>
> I did some testing on this issue tonight and I've discovered an
> interesting issue.
>
> First of all I am using LVS-Tun and I have created a tunnel interface on
> each of my back end servers. This tunnel interface is of course the VIP
> you see below - 172.28.185.54.
>
> When we started our tests this evening all requests were going to one
> back end server, 172.28.185.57. As a test I decided to completely
> shutdown the LVS servers by shutting down LinuxHA which shuts down
> ldirectord.
>
> To my surprise, I was still able to send connections to the VIP. When I
> ssh'ed into the VIP I ended up on the back end server that was receiving
> the connections. Of course the VIP should be down since LVS was down. I
> rebooted both back end servers, started up LVS and like magic everything
> worked. Load balancing was working perfectly.
>
> So the problem was the backend server somehow took ownership of the VIP
> and as a result was grabbing all requests. A reboot resolved it. Can you
> tell me why this may have happened? What could be wrong on my backend
> servers that they would grab the VIP like this?
>
> Any help would be appreciated greatly.
>
> Mike
> On Thu, 2010-09-09 at 18:15 +0900, Simon Horman wrote:
>
>> On Tue, Sep 07, 2010 at 03:57:52PM -0300, Mike wrote:
>>
>>> Hello all,
>>>
>>> I've implemented a LVS cluster using ldirectord and LinuxHA. Here is a
>>> snippet from my ldirectord.cf file:
>>>
>>> virtual=172.28.185.54:8080
>>> protocol=tcp
>>> scheduler=wrr
>>> checktype=connect
>>> checkport=8080
>>> #service=ldap
>>> real=172.28.185.57:8080 ipip
>>> real=172.28.185.58:8080 ipip
>>> #service=http
>>> checktimeout=10
>>> checkinterval=10
>>>
>>> While running tests that hit the index.html page on the VIP above I
>>> noticed that we seem to be hitting the same backend server repeatedly.
>>> When I run the test from another server, we start hitting the other back
>>> end server repeatedly. I'm curious if anyone can tell me if there is a
>>> built in affinity here that I'm missing. I'm thinking that ldirecotor
>>> seems to be smart enough (or dumb enough depending on your point of
>>> view) to notice that a connection attempt from ip address guest will be
>>> sent to the same backend server for productivity reasons.
>>>
>>> Anyone know if thee is a natural built in affinity here?
>>>
>> If you have persistence enabled, then yes that is the expected behaviour.
>> Otherwise its a bit odd.
>>
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA [at] lists
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
>

_______________________________________________
Linux-HA mailing list
Linux-HA [at] lists
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Linux-HA users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.