Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux Virtual Server: Users

[lvs-users] problems about TCP/UDP connection timer and perstient template timer

 

 

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded


j.lu at scasy

Oct 15, 2009, 3:55 AM

Post #1 of 3 (251 views)
Permalink
[lvs-users] problems about TCP/UDP connection timer and perstient template timer

Hi,
I have set up a LVS-DR cluster with four consumer PCs: two Directors,
and two Real servers. Only one director is active, the other director is
a hot backup by heartbeat. On the director server, i installed
ldirectord (http://www.vergenet.net/linux/ldirectord/) to manage my real
server connection status. My ldirectord configure file in heartbeat is:

checktimeout=20
checkinterval=5
autoreload=yes
quiescent=no
virtual=45
real=192.168.0.240 gate 1 ".healthcheck.html", "OKAY"
real=192.168.0.241 gate 1 ".healthcheck.html", "OKAY"
service=http
persistent=36000
checkport=80
protocol=fwm
scheduler=wrr
checktype=negotiate

Everything works fine except for the timer on the director server. For
example, I use "ipvsadm -Lnc|grep ERR|grep 91.113.65.128" to get the
following persistent template:
IP 00:16 ERR! 91.113.65.128:0 0.0.0.45:0
192.168.0.240:0

After a while, I check it again and get the following result:
IP 00:50 ERR! 91.113.65.128:0 0.0.0.45:0
192.168.0.240:0

Note here the expiration is greater than 16 seconds!

I then check it every second using the above shell command, and found
that each time when the timeout for the persistent template is up, it
reset to 60 seconds again. So it never disappear from the lvs connection
table. Also this is the same for some TCP connections with FIN_WAIT
status. But I found some tempalte works with the correct 10 hour
(persistent=36000) expiration time.

I tried the backup director server, it never has such problems.

So my question is why there are some TCP connections and persistent
template's timer doesn't work as expected on the first director server?

My Kernel version is: 2.6.9-42; heartbeat: 2.0.8; ipvsadm: 1.24-6

Thanks!


regards,
jason



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users[at]LinuxVirtualServer.org
Send requests to lvs-users-request[at]LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


horms at verge

Oct 15, 2009, 5:28 AM

Post #2 of 3 (233 views)
Permalink
Re: [lvs-users] problems about TCP/UDP connection timer and perstient template timer [In reply to]

On Thu, Oct 15, 2009 at 06:55:47PM +0800, Jason Lu wrote:
> Hi,
> I have set up a LVS-DR cluster with four consumer PCs: two Directors,
> and two Real servers. Only one director is active, the other director is
> a hot backup by heartbeat. On the director server, i installed
> ldirectord (http://www.vergenet.net/linux/ldirectord/) to manage my real
> server connection status. My ldirectord configure file in heartbeat is:
>
> checktimeout=20
> checkinterval=5
> autoreload=yes
> quiescent=no
> virtual=45
> real=192.168.0.240 gate 1 ".healthcheck.html", "OKAY"
> real=192.168.0.241 gate 1 ".healthcheck.html", "OKAY"
> service=http
> persistent=36000
> checkport=80
> protocol=fwm
> scheduler=wrr
> checktype=negotiate
>
> Everything works fine except for the timer on the director server. For
> example, I use "ipvsadm -Lnc|grep ERR|grep 91.113.65.128" to get the
> following persistent template:
> IP 00:16 ERR! 91.113.65.128:0 0.0.0.45:0
> 192.168.0.240:0
>
> After a while, I check it again and get the following result:
> IP 00:50 ERR! 91.113.65.128:0 0.0.0.45:0
> 192.168.0.240:0
>
> Note here the expiration is greater than 16 seconds!
>
> I then check it every second using the above shell command, and found
> that each time when the timeout for the persistent template is up, it
> reset to 60 seconds again. So it never disappear from the lvs connection
> table. Also this is the same for some TCP connections with FIN_WAIT
> status. But I found some tempalte works with the correct 10 hour
> (persistent=36000) expiration time.
>
> I tried the backup director server, it never has such problems.
>
> So my question is why there are some TCP connections and persistent
> template's timer doesn't work as expected on the first director server?
>
> My Kernel version is: 2.6.9-42; heartbeat: 2.0.8; ipvsadm: 1.24-6

Hi Jason,

They are pretty ancient versions of both the kernel and heartbeat,
but upgrading is unlikely to change the behaviour that you are observing.

What I suspect is happening is that there are still connections
associated with the persistence template. And thus when its timeout
reaches zero it can't be removed and at that point its time is reset.
Repeat.

If that is what is happening then it is the expected behaviour - by which I
mean, that is how the code is written to work.


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users[at]LinuxVirtualServer.org
Send requests to lvs-users-request[at]LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


j.lu at scasy

Oct 15, 2009, 7:17 PM

Post #3 of 3 (230 views)
Permalink
Re: [lvs-users] problems about TCP/UDP connection timer and perstient template timer [In reply to]

Thank you for your reply. It is understandable the persistent template
may have connections associated with it. But it seems weird for FIN_WAIT
tcp connection to have the same observation. For example, the following
line:

TCP 00:46 FIN_WAIT 85.2.180.230:50224 192.168.0.245:https
192.168.0.241:https

The expiration time repeats with 60 seconds after counting down to zero!
(From my expectation, by default the timeout value should be 120
seconds, see the output of "ipvsadm -L --timeout") But not all such
FIN_WAIT connection have the same result. Only a fraction of them hangs
around in the connection table permanently.

Btw, it is possible to set the timeout for ESTABLISHED,FIN_WAIT timeout
in ldirectord? When I check with "ipvsadm -L --timeout", the result is:

Timeout (tcp tcpfin udp): 900 120 300
But I don't know how to set those values in ldirectord.


regards,
jason


On Thu, 2009-10-15 at 23:28 +1100, Simon Horman wrote:

> Hi Jason,
>
> They are pretty ancient versions of both the kernel and heartbeat,
> but upgrading is unlikely to change the behaviour that you are observing.
>
> What I suspect is happening is that there are still connections
> associated with the persistence template. And thus when its timeout
> reaches zero it can't be removed and at that point its time is reset.
> Repeat.
>
> If that is what is happening then it is the expected behaviour - by which I
> mean, that is how the code is written to work.
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users[at]LinuxVirtualServer.org
> Send requests to lvs-users-request[at]LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users[at]LinuxVirtualServer.org
Send requests to lvs-users-request[at]LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.