Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux Virtual Server: Users

[lvs-users] WLC Slow start for 2.6.x kernel

 

 

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded


aleksey.chudov at gmail

Sep 28, 2011, 12:38 PM

Post #1 of 11 (856 views)
Permalink
[lvs-users] WLC Slow start for 2.6.x kernel

Hello!



Is the a WLC Slow start patch for 2.6.x kernels like described in
http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.ipvsadm.html#thunderi
ng_herd ?



Best regards,

Aleksey



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


horms at verge

Sep 29, 2011, 3:15 PM

Post #2 of 11 (823 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

On Wed, Sep 28, 2011 at 10:38:18PM +0300, Aleksey Chudov wrote:
> Hello!
>
>
>
> Is the a WLC Slow start patch for 2.6.x kernels like described in
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.ipvsadm.html#thunderi
> ng_herd ?

I did not write a 2.6 (or 3) version of that patch,
however I imagine that doing so would not be difficult.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


aleksey.chudov at gmail

Sep 30, 2011, 8:15 AM

Post #3 of 11 (819 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

It will be difficult because I'm not C/C++ programmer.
Could you please port this patch to 2.6 kernel?

Best regards,
Aleksey


-----Original Message-----
From: Simon Horman [mailto:horms [at] verge]
Sent: Friday, September 30, 2011 1:16 AM
To: Aleksey Chudov
Cc: lvs-users [at] linuxvirtualserver
Subject: Re: [lvs-users] WLC Slow start for 2.6.x kernel

On Wed, Sep 28, 2011 at 10:38:18PM +0300, Aleksey Chudov wrote:
> Hello!
>
>
>
> Is the a WLC Slow start patch for 2.6.x kernels like described in
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.ipvsadm.html#th
> underi
> ng_herd ?

I did not write a 2.6 (or 3) version of that patch, however I imagine that
doing so would not be difficult.


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


horms at verge

Sep 30, 2011, 8:05 PM

Post #4 of 11 (821 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

On Fri, Sep 30, 2011 at 06:15:58PM +0300, Aleksey Chudov wrote:
> It will be difficult because I'm not C/C++ programmer.
> Could you please port this patch to 2.6 kernel?

Sure, I will try and find some time to up-port the patch.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


aleksey.chudov at gmail

Oct 7, 2011, 2:44 AM

Post #5 of 11 (730 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

It will be difficult because I'm not C/C++ programmer.
Could you please port this patch to 2.6 kernel?

Best regards,
Aleksey


-----Original Message-----
From: Simon Horman [mailto:horms [at] verge]
Sent: Friday, September 30, 2011 1:16 AM
To: Aleksey Chudov
Cc: lvs-users [at] linuxvirtualserver
Subject: Re: [lvs-users] WLC Slow start for 2.6.x kernel

On Wed, Sep 28, 2011 at 10:38:18PM +0300, Aleksey Chudov wrote:
> Hello!
>
>
>
> Is the a WLC Slow start patch for 2.6.x kernels like described in
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.ipvsadm.html#th
> underi
> ng_herd ?

I did not write a 2.6 (or 3) version of that patch, however I imagine that
doing so would not be difficult.


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


aleksey.chudov at gmail

Oct 22, 2011, 3:42 PM

Post #6 of 11 (636 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

Hello!

Thank you for great work. We have few heavy loaded LVS system with a lot of connections.
I'll test patch as soon as possible.

Best regards,
Aleksey


-----Original Message-----
From: kswilczynski [at] googlemail [mailto:kswilczynski [at] googlemail] On Behalf Of Krzysztof Wilczynski
Sent: Saturday, October 22, 2011 4:08 PM
To: Aleksey Chudov; Simon Horman
Cc: lvs-users [at] linuxvirtualserver
Subject: WLC Slow start for 2.6.x kernel

Hello,

[...]
> > It will be difficult because I'm not C/C++ programmer.
> > Could you please port this patch to 2.6 kernel?
>
> Sure, I will try and find some time to up-port the patch.
[...]

I took Simon's patch and made an attempt to forward-port onto 2.6 and latest 3.0.x (working mainly with "ipvs-next" and "net-next" trees):

https://github.com/kwilczynski/miscellaneous/blob/master/ipvs-thundering-herd-2.6.patch
https://github.com/kwilczynski/miscellaneous/blob/master/ipvs-thundering-herd-3.0.patch

Making sure that it holds water in my test environment / set-up.
Unfortunately, I do not have the ability to test this under a real life load and/or in a network / system that was suffering from the problem that this patch is trying to address. I other words, my notebook solely is not enough to see real benefit of the slow-start introduced by the patch, since it will reach limit of its capacity long before I am able to create enough congestion to spot the difference in a statistically significant manner.

Therefore, would you help me Aleksey and test it for me? :-) I am asking for your assistance, since you requested the forward-port in the first place, therefore you are probably the person whom will benefit from it the most.

Hopefully, everything will work as it says on the tin, and if so, then we can start working on merging this with upstream kernel and exposing ability to toggle this functionality on-demand via /proc and/or /sys entries per algorithm.

Thanks for help in advance! :)

KW


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


krzysztof.wilczynski at linux

Oct 22, 2011, 4:35 PM

Post #7 of 11 (633 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

Hello,

> Thank you for great work.

You are very welcome! :)

> We have few heavy loaded LVS system with a lot of connections.

Sounds nice, but please treat this as experimental feature and
therefore be careful if your systems are mission critical, and your
production environment cannot run into an inconvenience at any time. I
would want to avoid breaking your infrastructure if possible :-)

> I'll test patch as soon as possible.
[...]

Amazing! Thank you in advance ;-)

KW

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


aleksey.chudov at gmail

Oct 23, 2011, 12:48 AM

Post #8 of 11 (624 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

Hello,

>> Thank you for great work.

> You are very welcome! :)

>> We have few heavy loaded LVS system with a lot of connections.

> Sounds nice, but please treat this as experimental feature and therefore be careful if your systems are mission critical,
> and your production environment cannot run into an inconvenience at any time. I would want to avoid breaking your infrastructure if possible :-)

I'll plan to install patched Kernel on backup LVS node and perform tests by directing traffic thru the slave node during night hours.

>> I'll test patch as soon as possible.

>Amazing! Thank you in advance ;-)

Could you please explain how exactly slow start mechanism works?
Is there any configuration for slow start period of time, limit for maximum connections or/and connection rate?

For example, by now we are using the following shell script to bring up failed real servers:
i=1; while [ $i -le 100 ]; do ipvsadm -e -t <VIP>:http -r <RS>:http -i -w $i; echo "$(date +"%F %T") Weight set to $i"; sleep 10; ((i++)); done

It is not the best solution because depending of servers load sleep interval must be changed.
So limiting maximum connections or/and connection rate sounds more convenient.

Best regards,
Aleksey



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


krzysztof.wilczynski at linux

Oct 23, 2011, 8:33 AM

Post #9 of 11 (631 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

Hello,

[...]
>> Sounds nice, but please treat this as experimental feature and therefore be careful if your systems are mission critical,
>> and your production environment cannot run into an inconvenience at any time. I would want to avoid breaking your infrastructure if possible :-)
>
> I'll plan to install patched Kernel on backup LVS node and perform tests by directing traffic thru the slave node during night hours.
[...]

Again, thank you for helping :-)

> Could you please explain how exactly slow start mechanism works?

In principle, the idea behind a slow-start (which only really applies
to least-connection shedulers family) is to make a real server appear
as a little bit busier than it really is in terms of number of active
connections it handles, either when it is initially added to the pool
or when its weight has changed.

This will in turn lower the number of new connections that a scheduler
will be keen on allocating to it and should prevent a real server from
being overwhelmed by a substantial number of connections appearing
suddenly for it to handle.

In order to make that happen, we introduce a concept of artrificial
handicap factor (we are simply making a real server less attractive,
so to speak), which will decay over time allowing for more connections
to be gradually handed over to the real server by the scheduler.

I hope it makes sense :-)

> Is there any configuration for slow start period of time, limit for maximum connections or/and connection rate?

At present, there is no way to set and/or adjust handicap factor
(which is calculated automatically) and any other thresholds like
maximum connections and/or connection rate.

> For example, by now we are using the following shell script to bring up failed real servers:
> i=1; while [ $i -le 100 ]; do ipvsadm -e -t <VIP>:http -r <RS>:http -i -w $i; echo "$(date +"%F %T") Weight set to $i"; sleep 10; ((i++)); done

Are you starting with weight set to 0?

> It is not the best solution because depending of servers load sleep interval must be changed.

Which is quite a manual procedure :-(

> So limiting maximum connections or/and connection rate sounds more convenient.

Perhaps a completely new scheduler should be introduced to address
some of these problems, or perhaps we should expose some values from
existing schedulers for users to tweak as needed. No idea at this
point in time :-)

KW

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


krzysztof.wilczynski at linux

Oct 24, 2011, 1:01 PM

Post #10 of 11 (615 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

Hello,

[...]

I am adding version that can be applied onto even older kernels. This
one was requested by Aleksey particularly for 2.6.34.6:

https://github.com/kwilczynski/miscellaneous/blob/master/ipvs-thundering-herd-2.6.34.6.patch

KW

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users


jmack at wm7d

Oct 25, 2011, 9:09 AM

Post #11 of 11 (613 views)
Permalink
Re: [lvs-users] WLC Slow start for 2.6.x kernel [In reply to]

On Sun, 23 Oct 2011, Krzysztof Wilczynski wrote:

> In principle, the idea behind a slow-start (which only really applies
> to least-connection shedulers family) is to make a real server appear
> as a little bit busier than it really is in terms of number of active
> connections it handles, either when it is initially added to the pool
> or when its weight has changed.
.
.
> In order to make that happen, we introduce a concept of artrificial
> handicap factor (we are simply making a real server less attractive,
> so to speak), which will decay over time allowing for more connections
> to be gradually handed over to the real server by the scheduler.

Is this just a matter of setting the number of connections
to some arbitary value (eg the same as the most connected
realserver) and letting the number of connections decay as
they time out (as happens for regular connections)?

Joe


--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.