horms at verge
Dec 20, 2010, 2:41 PM
Post #2 of 3
(662 views)
Permalink
|
|
Re: [lvs-users] ipvs does not sync DNATted or fwmarked connection state
[In reply to]
|
|
On Mon, Dec 20, 2010 at 09:43:35PM +0100, Patrick Schaaf wrote:
> Is the following known / does a solution exist?
>
> I'm setting up two machines with kernel 2.6.36.2 as master/backup ipvs
> directors, with keepalived checking real servers and implementing vrrp
> failover.
>
> Virtual service is for HTTP connections, using NAT method towards the
> real servers.
>
> The basic setup has been working fine, with an exemplary set of three
> virtual IPs balancing to some real servers, replicating connection state
> (ipvsadm -ln counters increasing on the backup, -lc state visible
> there).
>
> However, for the production setup, I have to implement roughly 200
> different virtual IP addresses, all running onto the same (rather small)
> set of real servers.
>
> As is well known, doing that with the corresponding number of different
> ipvs virtual services presents problems, as the real server state
> (connection count) is kept for each individual virtual service,
> resulting in suboptimal balancing.
>
> As a solution to that, I have been testing two different approaches:
>
> 1) using fwmark, with --set-mark in the mangle table to mark the
> incoming packets for the different virtual IPs, and an fwmark virtual
> service set up as usual.
> iptables -t mangle -A PREROUTING -m ... -j MARK --set-mark 80
> ipvsadm -A -f 80 ...
>
> and alternatively
>
> 2) using iptables DNAT in PREROUTING to rewrite the various virtual IPs
> to specific (few) virtual IPs set up as ipvs services.
> iptables -t nat -A PREROUTING -m ... -j DNAT --to-dest 10.0.0.1
> ipvsadm -A -t 10.0.0.1:80 ...
>
> Both approaches work fine WRT balancing, reaching the real servers, and
> everything.
>
> BUT: no connection state is synchronized, in either of the approaches.
> The backup server does not show -ln counter increase, nor -lc
> connections, when I test it.
>
> I have even set up the fully working (normal) approach at the same time
> as as 1) and/or 2), for different addresses, and the sync-to-backup is
> working OK for the normal addresses, but not sending connection state
> for stuff covered by approaches 1) or 2).
>
> Any suggestions as to why this happens? Patches to apply? Good chance
> 2.6.37-rcX could work? More info needed?
Hi Patrick,
while there are a number of limitations in the synchronisation code
I believe that what you are trying to do should work. On the backup
do you see connections showing up in the output of ipvsadm -Lcn ?
On the topic of connection synchronisation, a new synchronisation
protocol (v1) has been developed which addresses a number of the problems
in the existing code (v0). This includes the explicit synchronisation
of fwmarks. The code is currently available in the lvs-test-2.6 tree
on git.kernel.org. It should appear in 2.6.38-rc1.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
