Mailing List Archive: Linux Virtual Server: Users

[lvs-users] annoying routing problem with a lvs cluster

Index | Next | Previous | View Threaded

dean.scothern at eduserv

Mar 1, 2012, 4:55 AM

Post #1 of 7 (470 views)
Permalink

[lvs-users] annoying routing problem with a lvs cluster

Hi,

I've been experimenting with a slightly non standard lvs cluster arrangement.

I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.

All machines are connected directly to the same two networks: frontend and backend.

The real servers/real clients connect to a service ip on the lvs machines on the frontend network.
The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.
I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.

This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.
I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.
Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.

Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.
Turning on routing had not beneficial effect either.

Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.
So far attempts to to do this have failed

Has anyone managed to do this?

Its more of a routing question so apologies for being slightly off topic.

Best Regards

Dean Scothern
Dr Dean Scothern
Infrastructure
[Description: Eduserv]
E: dean.scothern [at] eduserv<mailto:forename.surname [at] eduserv>

T: +44 (0)1225 474379

F: +44 (0)1225 474301

www.eduserv.org.uk<http://www.eduserv.org.uk/>
Eduserv is a company limited by guarantee (registered in England & Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.

Attachments:

image001.gif (1.00 KB)

david at davidcoulson

Mar 1, 2012, 5:03 AM

Post #2 of 7 (444 views)
Permalink

Re: [lvs-users] annoying routing problem with a lvs cluster [In reply to]

You need to SNAT real server traffic going to your real servers.

Quick google found this:

http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/

I'm presuming it's in mainline by now, but I know it's not in RHEL/SuSE yet.

David

On 3/1/12 7:55 AM, Dean Scothern wrote:
> Hi,
>
> I've been experimenting with a slightly non standard lvs cluster arrangement.
>
> I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.
>
> All machines are connected directly to the same two networks: frontend and backend.
>
> The real servers/real clients connect to a service ip on the lvs machines on the frontend network.
> The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.
> I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.
>
> This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.
> I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.
> Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.
>
> Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.
> Turning on routing had not beneficial effect either.
>
> Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.
> So far attempts to to do this have failed
>
> Has anyone managed to do this?
>
> Its more of a routing question so apologies for being slightly off topic.
>
> Best Regards
>
> Dean Scothern
> Dr Dean Scothern
> Infrastructure
> [Description: Eduserv]
> E: dean.scothern [at] eduserv<mailto:forename.surname [at] eduserv>
>
> T: +44 (0)1225 474379
>
> F: +44 (0)1225 474301
>
> www.eduserv.org.uk<http://www.eduserv.org.uk/>
> Eduserv is a company limited by guarantee (registered in England& Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.
>
>
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

dean.scothern at eduserv

Mar 1, 2012, 5:59 AM

Post #3 of 7 (443 views)
Permalink

Re: [lvs-users] annoying routing problem with a lvs cluster [In reply to]

Thank you for your quick reply.

Whilst snat would work I would prefer not to use it as it hides the source ip of the packets, making applications that use ip access lists more problem atic to configure, eg mailservers. Eventually I would expand the clients to include other networks (internet), and would like log analysis to work.
I would prefer not to use a proxy and pass magic headers with the remote ip them either.
The link in question also probably cannot easily apply to redhat/centos 6 as they are based on 2.6.32 kernel and the link mentions 2.6.35, 2.6.36.
Reading further it might be possible to apply the patch set and rebuild the associated kernel modules.

To be honest I hoping for some route configuration magicry, I feel so close and surely there must be a way.

Many Thanks

From: David Coulson [mailto:david [at] davidcoulson]
Sent: 01 March 2012 13:04
To: LinuxVirtualServer.org users mailing list.
Cc: Dean Scothern
Subject: Re: [lvs-users] annoying routing problem with a lvs cluster

You need to SNAT real server traffic going to your real servers.

Quick google found this:

http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/

I'm presuming it's in mainline by now, but I know it's not in RHEL/SuSE yet.

David

On 3/1/12 7:55 AM, Dean Scothern wrote:

Hi,

I've been experimenting with a slightly non standard lvs cluster arrangement.

I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.

All machines are connected directly to the same two networks: frontend and backend.

The real servers/real clients connect to a service ip on the lvs machines on the frontend network.

The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.

I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.

This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.

I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.

Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.

Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.

Turning on routing had not beneficial effect either.

Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.

So far attempts to to do this have failed

Has anyone managed to do this?

Its more of a routing question so apologies for being slightly off topic.

Best Regards

Dean Scothern

Dr Dean Scothern

Infrastructure

[Description: Eduserv]

E: dean.scothern [at] eduserv<mailto:dean.scothern [at] eduserv><mailto:forename.surname [at] eduserv><mailto:forename.surname [at] eduserv>

T: +44 (0)1225 474379

F: +44 (0)1225 474301

www.eduserv.org.uk<http://www.eduserv.org.uk><http://www.eduserv.org.uk/><http://www.eduserv.org.uk/>

Eduserv is a company limited by guarantee (registered in England & Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.

_______________________________________________

Please read the documentation before posting - it's available at:

http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer<mailto:lvs-users [at] LinuxVirtualServer>

Send requests to lvs-users-request [at] LinuxVirtualServer<mailto:lvs-users-request [at] LinuxVirtualServer>

or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

david at davidcoulson

Mar 1, 2012, 6:12 AM

Post #4 of 7 (445 views)
Permalink

Re: [lvs-users] annoying routing problem with a lvs cluster [In reply to]

You can either snat or run in dr or tun mode.

Sent from my iPhone

On Mar 1, 2012, at 8:59 AM, Dean Scothern <dean.scothern [at] eduserv> wrote:

> Thank you for your quick reply.
>
> Whilst snat would work I would prefer not to use it as it hides the source ip of the packets, making applications that use ip access lists more problem atic to configure, eg mailservers. Eventually I would expand the clients to include other networks (internet), and would like log analysis to work.
> I would prefer not to use a proxy and pass magic headers with the remote ip them either.
> The link in question also probably cannot easily apply to redhat/centos 6 as they are based on 2.6.32 kernel and the link mentions 2.6.35, 2.6.36.
> Reading further it might be possible to apply the patch set and rebuild the associated kernel modules.
>
> To be honest I hoping for some route configuration magicry, I feel so close and surely there must be a way.
>
> Many Thanks
>
>
> From: David Coulson [mailto:david [at] davidcoulson]
> Sent: 01 March 2012 13:04
> To: LinuxVirtualServer.org users mailing list.
> Cc: Dean Scothern
> Subject: Re: [lvs-users] annoying routing problem with a lvs cluster
>
> You need to SNAT real server traffic going to your real servers.
>
> Quick google found this:
>
> http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/
>
> I'm presuming it's in mainline by now, but I know it's not in RHEL/SuSE yet.
>
> David
>
> On 3/1/12 7:55 AM, Dean Scothern wrote:
> Hi,
>
> I've been experimenting with a slightly non standard lvs cluster arrangement.
>
> I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.
>
> All machines are connected directly to the same two networks: frontend and backend.
>
> The real servers/real clients connect to a service ip on the lvs machines on the frontend network.
> The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.
> I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.
>
> This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.
> I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.
> Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.
>
> Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.
> Turning on routing had not beneficial effect either.
>
> Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.
> So far attempts to to do this have failed
>
> Has anyone managed to do this?
>
> Its more of a routing question so apologies for being slightly off topic.
>
> Best Regards
>
> Dean Scothern
> Dr Dean Scothern
> Infrastructure
> [Description: Eduserv]
> E: dean.scothern [at] eduserv<mailto:forename.surname [at] eduserv>
>
> T: +44 (0)1225 474379
>
> F: +44 (0)1225 474301
>
> www.eduserv.org.uk<http://www.eduserv.org.uk/>
> Eduserv is a company limited by guarantee (registered in England & Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.
>
>
>
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

david at davidcoulson

Mar 1, 2012, 1:36 PM

Post #5 of 7 (441 views)
Permalink

Re: [lvs-users] annoying routing problem with a lvs cluster [In reply to]

Had a thought on this - You could use iptables to mark your return packet, then run it through a separate routing table based on a 'i ip ru add fwmark x table y' option. Your separate routing table would just have a default gw pointing to the VIP from your LVS cluster.

Not sure if that would work well, but it's worth a shot.

On Mar 1, 2012, at 8:59 AM, Dean Scothern wrote:

> Thank you for your quick reply.
>
> Whilst snat would work I would prefer not to use it as it hides the source ip of the packets, making applications that use ip access lists more problem atic to configure, eg mailservers. Eventually I would expand the clients to include other networks (internet), and would like log analysis to work.
> I would prefer not to use a proxy and pass magic headers with the remote ip them either.
> The link in question also probably cannot easily apply to redhat/centos 6 as they are based on 2.6.32 kernel and the link mentions 2.6.35, 2.6.36.
> Reading further it might be possible to apply the patch set and rebuild the associated kernel modules.
>
> To be honest I hoping for some route configuration magicry, I feel so close and surely there must be a way.
>
> Many Thanks
>
>
> From: David Coulson [mailto:david [at] davidcoulson]
> Sent: 01 March 2012 13:04
> To: LinuxVirtualServer.org users mailing list.
> Cc: Dean Scothern
> Subject: Re: [lvs-users] annoying routing problem with a lvs cluster
>
> You need to SNAT real server traffic going to your real servers.
>
> Quick google found this:
>
> http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/
>
> I'm presuming it's in mainline by now, but I know it's not in RHEL/SuSE yet.
>
> David
>
> On 3/1/12 7:55 AM, Dean Scothern wrote:
> Hi,
>
> I've been experimenting with a slightly non standard lvs cluster arrangement.
>
> I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.
>
> All machines are connected directly to the same two networks: frontend and backend.
>
> The real servers/real clients connect to a service ip on the lvs machines on the frontend network.
> The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.
> I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.
>
> This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.
> I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.
> Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.
>
> Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.
> Turning on routing had not beneficial effect either.
>
> Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.
> So far attempts to to do this have failed
>
> Has anyone managed to do this?
>
> Its more of a routing question so apologies for being slightly off topic.
>
> Best Regards
>
> Dean Scothern
> Dr Dean Scothern
> Infrastructure
> [Description: Eduserv]
> E: dean.scothern [at] eduserv<mailto:forename.surname [at] eduserv>
>
> T: +44 (0)1225 474379
>
> F: +44 (0)1225 474301
>
> www.eduserv.org.uk<http://www.eduserv.org.uk/>
> Eduserv is a company limited by guarantee (registered in England & Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.
>
>
>
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

dean.scothern at eduserv

Mar 2, 2012, 2:12 AM

Post #6 of 7 (436 views)
Permalink

Re: [lvs-users] annoying routing problem with a lvs cluster [In reply to]

Thanks,

It's worth a try but I think it will fall foul of the host route in the local table, which will apply first.

Best Regards

From: David Coulson [mailto:david [at] davidcoulson]
Sent: 01 March 2012 21:36
To: Dean Scothern
Cc: LinuxVirtualServer.org users mailing list.
Subject: Re: [lvs-users] annoying routing problem with a lvs cluster

Had a thought on this - You could use iptables to mark your return packet, then run it through a separate routing table based on a 'i ip ru add fwmark x table y' option. Your separate routing table would just have a default gw pointing to the VIP from your LVS cluster.

Not sure if that would work well, but it's worth a shot.

On Mar 1, 2012, at 8:59 AM, Dean Scothern wrote:

Thank you for your quick reply.

Whilst snat would work I would prefer not to use it as it hides the source ip of the packets, making applications that use ip access lists more problem atic to configure, eg mailservers. Eventually I would expand the clients to include other networks (internet), and would like log analysis to work.
I would prefer not to use a proxy and pass magic headers with the remote ip them either.
The link in question also probably cannot easily apply to redhat/centos 6 as they are based on 2.6.32 kernel and the link mentions 2.6.35, 2.6.36.
Reading further it might be possible to apply the patch set and rebuild the associated kernel modules.

To be honest I hoping for some route configuration magicry, I feel so close and surely there must be a way.

Many Thanks

From: David Coulson [mailto:david [at] davidcoulson]<mailto:[mailto:david [at] davidcoulson]>
Sent: 01 March 2012 13:04
To: LinuxVirtualServer.org<http://LinuxVirtualServer.org> users mailing list.
Cc: Dean Scothern
Subject: Re: [lvs-users] annoying routing problem with a lvs cluster

You need to SNAT real server traffic going to your real servers.

Quick google found this:

http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/

I'm presuming it's in mainline by now, but I know it's not in RHEL/SuSE yet.

David

On 3/1/12 7:55 AM, Dean Scothern wrote:

Hi,

I've been experimenting with a slightly non standard lvs cluster arrangement.

I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.

All machines are connected directly to the same two networks: frontend and backend.

The real servers/real clients connect to a service ip on the lvs machines on the frontend network.

The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.

I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.

This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.

I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.

Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.

Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.

Turning on routing had not beneficial effect either.

Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.

So far attempts to to do this have failed

Has anyone managed to do this?

Its more of a routing question so apologies for being slightly off topic.

Best Regards

Dean Scothern

Dr Dean Scothern

Infrastructure

[Description: Eduserv]

E: dean.scothern [at] eduserv<mailto:dean.scothern [at] eduserv><mailto:forename.surname [at] eduserv><mailto:forename.surname [at] eduserv>

T: +44 (0)1225 474379

F: +44 (0)1225 474301

www.eduserv.org.uk<http://www.eduserv.org.uk><http://www.eduserv.org.uk/><http://www.eduserv.org.uk/>

Eduserv is a company limited by guarantee (registered in England & Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.

_______________________________________________

Please read the documentation before posting - it's available at:

http://www.linuxvirtualserver.org/

LinuxVirtualServer.org<http://LinuxVirtualServer.org> mailing list - lvs-users [at] LinuxVirtualServer<mailto:lvs-users [at] LinuxVirtualServer>

Send requests to lvs-users-request [at] LinuxVirtualServer<mailto:lvs-users-request [at] LinuxVirtualServer>

or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

dean.scothern at eduserv

Mar 8, 2012, 4:45 AM

Post #7 of 7 (357 views)
Permalink

Re: [lvs-users] annoying routing problem with a lvs cluster [In reply to]

Hi,

It seems that the solution to my problem involves the use of /proc/sys/net/ipv4/conf/eth<n>/accept_local
For test purposes I've installed a more recent kernel that offers this functionality and I've been able to test it with a simple
Eth to eth external test.
So far I've been unsuccessful with the more complicated policy routing scenario.
Has anyone used accept_local with lvs nat to make something like the scenario I outlined earlier work?

Best Regards

-----Original Message-----
From: lvs-users-bounces [at] linuxvirtualserver [mailto:lvs-users-bounces [at] linuxvirtualserver] On Behalf Of Dean Scothern
Sent: 02 March 2012 10:12
To: David Coulson
Cc: LinuxVirtualServer.org users mailing list.
Subject: Re: [lvs-users] annoying routing problem with a lvs cluster

Thanks,

It's worth a try but I think it will fall foul of the host route in the local table, which will apply first.

Best Regards

From: David Coulson [mailto:david [at] davidcoulson]
Sent: 01 March 2012 21:36
To: Dean Scothern
Cc: LinuxVirtualServer.org users mailing list.
Subject: Re: [lvs-users] annoying routing problem with a lvs cluster

Had a thought on this - You could use iptables to mark your return packet, then run it through a separate routing table based on a 'i ip ru add fwmark x table y' option. Your separate routing table would just have a default gw pointing to the VIP from your LVS cluster.

Not sure if that would work well, but it's worth a shot.

On Mar 1, 2012, at 8:59 AM, Dean Scothern wrote:

Thank you for your quick reply.

Whilst snat would work I would prefer not to use it as it hides the source ip of the packets, making applications that use ip access lists more problem atic to configure, eg mailservers. Eventually I would expand the clients to include other networks (internet), and would like log analysis to work.
I would prefer not to use a proxy and pass magic headers with the remote ip them either.
The link in question also probably cannot easily apply to redhat/centos 6 as they are based on 2.6.32 kernel and the link mentions 2.6.35, 2.6.36.
Reading further it might be possible to apply the patch set and rebuild the associated kernel modules.

To be honest I hoping for some route configuration magicry, I feel so close and surely there must be a way.

Many Thanks

From: David Coulson [mailto:david [at] davidcoulson]<mailto:[mailto:david [at] davidcoulson]>
Sent: 01 March 2012 13:04
To: LinuxVirtualServer.org<http://LinuxVirtualServer.org> users mailing list.
Cc: Dean Scothern
Subject: Re: [lvs-users] annoying routing problem with a lvs cluster

You need to SNAT real server traffic going to your real servers.

Quick google found this:

http://blog.loadbalancer.org/enabling-snat-in-lvs-xt_ipvs-and-iptables/

I'm presuming it's in mainline by now, but I know it's not in RHEL/SuSE yet.

David

On 3/1/12 7:55 AM, Dean Scothern wrote:

Hi,

I've been experimenting with a slightly non standard lvs cluster arrangement.

I have a set of combined real servers/real clients (each machine has both services and clients) and two machines running lvs as a cluster.

All machines are connected directly to the same two networks: frontend and backend.

The real servers/real clients connect to a service ip on the lvs machines on the frontend network.

The lvs machines run in masq mode and connect to the real servers/real clients on the backend network.

I've configured policy routing on the real servers/real clients backend interfaces to return traffic via a second gateway on the lvs hosts.

This works very well except when a real server/real client connects to its own backend interface via the lvs cluster ip.

I guessing that the local host route means that instead of returning the traffic via the backend gateway on the lvs it tries to go directly locally.

Tcpdump appears to support this guess and if I turn on martian logging I can see the traffic.

Initially I thought that reverse path filtering was preventing operation but the problem remained when it was disabled.

Turning on routing had not beneficial effect either.

Ideally I would like to setup routing to override the local table when the policy routing rules are applied, but I'm not sure how.

So far attempts to to do this have failed

Has anyone managed to do this?

Its more of a routing question so apologies for being slightly off topic.

Best Regards

Dean Scothern

Dr Dean Scothern

Infrastructure

[Description: Eduserv]

E: dean.scothern [at] eduserv<mailto:dean.scothern [at] eduserv><mailto:forename.surname [at] eduserv><mailto:forename.surname [at] eduserv>

T: +44 (0)1225 474379

F: +44 (0)1225 474301

www.eduserv.org.uk<http://www.eduserv.org.uk><http://www.eduserv.org.uk/><http://www.eduserv.org.uk/>

Eduserv is a company limited by guarantee (registered in England & Wales, company number: 3763109) and a charity (charity number 1079456), whose registered office is at Royal Mead, Railway Place, Bath, BA1 1SR.

_______________________________________________

Please read the documentation before posting - it's available at:

http://www.linuxvirtualserver.org/

LinuxVirtualServer.org<http://LinuxVirtualServer.org> mailing list - lvs-users [at] LinuxVirtualServer<mailto:lvs-users [at] LinuxVirtualServer>

Send requests to lvs-users-request [at] LinuxVirtualServer<mailto:lvs-users-request [at] LinuxVirtualServer>

or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads