Mailing List Archive: Linux Virtual Server: Users

[lvs-users] FTP in active mode?

Index | Next | Previous | View Threaded

nicolas.haller at corp

Sep 16, 2009, 6:38 AM

Post #1 of 12 (1681 views)
Permalink

[lvs-users] FTP in active mode?

Hi all,

I try to make a load-balanced ftp and I have a little problem. I can't
use FTP in active mode. Someone can help me?

The lvsadm conf
TCP lo-pp.nerim.net:ftp wlc persistent 120
-> harrisburg.nerim.net:ftp Tunnel 1 0 0
-> sellafield.nerim.net:ftp Tunnel 1 0 0

Thanks,

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

malcolm at loadbalancer

Sep 16, 2009, 6:43 AM

Post #2 of 12 (1614 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

Nicolas,

With TUN or DR mode its probably best to use firewall marks to group
all of the possible ports together:

active 20,21
passive 21,high_port

# This example marks groups the active FTP ports
VIP1="192.168.0.66"
# First two rules are for Active connections
iptables -t mangle -A PREROUTING -p tcp -d $VIP1 --dport 21 -j MARK
--set-mark 1
iptables -t mangle -A PREROUTING -p tcp -d $VIP1 --dport 20 -j MARK
--set-mark 1
# Third additional rule for passive
iptables -t mangle -A PREROUTING -p tcp -d $VIP1 --dport 1024: -j
MARK --set-mark 1

2009/9/16 Nicolas Haller <nicolas.haller [at] corp>
>
> Hi all,
>
> I try to make a load-balanced ftp and I have a little problem. I can't
> use FTP in active mode. Someone can help me?
>
> The lvsadm conf
> TCP lo-pp.nerim.net:ftp wlc persistent 120
> -> harrisburg.nerim.net:ftp Tunnel 1 0 0
> -> sellafield.nerim.net:ftp Tunnel 1 0 0
>
> Thanks,
>
> --
> Nicolas Haller
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

--
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

nicolas.haller at corp

Sep 16, 2009, 7:39 AM

Post #3 of 12 (1623 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, Sep 16, 2009 at 02:43:53PM +0100, Malcolm Turnbull wrote:
> Nicolas,

> With TUN or DR mode its probably best to use firewall marks to group
> all of the possible ports together:

> active 20,21
> passive 21,high_port

> # This example marks groups the active FTP ports
> VIP1="192.168.0.66"
> # First two rules are for Active connections
> iptables -t mangle -A PREROUTING -p tcp -d $VIP1 --dport 21 -j MARK
> --set-mark 1
> iptables -t mangle -A PREROUTING -p tcp -d $VIP1 --dport 20 -j MARK
> --set-mark 1
> # Third additional rule for passive
> iptables -t mangle -A PREROUTING -p tcp -d $VIP1 --dport 1024: -j
> MARK --set-mark 1

Malcolm,

Thanks for your answer. It's a good track but it's seem LVS don't care
about my marks :-)

here is my new ipvsadm conf
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP lo-pp.nerim.net:www wlc persistent 120
-> harrisburg.nerim.net:www Tunnel 1 0 0
-> sellafield.nerim.net:www Tunnel 1 0 0
FWM 1 wlc persistent 120
-> harrisburg.nerim.net:0 Tunnel 1 0 0
-> sellafield.nerim.net:0 Tunnel 1 0 0

and the iptables conf
Chain PREROUTING (policy ACCEPT 20638 packets, 3081K bytes)
pkts bytes target prot opt in out source destination
14 880 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpt:ftp MARK xset 0x1/0xffffffff
1 64 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpt:ftp-data MARK xset 0x1/0xffffffff
0 0 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpts:1024:65535 MARK xset 0x1/0xffffffff

As you can see, some packets match these mark rules but they are not load-balanced.

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

malcolm at loadbalancer

Sep 16, 2009, 7:46 AM

Post #4 of 12 (1620 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

Nicolas,

Not sure but don't you need to remove the first clashing entry?

TCP lo-pp.nerim.net:www wlc persistent 120
-> harrisburg.nerim.net:www Tunnel 1 0 0
-> sellafield.nerim.net:www Tunnel 1 0 0

i.e. they both match the same dest ip?

> Malcolm,
>
> Thanks for your answer. It's a good track but it's seem LVS don't care
> about my marks :-)
>
> here is my new ipvsadm conf
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP lo-pp.nerim.net:www wlc persistent 120
> -> harrisburg.nerim.net:www Tunnel 1 0 0
> -> sellafield.nerim.net:www Tunnel 1 0 0
> FWM 1 wlc persistent 120
> -> harrisburg.nerim.net:0 Tunnel 1 0 0
> -> sellafield.nerim.net:0 Tunnel 1 0 0
>
> and the iptables conf
> Chain PREROUTING (policy ACCEPT 20638 packets, 3081K bytes)
> pkts bytes target prot opt in out source destination
> 14 880 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpt:ftp MARK xset 0x1/0xffffffff
> 1 64 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpt:ftp-data MARK xset 0x1/0xffffffff
> 0 0 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpts:1024:65535 MARK xset 0x1/0xffffffff
>
> As you can see, some packets match these mark rules but they are not load-balanced.
>
> --
> Nicolas Haller
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>

--
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

nicolas.haller at corp

Sep 16, 2009, 8:11 AM

Post #5 of 12 (1618 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, Sep 16, 2009 at 03:46:08PM +0100, Malcolm Turnbull wrote:
> Nicolas,

> Not sure but don't you need to remove the first clashing entry?

> TCP lo-pp.nerim.net:www wlc persistent 120
> -> harrisburg.nerim.net:www Tunnel 1 0 0
> -> sellafield.nerim.net:www Tunnel 1 0 0

> i.e. they both match the same dest ip?

Yes it's the same dest ip. I've tried to keep only the FWM rule but
there is no effect.

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

jmack at wm7d

Sep 16, 2009, 8:28 AM

Post #6 of 12 (1626 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, 16 Sep 2009, Nicolas Haller wrote:

> Hi all,
>
> I try to make a load-balanced ftp and I have a little problem. I can't
> use FTP in active mode. Someone can help me?

there's a lengthy description of ftp in the HOWTO. Does it
not work anymore?

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

nicolas.haller at corp

Sep 16, 2009, 9:43 AM

Post #7 of 12 (1616 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, Sep 16, 2009 at 05:11:10PM +0200, Nicolas Haller wrote:
> On Wed, Sep 16, 2009 at 03:46:08PM +0100, Malcolm Turnbull wrote:
> > Nicolas,

> > Not sure but don't you need to remove the first clashing entry?

> > TCP lo-pp.nerim.net:www wlc persistent 120
> > -> harrisburg.nerim.net:www Tunnel 1 0 0
> > -> sellafield.nerim.net:www Tunnel 1 0 0

> > i.e. they both match the same dest ip?

> Yes it's the same dest ip. I've tried to keep only the FWM rule but
> there is no effect.

Ok, I confirm netfilter put marks on packets. It's seems IPVS don't care
about marks.

Here some logs:

Sep 16 18:38:58 balancoire-1j kernel: [ 1886.901835] IN=eth0 OUT=
MAC=00:30:48:5a:4e:4e:00:1f:9d:9f:89:42:08:00 SRC=62.4.16.150
DST=194.79.128.128 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=52649 DF
PROTO=TCP SPT=52072 DPT=21 WINDOW=65535 RES=0x00 SYN URGP=0 MARK=0x1

Sep 16 18:38:58 balancoire-1j kernel: [ 1886.902074] IPVS: lookup/in TCP
62.4.16.150:52072->194.79.128.128:21 not hit

Sep 16 18:38:58 balancoire-1j kernel: [ 1886.902078] IPVS: lookup/out
TCP 62.4.16.150:52072->194.79.128.128:21 not hit

Sep 16 18:38:58 balancoire-1j kernel: [ 1886.902086] IPVS: p-schedule:
src 62.4.16.150:52072 dest 194.79.128.128:21 mnet 62.4.16.150

Sep 16 18:38:58 balancoire-1j kernel: [ 1886.902091] IPVS: template
lookup/in IP 62.4.16.150:0->0.0.0.1:0 not hit

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

heder at google

Sep 16, 2009, 1:24 PM

Post #8 of 12 (1615 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, Sep 16, 2009 at 15:38, Nicolas Haller
<nicolas.haller [at] corp> wrote:
> I try to make a load-balanced ftp and I have a little problem. I can't
> use FTP in active mode. Someone can help me?
>
> The lvsadm conf
> TCP lo-pp.nerim.net:ftp wlc persistent 120
> -> harrisburg.nerim.net:ftp Tunnel 1 0 0
> -> sellafield.nerim.net:ftp Tunnel 1 0 0

Just to be sure, did you load the ip_vs_ftp module?

Best,
-Hannes

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

nicolas.haller at corp

Sep 17, 2009, 12:28 AM

Post #9 of 12 (1610 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, Sep 16, 2009 at 10:24:33PM +0200, Hannes Eder wrote:
> On Wed, Sep 16, 2009 at 15:38, Nicolas Haller
> <nicolas.haller [at] corp> wrote:
> > I try to make a load-balanced ftp and I have a little problem. I can't
> > use FTP in active mode. Someone can help me?
> >
> > The lvsadm conf
> > TCP lo-pp.nerim.net:ftp wlc persistent 120
> > -> harrisburg.nerim.net:ftp Tunnel 1 0 0
> > -> sellafield.nerim.net:ftp Tunnel 1 0 0

> Just to be sure, did you load the ip_vs_ftp module?

I try but no effect. :-/

Thanks,

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

nicolas.haller at corp

Sep 17, 2009, 12:30 AM

Post #10 of 12 (1605 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Wed, Sep 16, 2009 at 08:28:02AM -0700, Joseph Mack NA3T wrote:
> On Wed, 16 Sep 2009, Nicolas Haller wrote:

> > Hi all,
> >
> > I try to make a load-balanced ftp and I have a little problem. I can't
> > use FTP in active mode. Someone can help me?

> there's a lengthy description of ftp in the HOWTO. Does it
> not work anymore?

Hi,

I'm trying the conf with FWM because it can be usefull in the future.
But it's seems ipvs don't care about my netfilter marks.

Thanks for the howto,

Regards,

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

malcolm at loadbalancer

Sep 17, 2009, 2:38 AM

Post #11 of 12 (1606 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

Nicolas,

Try and get a simple service like telnet or HTTP working in the local
subnet with DR mode first, then FWM and DR mode, then FWM with TUN
i.e. progress from simple to complicated.
Then swap to FTP.

I can't see any reason why it wouldn't work, apart from the TUN bit
which I don't have much experience of.

2009/9/17 Nicolas Haller <nicolas.haller [at] corp>:
> On Wed, Sep 16, 2009 at 08:28:02AM -0700, Joseph Mack NA3T wrote:
>> On Wed, 16 Sep 2009, Nicolas Haller wrote:
>
>> > Hi all,
>> >
>> > I try to make a load-balanced ftp and I have a little problem. I can't
>> > use FTP in active mode. Someone can help me?
>
>> there's a lengthy description of ftp in the HOWTO. Does it
>> not work anymore?
>
> Hi,
>
> I'm trying the conf with FWM because it can be usefull in the future.
> But it's seems ipvs don't care about my netfilter marks.
>
> Thanks for the howto,
>
> Regards,
>
> --
> Nicolas Haller
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
> Send requests to lvs-users-request [at] LinuxVirtualServer
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>

--
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

nicolas.haller at corp

Sep 17, 2009, 3:04 AM

Post #12 of 12 (1598 views)
Permalink

Re: [lvs-users] FTP in active mode? [In reply to]

On Thu, Sep 17, 2009 at 10:38:14AM +0100, Malcolm Turnbull wrote:
> Nicolas,

> Try and get a simple service like telnet or HTTP working in the local
> subnet with DR mode first, then FWM and DR mode, then FWM with TUN
> i.e. progress from simple to complicated.
> Then swap to FTP.

> I can't see any reason why it wouldn't work, apart from the TUN bit
> which I don't have much experience of.

Hi,

I redo some test this morning and I have more informations.

This is my configuration:
Iptables:
Chain PREROUTING (policy ACCEPT 297K packets, 47M bytes)
pkts bytes target prot opt in out source destination
6 384 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpt:ftp MARK xset 0x1/0xffffffff
0 0 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpt:ftp-data MARK xset 0x1/0xffffffff
0 0 MARK tcp -- any any anywhere lo-pp.nerim.net tcp dpts:1024:65535 MARK xset 0x1/0xffffffff

Ipvsadm:
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP lo-pp.nerim.net:ftp wlc persistent 120
-> harrisburg.nerim.net:ftp Tunnel 1 0 0
-> sellafield.nerim.net:ftp Tunnel 1 0 0
TCP lo-pp.nerim.net:www wlc persistent 120
-> harrisburg.nerim.net:www Tunnel 1 0 0
-> sellafield.nerim.net:www Tunnel 1 0 0
FWM 1 wlc persistent 120
-> harrisburg.nerim.net:0 Tunnel 1 0 0
-> sellafield.nerim.net:0 Tunnel 1 0 0

And this is some debug trace:

Sep 17 11:45:24 balancoire-1j kernel: [63473.631331] IN=eth0 OUT= MAC=00:30:48:5a:4e:4e:00:1f:9d:9f:89:42:08:00 SRC=62.4.16.150 DST=194.79.128.128 LEN=64 TOS
=0x00 PREC=0x00 TTL=63 ID=25788 DF PROTO=TCP SPT=61341 DPT=21 WINDOW=65535 RES=0x00 SYN URGP=0 MARK=0x1
Sep 17 11:45:24 balancoire-1j kernel: [63473.631564] IPVS: lookup/in TCP 62.4.16.150:61341->194.79.128.128:21 not hit
Sep 17 11:45:24 balancoire-1j kernel: [63473.631568] IPVS: lookup/out TCP 62.4.16.150:61341->194.79.128.128:21 not hit
Sep 17 11:45:24 balancoire-1j kernel: [63473.631572] IPVS: lookup service: fwm 1 TCP 194.79.128.128:21 hit
Sep 17 11:45:24 balancoire-1j kernel: [63473.631576] IPVS: p-schedule: src 62.4.16.150:61341 dest 194.79.128.128:21 mnet 62.4.16.150
Sep 17 11:45:24 balancoire-1j kernel: [63473.631580] IPVS: template lookup/in IP 62.4.16.150:0->0.0.0.1:0 not hit

As you can see, the director receive a FTP packet, IPVS match the FWM service
and then... nothing. I don't know why after detect the packet, IPVS don't
transmit the packet.

--
Nicolas Haller

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads