Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux Virtual Server: Users

Source IP address

 

 

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded


Nathan.Polonski at newsedge

Nov 28, 2000, 9:39 AM

Post #1 of 8 (573 views)
Permalink
Source IP address

I'm currently using a Piranha based LVS system. NAT configuration, kernel
2.2.17 with patches. VS patch 1.0.0.

The main use of the system is ftp. The system is to be behind a firewall and
I have run into an interesting problem.
In my testing I have found that the source IP address of some of the "load
balanced" data does not come from the VIP, but from the IP address of one of
the directors.

If I open up an FTP connection to my cluster, all of the packets are sent to
and come from the VIP. Data looks good. However, when I try to run an "ls"
or "dir" command against the FTP server, I get a "Cannot build Data
Connection" error.
My packet sniffing has shown me that all of the data going to and from the
cluster is addressed to the VIP.
This holds true, up until the directory listing request.
When I run either command, packets come from the IP address of the active
LVS director.

Is this supposed to happen? Does anyone know why it happens.
I'm sure there is a plausible explanation.

--Nathan


horms at vergenet

Nov 28, 2000, 9:52 AM

Post #2 of 8 (566 views)
Permalink
Re: Source IP address [In reply to]

On Tue, Nov 28, 2000 at 11:39:47AM -0500, Nathan Polonski wrote:
> I'm currently using a Piranha based LVS system. NAT configuration, kernel
> 2.2.17 with patches. VS patch 1.0.0.
>
> The main use of the system is ftp. The system is to be behind a firewall and
> I have run into an interesting problem.
> In my testing I have found that the source IP address of some of the "load
> balanced" data does not come from the VIP, but from the IP address of one of
> the directors.
>
> If I open up an FTP connection to my cluster, all of the packets are sent to
> and come from the VIP. Data looks good. However, when I try to run an "ls"
> or "dir" command against the FTP server, I get a "Cannot build Data
> Connection" error.
> My packet sniffing has shown me that all of the data going to and from the
> cluster is addressed to the VIP.
> This holds true, up until the directory listing request.
> When I run either command, packets come from the IP address of the active
> LVS director.
>
> Is this supposed to happen? Does anyone know why it happens.
> I'm sure there is a plausible explanation.

The problem is that when you do an ls your client tries to open
another connection to the ftp server, the VIP. The Linux Director
is allocating that connection to a different real server to the
original control connection, that real server isn't listening for
the data conenction. boom.

From the near to latest revision of the ipvsadm man page:

Note: If a virtual service is to handle FTP connec-
tions then persistence must be set for the virtual
service if Direct Routing or NAT is used as the
forwarding mechanism. If masquerading is used in
conjunction with an FTP service than persistence is
not necessary, but the ip_masq_ftp kernel module
must be used. This module may be manually inserted
into the kernel using insmod(8).



Of course reading that I notice a bug, "NAT" should read "Tunnelling".

Wensong can you update the tree.

Thanks


--
Horms


ja at ssi

Nov 28, 2000, 2:29 PM

Post #3 of 8 (568 views)
Permalink
Re: Source IP address [In reply to]

Hello,

On Tue, 28 Nov 2000, Nathan Polonski wrote:

> I'm currently using a Piranha based LVS system. NAT configuration, kernel
> 2.2.17 with patches. VS patch 1.0.0.
>
> The main use of the system is ftp. The system is to be behind a firewall and
> I have run into an interesting problem.
> In my testing I have found that the source IP address of some of the "load
> balanced" data does not come from the VIP, but from the IP address of one of
> the directors.
>
> If I open up an FTP connection to my cluster, all of the packets are sent to
> and come from the VIP. Data looks good. However, when I try to run an "ls"
> or "dir" command against the FTP server, I get a "Cannot build Data
> Connection" error.
> My packet sniffing has shown me that all of the data going to and from the
> cluster is addressed to the VIP.

With or without ip_masq_ftp? Show us a tcpdump output and
ftp debug output:

tcpdump -n host ftpvirtserver

# ftp
...

Show us the ipvsadm commands you use.

> This holds true, up until the directory listing request.
> When I run either command, packets come from the IP address of the active
> LVS director.
>
> Is this supposed to happen? Does anyone know why it happens.
> I'm sure there is a plausible explanation.
>
> --Nathan


Regards

--
Julian Anastasov <ja [at] ssi>


lmb at suse

Nov 29, 2000, 7:32 AM

Post #4 of 8 (567 views)
Permalink
Re: RE: Source IP address [In reply to]

On 2000-11-29T09:33:22,
Nathan Polonski <Nathan.Polonski [at] newsedge> said:

> Horms,
> I ran the Bastille script against the server, and during that script I
> enabled the ip_masq_ftp module.
> However, I am not quite sure how to test to make sure that the module is
> being loaded properly.
>
> If I run insmod -v -p ip_masq_ftp.o in my /lib/modules/2.2.17 directory I
> get the following ouput:

You should be using "modprobe ip_masq_ftp". This will also load all the
modules ip_masq_ftp depends on.

Sincerely,
Lars Marowsky-Brée <lmb [at] suse>
Development HA

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl


Nathan.Polonski at newsedge

Nov 29, 2000, 7:33 AM

Post #5 of 8 (567 views)
Permalink
RE: Source IP address [In reply to]

Horms,
I ran the Bastille script against the server, and during that script I
enabled the ip_masq_ftp module.
However, I am not quite sure how to test to make sure that the module is
being loaded properly.

If I run insmod -v -p ip_masq_ftp.o in my /lib/modules/2.2.17 directory I
get the following ouput:
Using ip_masq_ftp.o
ip_masq_ftp.o: unresolved symbol ip_masq_skb_replace_Rsmp_672fb649
ip_masq_ftp.o: unresolved symbol ip_masq_control_add_Rsmp_78be8c78
ip_masq_ftp.o: unresolved symbol ip_masq_put_Rsmp_5e752b0d
ip_masq_ftp.o: unresolved symbol ip_masq_new_Rsmp_1cc34fd1
ip_masq_ftp.o: unresolved symbol ip_masq_in_get_Rsmp_3fbd43b0
ip_masq_ftp.o: unresolved symbol unregister_ip_masq_app_Rsmp_bbc84e34
ip_masq_ftp.o: unresolved symbol ip_masq_listen_Rsmp_8e292da2
ip_masq_ftp.o: unresolved symbol ip_masq_out_get_Rsmp_27b7c4d9
ip_masq_ftp.o: unresolved symbol register_ip_masq_app_Rsmp_938aa0b0

does this indicate some sort of version incompatibility?
I recompiled the modules, but I still get this. I'm not sure what it means.
How can I test to see if the module is loading properly?



-----Original Message-----
From: Horms [mailto:horms [at] vergenet]
Sent: Tuesday, November 28, 2000 11:53 AM
To: lvs-users [at] LinuxVirtualServer
Subject: Re: Source IP address


On Tue, Nov 28, 2000 at 11:39:47AM -0500, Nathan Polonski wrote:
> I'm currently using a Piranha based LVS system. NAT configuration, kernel
> 2.2.17 with patches. VS patch 1.0.0.
>
> The main use of the system is ftp. The system is to be behind a firewall
and
> I have run into an interesting problem.
> In my testing I have found that the source IP address of some of the "load
> balanced" data does not come from the VIP, but from the IP address of one
of
> the directors.
>
> If I open up an FTP connection to my cluster, all of the packets are sent
to
> and come from the VIP. Data looks good. However, when I try to run an "ls"
> or "dir" command against the FTP server, I get a "Cannot build Data
> Connection" error.
> My packet sniffing has shown me that all of the data going to and from the
> cluster is addressed to the VIP.
> This holds true, up until the directory listing request.
> When I run either command, packets come from the IP address of the active
> LVS director.
>
> Is this supposed to happen? Does anyone know why it happens.
> I'm sure there is a plausible explanation.

The problem is that when you do an ls your client tries to open
another connection to the ftp server, the VIP. The Linux Director
is allocating that connection to a different real server to the
original control connection, that real server isn't listening for
the data conenction. boom.

From the near to latest revision of the ipvsadm man page:

Note: If a virtual service is to handle FTP connec-
tions then persistence must be set for the virtual
service if Direct Routing or NAT is used as the
forwarding mechanism. If masquerading is used in
conjunction with an FTP service than persistence is
not necessary, but the ip_masq_ftp kernel module
must be used. This module may be manually inserted
into the kernel using insmod(8).



Of course reading that I notice a bug, "NAT" should read "Tunnelling".

Wensong can you update the tree.

Thanks


--
Horms

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users


lmb at suse

Nov 29, 2000, 7:44 AM

Post #6 of 8 (570 views)
Permalink
Re: RE: RE: Source IP address [In reply to]

On 2000-11-29T09:46:33,
Nathan Polonski <Nathan.Polonski [at] newsedge> said:

> Lars,
> Thank you for the suggestion. I tried it and looked at the man page again.
> However, I still get a similar response.
>
> /lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
> ip_masq_skb_replace_Rsmp_672fb649

Is it possible that you compiled your modules with SMP support but your
running kernel doesn't have SMP ?

Your module / kernel setup is seriously screwed. Please recompile your entire
kernel as in

make menuconfig && make clean && make dep && make modules && \
make modules_install && make bzImage

and install the resulting bzImage as your new kernel.

Sincerely,
Lars Marowsky-Brée <lmb [at] suse>
Development HA

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl


Nathan.Polonski at newsedge

Nov 29, 2000, 7:46 AM

Post #7 of 8 (564 views)
Permalink
RE: RE: Source IP address [In reply to]

Lars,
Thank you for the suggestion. I tried it and looked at the man page again.
However, I still get a similar response.

/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_skb_replace_Rsmp_672fb649
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_control_add_Rsmp_78be8c78
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_put_Rsmp_5e752b0d
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_new_Rsmp_1cc34fd1
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_in_get_Rsmp_3fbd43b0
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
unregister_ip_masq_app_Rsmp_bbc84e34
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_listen_Rsmp_8e292da2
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_out_get_Rsmp_27b7c4d9
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
register_ip_masq_app_Rsmp_938aa0b0
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o: insmod
/lib/modules/2.2.17/ipv4/ip_masq_ftp.o failed
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_skb_replace_Rsmp_672fb649
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_control_add_Rsmp_78be8c78
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_put_Rsmp_5e752b0d
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_new_Rsmp_1cc34fd1
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_in_get_Rsmp_3fbd43b0
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
unregister_ip_masq_app_Rsmp_bbc84e34
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_listen_Rsmp_8e292da2
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
ip_masq_out_get_Rsmp_27b7c4d9
/lib/modules/default/ipv4/ip_masq_ftp.o: unresolved symbol
register_ip_masq_app_Rsmp_938aa0b0
/lib/modules/default/ipv4/ip_masq_ftp.o: insmod
/lib/modules/default/ipv4/ip_masq_ftp.o failed
/lib/modules/default/ipv4/ip_masq_ftp.o: insmod ip_masq_ftp failed

When looking through the man pages I found reference to a "default"
directory in the /lib/modules path. I didn't have one, so I made a link to
the directory of the current kernel version.
Whenever I run insmod to try and insert the module to the kernel, I get the
ip_masq_ftp failed error.

I wish it would be more specific when it says "unresolved symbol". Running a
-v doesn't produce any more info.

-----Original Message-----
From: Lars Marowsky-Bree [mailto:lmb [at] suse]
Sent: Wednesday, November 29, 2000 9:32 AM
To: lvs-users [at] LinuxVirtualServer
Subject: Re: RE: Source IP address


On 2000-11-29T09:33:22,
Nathan Polonski <Nathan.Polonski [at] newsedge> said:

> Horms,
> I ran the Bastille script against the server, and during that script I
> enabled the ip_masq_ftp module.
> However, I am not quite sure how to test to make sure that the module is
> being loaded properly.
>
> If I run insmod -v -p ip_masq_ftp.o in my /lib/modules/2.2.17 directory I
> get the following ouput:

You should be using "modprobe ip_masq_ftp". This will also load all the
modules ip_masq_ftp depends on.

Sincerely,
Lars Marowsky-Brée <lmb [at] suse>
Development HA

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl


_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users


Nathan.Polonski at newsedge

Nov 30, 2000, 10:23 AM

Post #8 of 8 (567 views)
Permalink
RE: RE: RE: Source IP address [In reply to]

That did it. I must not have copied over the new modules when I installed
the kernel.
I patched with the latest ipvs code and things look good.
However, I did notice that the ip_masq_ftp module is not automatically
loaded by anything.
Where do people usually start this from?
My firewall script is supposed to load it, but it doesn't appear to be
working.
I put it in rc.local and it works, but I don't know if there are any risks
involved with putting it there.

Thanks for all the help!
Nathan


-----Original Message-----
From: Lars Marowsky-Bree [mailto:lmb [at] suse]
Sent: Wednesday, November 29, 2000 9:44 AM
To: lvs-users [at] LinuxVirtualServer
Subject: Re: RE: RE: Source IP address


On 2000-11-29T09:46:33,
Nathan Polonski <Nathan.Polonski [at] newsedge> said:

> Lars,
> Thank you for the suggestion. I tried it and looked at the man page again.
> However, I still get a similar response.
>
> /lib/modules/2.2.17/ipv4/ip_masq_ftp.o: unresolved symbol
> ip_masq_skb_replace_Rsmp_672fb649

Is it possible that you compiled your modules with SMP support but your
running kernel doesn't have SMP ?

Your module / kernel setup is seriously screwed. Please recompile your
entire
kernel as in

make menuconfig && make clean && make dep && make modules && \
make modules_install && make bzImage

and install the resulting bzImage as your new kernel.

Sincerely,
Lars Marowsky-Brée <lmb [at] suse>
Development HA

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl


_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users [at] LinuxVirtualServer
Send requests to lvs-users-request [at] LinuxVirtualServer
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

Linux Virtual Server users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.