Mailing List Archive: ModPerl: ModPerl

User abort/stop, modperl 2 and TCP FIN / RST?

Index | Next | Previous | View Threaded

peter at morch

Jun 10, 2012, 11:23 AM

Post #1 of 8 (734 views)
Permalink

User abort/stop, modperl 2 and TCP FIN / RST?

Hi,

I'm trying to find out how to detect user hitting 'stop' aka 'abort'
in modperl 2. I found documentation on how it works in modperl 1 (
http://perl.apache.org/docs/1.0/guide/debug.html#Detecting_Aborted_Connections
- short version: $r->print returns success and $r->connection->aborted
tells whether user hit abort ).

However, I've tested the situation to be quite different in modperl 2:

When the user hits 'stop', the *second* $r->rflush() generates an exception
> Apache2::RequestIO::rflush: (103) Software caused connection
as long as ~ 100ms has passed between the two rflush-es.

Here is my understanding of what happens:

When the user hits 'stop' in the browser, the browser sends a TCP
packet with the FIN flag set. Apache2/modperl doesn't react to that
and $r->connection->aborted still returns false. A subsequent
$r->print and $r->rflush works fine. Apache now sends whatever was
printed to the browser. The browser now sends RST ("Hey, I really want
to kill this connection!") to Apache and after that, $r->print still
succeeds (returns true), but $r->rflush dies (because now the client
has closed the socket hard). "Second" $r->rflush really means "the
first $r->rflush after Apache received the client's 'RST' ", so if I
issue many $r->print("foo"); $r->rflush() in quick succession they all
pass. Around 100ms needs to pass between $r->rflush-es for the second
$r->rflush to fail. If tested this with Firefox and Chromium.

So: After the user hits 'stop', the second $r->rflush (requiring a
delay) generates an exception that can be used to determine that the
user has hit 'stop'.

Have I understood this correctly? Is there any way I can get Apache to
react to the first TCP FIN, and have $r->connection->aborted return
true at that point? Or otherwise detect reliably when the user has hit
'stop' in modperl 2 without having to wait an additional
server-client-server roundtrip time?

Using standard Debian squeeze packages:
> dpkg-query -W | grep apache2
apache2 2.2.16-6+squeeze7
apache2-mpm-worker 2.2.16-6+squeeze7
apache2-utils 2.2.16-6+squeeze7
apache2.2-bin 2.2.16-6+squeeze7
apache2.2-common 2.2.16-6+squeeze7
libapache2-mod-perl2 2.0.4-7
libapache2-reload-perl 0.10-2

Apache config:
<Directory /var/www/foo>
Options +ExecCGI
AddDefaultCharset On
<Files flush.cgi>
SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
</Files>
</Directory>

flush.cgi:
#!/usr/bin/perl -w
use strict;

use Apache2::ServerUtil;
use Apache2::RequestUtil;
use Apache2::RequestRec;
use Apache2::Connection;

use Time::HiRes qw(sleep);

sub doCGI {
my $r = Apache2::RequestUtil->request;
my $c = $r->connection();
$r->print("Content-type: text/html\n\n");
$r->print("Starting up $$<br>\n");
LOOP_I: foreach my $i (0..3) {

# User hits STOP in browser here for some $i

foreach my $j (0..1) {
my $printRetval = $r->print("$i / $j<br>\n");
$r->log_error(sprintf "%d / %d pre-flush : %s print %s",
$i, $j,
($c->aborted ? "aborted" : "not aborted"),
( $printRetval ? 'succeeded' : 'failed')
);

# This always dies when $j == 1. Why 1 and not 0?
# This is the first place we notice when the user has hit STOP in
# the browser.
eval {
$r->rflush;
};

my $err = $@;
$r->log_error(sprintf "%d / %d post-flush : %s flush %s",
$i, $j,
( $c->aborted ? "aborted" : "not aborted" ),
( $err ? 'failed' : 'succeeded')
);
if ($err) {
$r->log_error("exiting loops");
last LOOP_I;
}

# Why sleep 0.1? 0.01 is mostly enough, but 0.001 is always too
# little. What determines this number?
sleep 0.1;
}
sleep 5;
}
};
doCGI();

A wireshark trace of the traffic between browser and server can be
found here: http://ge.tt/6XGcPvI/v/0?c

Thanks for reading this far.

Peter
--
Peter Valdemar Mørch
http://www.morch.com

Steve.Hay at verosoftware

Jun 11, 2012, 12:16 AM

Post #2 of 8 (686 views)
Permalink

RE: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

Peter Valdemar Mørch wrote on 2012-06-10:
> Hi,
>
> I'm trying to find out how to detect user hitting 'stop' aka 'abort' in
> modperl 2. I found documentation on how it works in modperl 1 (
> http://perl.apache.org/docs/1.0/guide/debug.html#Detecting_Aborted_Co
> nnections - short version: $r->print returns success and
> $r->connection->aborted tells whether user hit abort ).
>
> However, I've tested the situation to be quite different in modperl 2:
>
> When the user hits 'stop', the *second* $r->rflush() generates an
> exception
>> Apache2::RequestIO::rflush: (103) Software caused connection
> as long as ~ 100ms has passed between the two rflush-es.
>
[...]
> Using standard Debian squeeze packages:
>> dpkg-query -W | grep apache2
> apache2 2.2.16-6+squeeze7 apache2-mpm-worker 2.2.16-6+squeeze7
> apache2-utils 2.2.16-6+squeeze7 apache2.2-bin 2.2.16-6+squeeze7
> apache2.2-common 2.2.16-6+squeeze7 libapache2-mod-perl2 2.0.4-7
> libapache2-reload-perl 0.10-2

I notice that you're using mod_perl-2.0.4. The behaviour of rflush() was changed in mod_perl-2.0.6 so that it no longer throws an exception on reset/aborted connections, so it would certainly be worth upgrading to 2.0.6 or 2.0.7 to see if that helps you.

peter at morch

Jun 11, 2012, 2:53 AM

Post #3 of 8 (680 views)
Permalink

Re: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

Thanks for your reply!

On Mon, Jun 11, 2012 at 9:16 AM, Steve Hay <Steve.Hay [at] verosoftware> wrote:
> it would certainly be worth upgrading to 2.0.6 or 2.0.7 to see if that helps you.

As pr. your suggestion I've installed a fresh debian testing/wheezy
system that runs these versions:

# dpkg-query -W | grep apache2
apache2 2.2.22-6
apache2-mpm-worker 2.2.22-6
apache2-utils 2.2.22-6
apache2.2-bin 2.2.22-6
apache2.2-common 2.2.22-6
libapache2-mod-perl2 2.0.6-2
libapache2-reload-perl 0.11-2

And here the only difference I can see is that $r->rflush doesn't die
(like you write). Instead of testing for whether $r->rflush() dies one
can test for $r->connection->abort().

All the rest is the same. It it only possible to detect 'stop' after
the second print/rflush cycle as described in my original post. I'm
still hoping there is some way to detect 'stop' earlier...

Peter
--
Peter Valdemar Mørch
http://www.morch.com

aw at ice-sa

Jun 11, 2012, 3:29 AM

Post #4 of 8 (682 views)
Permalink

Re: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

Peter Valdemar Mørch wrote:
> Hi,
>
> I'm trying to find out how to detect user hitting 'stop' aka 'abort'
> in modperl 2. I found documentation on how it works in modperl 1 (
> http://perl.apache.org/docs/1.0/guide/debug.html#Detecting_Aborted_Connections
> - short version: $r->print returns success and $r->connection->aborted
> tells whether user hit abort ).
>
> However, I've tested the situation to be quite different in modperl 2:
>
> When the user hits 'stop', the *second* $r->rflush() generates an exception
>> Apache2::RequestIO::rflush: (103) Software caused connection
> as long as ~ 100ms has passed between the two rflush-es.
>
> Here is my understanding of what happens:
>
> When the user hits 'stop' in the browser, the browser sends a TCP
> packet with the FIN flag set. Apache2/modperl doesn't react to that
> and $r->connection->aborted still returns false. A subsequent
> $r->print and $r->rflush works fine. Apache now sends whatever was
> printed to the browser. The browser now sends RST ("Hey, I really want
> to kill this connection!") to Apache and after that, $r->print still
> succeeds (returns true), but $r->rflush dies (because now the client
> has closed the socket hard). "Second" $r->rflush really means "the
> first $r->rflush after Apache received the client's 'RST' ", so if I
> issue many $r->print("foo"); $r->rflush() in quick succession they all
> pass. Around 100ms needs to pass between $r->rflush-es for the second
> $r->rflush to fail. If tested this with Firefox and Chromium.
>
> So: After the user hits 'stop', the second $r->rflush (requiring a
> delay) generates an exception that can be used to determine that the
> user has hit 'stop'.
>
> Have I understood this correctly? Is there any way I can get Apache to
> react to the first TCP FIN, and have $r->connection->aborted return
> true at that point? Or otherwise detect reliably when the user has hit
> 'stop' in modperl 2 without having to wait an additional
> server-client-server roundtrip time?
>

Hi.
Just my two cent, maybe just to dampen your expectations a little bit.
This topic is probably as old as the WWW itself, and there just is no magic bullet here.
Think of the following :
The whole FIN, ACK, RST etc.. exchange concerns /one/ TCP connection, between your browser
and whatever it is talking to directly. For example, a HTTP proxy server, or a firewall.
And then, this proxy/firewall has /another/ TCP connection with the next node in the chain
(which may be your webserver or still another intermediary, like a load-balancer e.g.).
And in-between, there are buffers and a lot of piping, and delays.
So, expecting your mod_perl script to be able to detect instantly when the user at the
browser half a planet away presses the "stop" button, is /never/ going to be guaranteed.
I'm not saying that you cannot try to catch such a thing as early as possible.
But if your ultimate aim is to find a way where instant detection is guaranteed, then give
it up. It is just not possible, given how the Internet, and TCP/IP and HTTP work.

randolf at modperl

Jun 11, 2012, 9:39 AM

Post #5 of 8 (682 views)
Permalink

Re: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

> Peter Valdemar Mørch wrote:
> > Hi,
> >
> > I'm trying to find out how to detect user hitting 'stop' aka 'abort'
> > in modperl 2. I found documentation on how it works in modperl 1 (
> > http://perl.apache.org/docs/1.0/guide/debug.html#Detecting_Aborted_Connections
> > - short version: $r->print returns success and $r->connection->aborted
> > tells whether user hit abort ).
> >
> > However, I've tested the situation to be quite different in modperl 2:
> >
> > When the user hits 'stop', the *second* $r->rflush() generates an exception
> >> Apache2::RequestIO::rflush: (103) Software caused connection
> > as long as ~ 100ms has passed between the two rflush-es.
> >
> > Here is my understanding of what happens:
> >
> > When the user hits 'stop' in the browser, the browser sends a TCP
> > packet with the FIN flag set. Apache2/modperl doesn't react to that
> > and $r->connection->aborted still returns false. A subsequent
> > $r->print and $r->rflush works fine. Apache now sends whatever was
> > printed to the browser. The browser now sends RST ("Hey, I really want
> > to kill this connection!") to Apache and after that, $r->print still
> > succeeds (returns true), but $r->rflush dies (because now the client
> > has closed the socket hard). "Second" $r->rflush really means "the
> > first $r->rflush after Apache received the client's 'RST' ", so if I
> > issue many $r->print("foo"); $r->rflush() in quick succession they all
> > pass. Around 100ms needs to pass between $r->rflush-es for the second
> > $r->rflush to fail. If tested this with Firefox and Chromium.
> >
> > So: After the user hits 'stop', the second $r->rflush (requiring a
> > delay) generates an exception that can be used to determine that the
> > user has hit 'stop'.
> >
> > Have I understood this correctly? Is there any way I can get Apache to
> > react to the first TCP FIN, and have $r->connection->aborted return
> > true at that point? Or otherwise detect reliably when the user has hit
> > 'stop' in modperl 2 without having to wait an additional
> > server-client-server roundtrip time?
>
> Hi.
> Just my two cent, maybe just to dampen your expectations a little bit.
> This topic is probably as old as the WWW itself, and there just is no magic bullet here.
> Think of the following :
> The whole FIN, ACK, RST etc.. exchange concerns /one/ TCP connection, between your browser
> and whatever it is talking to directly. For example, a HTTP proxy server, or a firewall.
> And then, this proxy/firewall has /another/ TCP connection with the next node in the chain
> (which may be your webserver or still another intermediary, like a load-balancer e.g.).
> And in-between, there are buffers and a lot of piping, and delays.
> So, expecting your mod_perl script to be able to detect instantly when the user at the
> browser half a planet away presses the "stop" button, is /never/ going to be guaranteed.
> I'm not saying that you cannot try to catch such a thing as early as possible.
> But if your ultimate aim is to find a way where instant detection is guaranteed, then give
> it up. It is just not possible, given how the Internet, and TCP/IP and HTTP work.

In addition to that, there is variation among web browser and smart
phone vendors in the way they tear down sockets. Some may close the
socket immediately (a good practice in my opinion), while others may
just let the resource get garbage collected or even have an unknown
bug that results in a temporary or permanent resource leak on the
client-side.

The same problems can occur with proxy servers, and for localized
end-user products that intercept all traffic (e.g., for the purpose
of scanning for viruses or other suspicious activity) there can be
problems as well. I've seen big problems with long timeouts with
some of these client-side products where the server has lingering
connections after the QUIT phase with POP3 connections, and removing
the firewalling software resolved the problem (but this isn't a
solution that helps you because many users insist on running these
programs, are working at companies that require it, etc.).

Making sure you have adequate resources to handle the load on the
server-side is likely less time-consuming than figuring out how every
web browser does (and will) handle the user pressing the "Stop"
button behind-the-scenes, although the KeepAlive and KeepAliveTImeout
directives may be helpful to you:

http://httpd.apache.org/docs/current/mod/core.html#keepalive
http://httpd.apache.org/docs/current/mod/core.html#keepalivetimeout

Note: The KeepAliveTimeout directive is only available starting
with Apache HTTPd v2.3.2.

Have you've already experimented with the Timeout directive? If
not, then I suggest you start with this directive before using the
KeepAlive and KeepAliveTimeout directives:

http://httpd.apache.org/docs/current/mod/core.html#timeout

Randolf Richardson - randolf [at] inter-corporate
Inter-Corporate Computer & Network Services, Inc.
Beautiful British Columbia, Canada
http://www.inter-corporate.com/

trawick at gmail

Jun 11, 2012, 10:50 AM

Post #6 of 8 (684 views)
Permalink

Re: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

On Mon, Jun 11, 2012 at 12:39 PM, Randolf Richardson <randolf [at] modperl> wrote:
> http://httpd.apache.org/docs/current/mod/core.html#keepalivetimeout
>
> Note: The KeepAliveTimeout directive is only available starting
> with Apache HTTPd v2.3.2.

It is ancient. 2.3.2 brought the ability to specify the timeout in
milliseconds.

peter at morch

Jun 11, 2012, 3:33 PM

Post #7 of 8 (685 views)
Permalink

Re: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

On Mon, Jun 11, 2012 at 6:39 PM, Randolf Richardson <randolf [at] modperl> wrote:
> Have you've already experimented with the Timeout directive? If
> not, then I suggest you start with this directive before using the
> KeepAlive and KeepAliveTimeout directives:
>
> http://httpd.apache.org/docs/current/mod/core.html#timeout

It is my impression that mod_perl doesn't honor the Timeout directive.
This comes from both reading and testing. We've had to resort to
http://perl.apache.org/docs/1.0/guide/debug.html#Handling_Server_Timeout_Cases_and_Working_with__SIG_ALRM_
. Have I missed something? (Using TimeOut would be so much nicer!)

Peter
--
Peter Valdemar Mørch
http://www.morch.com

peter at morch

Jun 11, 2012, 4:13 PM

Post #8 of 8 (678 views)
Permalink

Re: User abort/stop, modperl 2 and TCP FIN / RST? [In reply to]

On Mon, Jun 11, 2012 at 12:29 PM, André Warnier <aw [at] ice-sa> wrote:
> Just my two cent, maybe just to dampen your expectations a little bit.
> This topic is probably as old as the WWW itself, and there just is no magic
> bullet here.
...
> But if your ultimate aim is to find a way where instant detection is
> guaranteed, then give it up. It is just not possible, given how the
> Internet, and TCP/IP and HTTP work.

I was certain that normal cgis under Apache2 reacted immediately to
aborts. I was wrong ( As I discovered in my quest to devise a test
case proving you wrong... ;-) )

Ok, we're doing the best we can then. Fine. At least the next guy that
comes along will have this thread to read.

So pseudocode for the magic incantation then seems to be:

while (! $done) {
heavyCalculationBite();
$r->print($msg);
# Eval needed to support < 2.0.6 that dies if $r->connection->aborted
eval {
$r->rflush();
};
if ($r->connection->aborted) {
# Handle user abort
last;
}
}

Thank you so much for your enlightening replies!

Peter
--
Peter Valdemar Mørch
http://www.morch.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads