Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Interchange: cvs

[interchange] Increase size of socket read for much better performance

 

 

Interchange cvs RSS feed   Index | Next | Previous | View Threaded


interchange-cvs at icdevgroup

Jan 25, 2012, 2:29 PM

Post #1 of 1 (258 views)
Permalink
[interchange] Increase size of socket read for much better performance

commit f0479d191e4dffcfaec781d0e953f66d7c6f96c9
Author: Jon Jensen <jon [at] endpoint>
Date: Wed Jan 25 15:20:36 2012 -0700

Increase size of socket read for much better performance

ddavenport found a serious performance regression in Vend::Server, such
that each socket read was only for 1 byte, making large uploads
painfully slow:

https://github.com/interchange/interchange/pull/2

I found the regression was introduced in this commit:

commit 2a69743c5023ebb1198c0f2e7f7dcdce4935b5eb
Author: Kevin Walsh <kevin [at] cursor>
Date: Mon Feb 4 21:42:18 2008 +0000

* Changes in the _read() subroutine:

-- select() returns -1 upon error, whereas sysread() returns undef,
so we need to allow for both.

-- Catch EAGAIN as well as EINTR as soft errors to retry on.

-- Read the entire available amount of data in one hit instead of
forcing the data to be read in 512-byte chunks.

Kevin clearly intended to read as much data as was waiting, but select()
returns the number of waiting filehandles, not # of waiting bytes.

Rather than reverting to the original 512 bytes per read, or
ddavenport's suggested 1024, I used 1 MiB which is still arbitrary but
large enough to get most normal http requests in few passes, and handles
large uploads efficiently.

Thanks, ddavenport!

Original patch:

commit 2a6308dac12c3d8fafcfad02cd41a32f6adfefe0
Author: ddavenport <dances_with_peons [at] live>
Date: Fri Jul 15 20:50:54 2011 -0700

Fixed Vend::Server so it'll read more than a single byte at a time
(which was causing large file uploads to fail).

diff --git a/lib/Vend/Server.pm b/lib/Vend/Server.pm
index 5baf109..4a82ce8 100644
--- a/lib/Vend/Server.pm
+++ b/lib/Vend/Server.pm
@@ -795,7 +795,7 @@ sub _read {

do {
if (($r = select($rin, undef, undef, $Global::SocketReadTimeout || 1)) > 0) {
- $r = sysread($fh, $$in, $r, length($$in));
+ $r = sysread($fh, $$in, 1024, length($$in));
}
} while ((!defined($r) || $r == -1) && ($!{eintr} || $!{eagain}));

lib/Vend/Server.pm | 8 +++++---
1 files changed, 5 insertions(+), 3 deletions(-)
---
diff --git a/lib/Vend/Server.pm b/lib/Vend/Server.pm
index 884e724..99132dc 100644
--- a/lib/Vend/Server.pm
+++ b/lib/Vend/Server.pm
@@ -794,9 +794,11 @@ sub _read {
vec($rin,fileno($fh),1) = 1;

do {
- if (($r = select($rin, undef, undef, $Global::SocketReadTimeout || 1)) > 0) {
- $r = sysread($fh, $$in, $r, length($$in));
- }
+ if (($r = select($rin, undef, undef, $Global::SocketReadTimeout || 1)) > 0) {
+ # read up to an arbitrary 1 MiB at a time for efficiency
+ # (though the operating system may provide far less than that at a time anyway)
+ $r = sysread($fh, $$in, 1_048_576, length($$in));
+ }
} while ((!defined($r) || $r == -1) && ($!{eintr} || $!{eagain}));

die "read: $!" unless defined $r;

_______________________________________________
interchange-cvs mailing list
interchange-cvs [at] icdevgroup
http://www.icdevgroup.org/mailman/listinfo/interchange-cvs

Interchange cvs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.