andrew at cleverdomain
Sep 28, 2009, 9:07 PM
Post #10 of 19
(2201 views)
Permalink
|
On Monday 28 September 2009 12:56:36 am Fayland Lam wrote:
> I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8
Benchmark, as requested. View this message at http://p3m.org/pfn/3499 if your
mailer is too high-tech for fixed-width text.
The Setup:
Linux 2.6 OpenVZ, on Quad 2.2GHz Opteron.
Perl 5.10.0 + debian patches.
FastCGI via Apache 2.2, mpm_event, mod_fastcgi.
The two test instances were running on the same machine, with the same perl
and the same checkout of the app, but different local::lib directories.
FastCGI was set to 10 processes. The page I was hitting was from a real
checkout of a real production app, however it was the front page of the
site, which is fairly light on dynamic content. I figured this was
appropriate since it would better show any differences in Catalyst rather
than spending a lot of time in the backend. The code still hits several
models, 3 actions, and a view, but perhaps it was a little too fast since,
as you'll see below, my throughput was ultimately limited by the number of
running processes. Each instance was given a "warmup" run (the results of
which were discarded) before the following tests were run. My tool collects
statistics on the return status, but for all tests the returns were all 200
(success) so I've left out that row.
|| 20 requests/second (20 threads) for 60s
| Metric || Catalyst 5.7010 || Catalyst 5.8011
|===================||==========================||==========================
| Hits || 1200 || 1200
| Throughput || 20.00 req/s || 20.00 req/s
| Latency (mean) || 0.072s || 0.074s
| Latency (SD) || 0.013s || 0.017s
| Latency (Q1-Q3) || 0.064 - 0.078s || 0.066 - 0.080s
|| 40 requests/second (40 threads) for 60s
| Metric || Catalyst 5.7010 || Catalyst 5.8011
|===================||==========================||==========================
| Hits || 2400 || 2400
| Throughput || 40.00 req/s || 40.00 req/s
| Latency (mean) || 0.083s || 0.088s
| Latency (SD) || 0.020s || 0.024s
| Latency (Q1-Q3) || 0.069 - 0.095s || 0.072 - 0.100s
|| 80 requests/second (80 threads) for 60s
| Metric || Catalyst 5.7010 || Catalyst 5.8011
|===================||==========================||==========================
| Hits || 4675 || 4637
| Throughput || 77.92 req/s || 77.28 req/s
| Latency (mean) || 0.688s || 0.708s
| Latency (SD) || 0.178s || 0.187s
| Latency (Q1-Q3) || 0.617 - 0.800s || 0.726 - 0.811s
The difference between 5.7 and 5.8 in these results is consistently in favor
of 5.7, but by a margin of between 0% and 5% which is not a whole lot in my
book. By my unscientific measure (i.e. looking at "top") of memory usage,
5.7 used 138MB of RAM (for fcgi-pm + 10x fcgi children) whereas 5.8 used
184MB, so that's a 33% expansion, which is a more significant issue. I have
a feeling that most of that is shared, and so the difference wouldn't
increase much with an increase in the number of processes, but I haven't
investigated that yet.
Questions?
Andrew "hobbs" Rodland
_______________________________________________
List: Catalyst [at] lists
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst [at] lists/
Dev site: http://dev.catalyst.perl.org/
|
message-rfc822.eml
