kastner at devicen
Apr 4, 2005, 1:20 AM
Post #1 of 1
(4514 views)
Permalink
|
|
success story: www.cebit.de
|
|
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello to all,
after 4 years of working with mod_backhand, I thought it was now the
time to
share with you our experiences with this great piece of software.
Should there be someone not knowing anything about CeBIT, here's a
little
blurb from the CeBIT website:
"Once a display category within the HANNOVER FAIR - now the largest
trade show
in the world
With 6,270 exhibitors (52 percent from abroad) and a net display area of
309,000 square meters CeBIT was once again reasserting its singular role
among the world's IT and telecoms trade fairs in 2005. In just one and
a half
decades, CeBIT has grown from its origins as part of the HANNOVER FAIR
to
become the world's leading event for information technology,
telecommunications, software and services."
As you can imagine, the CeBIT web site attracts a lot of visitors,
particulary
in the weeks before and during the event. At peak times, the web site
recieved
several hundred hits a second, serving millions of requests a day.
So performance is crucial. To handle the load, the CeBIT web site is
set up
as a 3-tier cluster:
6 lean apache frontends (2 stand-by, 4 active) are hosting the
front-end IP
addresses (DNS round-robin). In the years before, we had 4 pairs of
machines,
each serving one address, connected via heartbeat. This year we used
wackamole to ensure high-availability of the front-end interfaces.
Compared to heartbeat, wackamole gave us higher availability with less
machines and allowed us to eliminate the timeout before takeover. In
fact, we
had (and still have) 100% availability. One minor glitch: I never
managed to
get the "preferred address" directive to work, the addresses jumped
around
rather randomly at startup but that was not really a problem.
Behind the frontends, the cluster sports a pool of 12 full-blown backend
machines, handling all the php-scripts for page generation, session
handling,
etc. which communicate via soap with a localhost tomcat. Tomcat hosts
web services providing database communication and "business logic".
Data is
delivered by a large Oracle installation and 2 PostgreSQL instances on
dedicated servers. Additionally, there are several machines
providing fuzzy search matches. Communication with these is handled also
by the tomcat java classes.
BTW: Totally separated from the mod_backhand-managed cluster I'm
describing
here, there is one machine handling static content (pictures, pdf files,
etc.) by a tux web server. But back to the cluster:
The load is balanced and distributed by mod_backhand (of course). This
works
perfectly, although not all backend servers are equally powered.
I took a screenshot of the backhand-page to give you an imagination of
our
setup. As you can see, the cluster was idling when I took the
screenshot.
<http://files.messe.de/backhand.jpg>
However, even at highest load, the cluster was happily humming along.
The only
problem we encountered were 2 short hickups when the frontends bailed
out
with lots of "mod_backhand: recursion detected bogus %p" and "failed to
establish umbilical to moderator". This situation required a reload of
the
frontend apaches and was triggered one time by one hanging backend and
a backend restart in the second event.
I know that Theo suggested to switch off connection pooling in this
case,
but this caused funny behaviour in combination with keepalive-on to the
clients. The years before we had keepalive off, but recieved sporadic
reports
of hanging connections from users. With a short keep-alive (5s) we
solved
this problem without exhausting or servers, but we had to switch
connection
pooling on.
Nevertheless, this years CeBIT web site was a great success which would
not
have been possible without your passion and work. So thanks to all and
especially to Theo for providing a great software! Please move on and
make
mod_backhand available for Apache 2. If I can help (unfortunately, I'm
not a
coder), I would be glad to to so!
Best regards,
Tilman
- -- Tilman Kastner abian GmbH
Tel.: (05 11) 9 29 99 66 Deisterstrasse 81
Fax: (05 11) 9 29 99 33 30449 Hannover
PGP key available Germany
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (Darwin)
iD8DBQFCTREDUmnYryA5+hIRArrhAJ9hiaex+/qHlZyBBW9UuBOavpNBbwCfcb9A
AVgATOBJ6PcTgZAOjJAocFM=
=OCSl
-----END PGP SIGNATURE-----
_______________________________________________
backhand-users mailing list
backhand-users [at] lists
http://lists.backhand.org/mailman/listinfo/backhand-users
|
