Gossamer Forum
Home : Products : Gossamer Forum : Discussion :

Forum hangs with high server load

Quote Reply
Forum hangs with high server load
Hi

We have an installation at http://www.sissify.com/juice/discussion/

I was getting these types of server loads:

4:09pm up 75 days, 9:54, 2 users, load average: 9.56, 4.60, 2.123
9 processes: 2 sleeping, 7 running, 0 zombie, 0 stopped3
CPU states: 134.7% user, 11.1% system, 0.0% nice, 0.0% idle3
Mem: 1036080K av, 626384K used, 409696K free, 258620K shrd, 77340K buff3
Swap: 130748K av, 348K used, 130400K free 43896K cached3
3
PID USER PRI NI SIZE RES SHRD STAT %CPU %MEM TIME COMMAND3
28856 enigma 17 0 7232 6604 912 R 22.6 0.6 0:18 perl gforum.cgi3
28822 enigma 17 0 7232 6604 912 R 21.9 0.6 0:20 perl gforum.cgi3
28863 enigma 17 0 7232 6604 912 R 20.9 0.6 0:17 perl gforum.cgi3
28870 enigma 17 0 7232 6604 912 R 20.3 0.6 0:17 perl gforum.cgi3
28762 enigma 17 0 7232 6604 912 R 19.3 0.6 0:24 perl gforum.cgi3
28826 enigma 17 0 7232 6604 912 R 19.3 0.6 0:20 perl gforum.cgi3
28833 enigma 17 0 7232 6604 912 S 19.3 0.6 0:19 perl gforum.cgi3
29104 enigma 2 0 884 428 304 R 1.9 0.0 0:00 top3505

I often get error reports from users saying the page times out when waiting for a load occasionally. I caught it today with this "Top".

I logged into the system fine with a cookie saved/logged in. But if I am not logged in , the server load goes bezerk and this is the error placed to the apache error logs (Thousands of guest_id_fk's until the server kills it with procmurder):
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126532' at /home/enigma/public_html/juice/discussion/admin/GT/SQL/Driver/MYSQL.pm line 35.
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126531' at /home/enigma/public_html/juice/discussion/admin/GT/SQL/Driver/MYSQL.pm line 35.
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126530' at /home/enigma/public_html/juice/discussion/admin/GT/SQL/Driver/MYSQL.pm line 35.
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126529' at /home/enigma/public_html/juice/discussion/admin/GT/SQL/Driver/MYSQL.pm line 35.
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126528' at /home/enigma/public_html/juice/discussion/admin/GT/SQL/Driver/MYSQL.pm line 35.
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126527' at /home/enigma/public_html/juice/discussion/admin/GT/SQL/Driver/MYSQL.pm line 35.
  • GT::SQL::Driver::MYSQL (19112): Preparing query: DELETE FROM gforum_Online WHERE guest_id_fk = '126526'



    The forum has been up for 6 months? The problem seems to have happened within the last week or so. No new software was added/changed. The machine is a dual PIII 600mhz with a gig of ram. It's dedicated, and runs about 2 gigs a day in transfer, about 10,000 uniques a day. It normally runs in the 1's without the gforum. Once the machine hits the 10's, it's on it's knees.

    I have disabled it until I can find out what's up!

    Thanks!




  • Diversity by Design

    Last edited by:

    diversity: Jul 30, 2002, 9:04 PM
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    Hi,

    The error itself stems from the guest cleanup, which Gossamer Forum tries to run only about once / week, in 0.9.6. It has since changed to once per day, which is a more reasonable time limit. Unfortunately, you have such a level of guests that cleaning once / week is too much for a single request - hence the browser timeout. The other problem is that in the time it takes one to do the cleanup, another comes along, and also tries to clean up, and another, and another, .... All of these build up the load quite a bit, and slow each other down so more will build up, etc. Although the delete _should_ be very quick, due to an old implementation flaw in our SQL module, it was executed as several (maybe hundreds, or more, depending on the number of guests hitting your site) SQL deletes instead of just the three that are needed. This was, however, fixed some time ago. I strongly recommend you upgrade to the latest stable version - keep in mind that 0.9.6 was still a beta release.

    If, for some reason, you are unable to upgrade, the easiest solution is going to be changing the relevant section of code in admin/GForum.pm. You need to find a line that looks like:

    if ($CFG->{last_guest_cleanup} + 7 * 24 * 60 * 60 < time) {

    (It was changed in some release since 0.9.6; the current 1.1.7 line looks like: if ($CFG->{last_guest_cleanup} + 24 * 60 * 60 < CORE::time) {

    If you change that to:

    if ($CFG->{last_guest_cleanup} + 60 < time) {

    it ought to fix up the problem.

    Although the above should make it go away, I strongly suggest upgrading to 1.1.7. Many modules have been streamlined and optimized to make the last few gforum.cgi releases significantly faster than 0.9.6.

    [snip] - I don't really recommend running 0.9.6 under mod_perl - it had some memory leak issues that have since been fixed up.

    Jason Rhinelander
    Gossamer Threads
    jason@gossamer-threads.com

    Last edited by:

    Jagerman: Jul 31, 2002, 1:24 AM
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    I just noticed you are still running 0.9.6 - I strongly suggest you upgrade. The problem is as I decribed above, however the fix is already present (and has been for quite some time, it turns out) which reduces the mass of queries into just the 3 or 4 required.

    Jason Rhinelander
    Gossamer Threads
    jason@gossamer-threads.com
    Quote Reply
    Re: [Jagerman] Forum hangs with high server load In reply to
    How do I go about downloading this upgrade without purchase?

    Thanks!

    Diversity by Design
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    The download page :)

    http://gossamer-threads.com/scripts/gforum/download.htm
    Quote Reply
    Re: [Paul] Forum hangs with high server load In reply to
    AAGGH!


    First off, the fix to the old version didn't work.

    I just installed the update, and it lost all my customized globals, my templates are broken, and it shows on the front page there are no forums available. Oh... I see you've changed template syntaxes! Sweet! <NOT!>

    So now I have a forum that doesn't work at all, with weeks of work wasted from my old templates? Tell me this isn't true!

    As a webmaster, I feel that nothing sucks more than having to do an upgrade due to the effects of bad coding that you purchased, then to find they changed the template system so much, that all efforts used before no longer work without starting from scratch. This isn't fair!

    Diversity by Design

    Last edited by:

    diversity: Jul 31, 2002, 10:20 AM
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    You should always make backups then you can simply restore your old copy.
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    I understand your frustration, but really. . . upgrading from 0.9.6 to 1.1.7? A lot of forethought and reading on your part was called for.
    Quote Reply
    Re: [Paul] Forum hangs with high server load In reply to
    Oh but I did. See, putting back the old files doesn't run either... because the way the old system was written with our server loads. See the first message of the thread.

    So a restore doesn't help. That's my frustration.

    Diversity by Design

    Last edited by:

    diversity: Jul 31, 2002, 11:16 AM
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    >>
    See, putting back the old files doesn't run either... because the way the old system was written with our server loads
    <<

    I'm not sure what you mean.
    Quote Reply
    Re: [Paul] Forum hangs with high server load In reply to
    Excuse my typing... I had a cycling accident yesterday, so I am thinking faster than I can type. ; )

    What I was saying is the original system was having server load troubles (per the first message) by how the purging of guests was coded.

    A simple restore, doesn't simply solve it. It's ok though guys.... I called a couple of clients, and made room for a day or two to fix it to this new version. I have restored a "few" files, and will sort it out for a nice new forum in this newest version.

    It's good to have this in the forum for searches though, so people can find the answer if they have the same troubles as moi.

    Thanks again!

    Diversity by Design
    Quote Reply
    Re: [diversity] Forum hangs with high server load In reply to
    An update after the upgrade!

    After getting our custom template system back in order (sometimes I think I customize stuff too much) and restoring the globals.txt file... we sent out emails to 1800 users to let them know it's back up and running properly.

    We had a concurrent 75 guest users, as well as 5 registered users at one point, and our server loads were down in the high 2's. Bravo!

    Whatever recoding GT did on this upgrade for speed and dexterity is a vastly noticeable improvement. We were concerned if we had enough hardware to run this forum! There is no question now how speedy, cool and bitchin this forum is!

    Gossamer Threads, our heroes!

    Wink

    Diversity by Design