Gossamer Forum
Home : Products : Gossamer Forum : Discussion :

Speed up the search

Quote Reply
Speed up the search
Hi

Currently our forum has about 400,000 postings and I would consider my forum heavily utilized. Under peak hours, whenever someone does a search in the forum, the first initial search takes 20-40 secs to complete. During this period, my forum will slow down to a crawl for the rest of the users who are just browsing the forum and my server load shoots up.

I am running GForum on mod_perl with mysql and my 2.8Ghz server has 2GB of ram dedicated to running GForum. I am also using the reverse proxy as recommend by the mod_perl guide for max performance.

Does anyone have any advise on how to finetune the search performance? I am not very familiar with mysql and I think that should be where I should focus my attention on?? I am using the sample my.cnf config file supplied from mysql for large servers with at least 512mb of ram.

I have already converted the following tables suggested by Jason here into InnoDB.
http://www.gossamer-threads.com/...orum.cgi?post=263938
The tables converted are
Online
PostNew
PostView
Post_tree

Do you think converting the Post table into InnoDB will help as well? According to phpMyAdmin, my Post table is taking up 250MB of space now.

What other steps should I take to optimise and speedup the search function. Any advise is appreciated from the gurus here.

Thanks!! Smile

Last edited by:

dauhee: Aug 15, 2004, 9:02 AM
Quote Reply
Re: [dauhee] Speed up the search In reply to
Are you using an indexed post table? If not, doing so will significantly speed up search results.

To index your post table, go to the Tools menu of the admin panel, and click on "Post" in the Table Editor section. At the very bottom of the page you'll see an "Indexing scheme" dropdown box; I recommend you change it to "INTERNAL", then click the "Update Table Post" button. After doing so, you will have to reindex your database. To do so, you can click the "Reindex" link in the Table Editor section of your admin panel; if you run into a problem with your browser timing out (i.e. the logo in the top right corner of most browsers stops moving), this can also be run from a shell (i.e. ssh) by running:

Code:
cd /system/path/to/your/admin/tools
./reindex.pl

Jason Rhinelander
Gossamer Threads
jason@gossamer-threads.com
Quote Reply
Re: [dauhee] Speed up the search In reply to
Hi.

Not sure how much it would help but you could also try mod_gzip in yr fe httpd. Perhaps GT could comment more on this as they are using fe/be httpd combination with mod_gzip in fe i presume....

Thanks
Anup
Quote Reply
Re: [anup123] Speed up the search In reply to
Dude.. you gotta stop using these abbreviations .. they are confusing Crazy

yr
fe
fe/be

heheh
Quote Reply
Re: [Andy] Speed up the search In reply to
Hi.

fe/be : Front End/Back End ...
GT : Gossamer Threads
GC : Gossamer Community
GM : Gossamer Mail
GF : Gossamer Forum
LSQL : Links SQL
yr : Your or Year depending on context of sentence :)

heheh : never used that

That's all that i use from the reuseable repository as i have not yet reached the typing speeds that you have... would however, try to shorten the list Wink

Anup
Quote Reply
Re: [anup123] Speed up the search In reply to
Quote:
That's all that i use from the reuseable repository as i have not yet reached the typing speeds that you have... would however, try to shorten the list

LOL!

Maybe just put them in caps lock then...i.e;

"GT" and not "gt"
"GF" and not "gf"
...etc

Just makes it easier to read :) (I've not replied to some of your threads, simply due to the fact I don't have time to try and understand what all the abbreviations mean, and thus what you mean Unimpressed).

Quote:
heheh : never used that

Thats one of mine =)

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Speed up the search In reply to
ooops i presume it's a hangover of working with case insensitive GC Smile

For example i create an account Administrator in GC
I can login typing administrator there.

Would correct that definitely as lc abbreviations get lost in the sentences if too long...

Anup
Quote Reply
Re: [anup123] Speed up the search In reply to
Quote:
For example i create an account Administrator in GC
I can login typing administrator there.

Yeah, I think thats because all usernames are "lower cased", to stop any problems with duplicate names (i.e "Andy" and "ANdy").

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Speed up the search In reply to
In fact on GM i enforced lc for username and for LSQL, the signup is from within GM so that's automatically ensured. With GC never reached that stage as i failed on few basic requirements wrt dates etc doing the way it was done in GM and LSQL.

On unresponded posts (with or w/o abbreviations) , most of them are in GC, though I am still poking at it...

Anup
Quote Reply
Re: [dauhee] Speed up the search In reply to
If you really want to speed things up, talk to Alex about the Lucene search. I know they're working on making this a commercial plugin. You can try it out on my site if you want.

http://www.dropzone.com/...forum.cgi?do=search;

Search for the word "jump"...

Your search for jump returned 117614 results in 0.354s.

Safe swoops
Sangiro
Quote Reply
Re: [Jagerman] Speed up the search In reply to
Ok I checked and I am not using an indexed post table. Will do what you recommended. I did a search in the forum about the internal indexing scheme and it seems the re-indexing process will take a while. Hence I will do this over the weekend. Will let everyone know the results. Thanks!!Smile
Quote Reply
Re: [anup123] Speed up the search In reply to
Hi Anup

Thanks for your suggestion. Yes I am already using mod_gzip on my web servers. It does reduce my bandwidth usage by quite a bit after I implemented it.

Cheers.
Quote Reply
Re: [sangiro] Speed up the search In reply to
Thanks Sangiro.

Wow, I checked out the search on your site. That sure is fast!!

Cheers
Quote Reply
Re: [Jagerman] Speed up the search In reply to
Hi Jason

After monitoring the results after using the internal indexing scheme, I am happy to say that the search is now noticeably faster. The timing for complex searches has reduced to 5-10 seconds on average.

For everyone else who is thinking of converting to the internal indexing scheme, just be careful that the initial indexing can take quite a while. On my forum with half a million postings, it took more than 4 hours to complete. During the process, everything is very slow so it is recommended to take your forum offline while doing it.

Regards
Quote Reply
Re: [dauhee] Speed up the search In reply to
Quote:
For everyone else who is thinking of converting to the internal indexing scheme, just be careful that the initial indexing can take quite a while. On my forum with half a million postings, it took more than 4 hours to complete. During the process, everything is very slow so it is recommended to take your forum offline while doing it.

Thats pretty quick =)

On LinksSQL, you can do similar things with the search options. It can take anywhere between 5-24 hours to complete on 1,000,000+ links :|

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [sangiro] Speed up the search In reply to
Your site is giving 403 error

Thanks
HyTC
==================================
Mail Me If Contacting Privately Is That Necessary.
==================================