Gossamer Forum
Home : Products : Gossamer Forum : Development, Plugins and Globals :

Static vs. dynamic bulletin boards for search engines

Quote Reply
Static vs. dynamic bulletin boards for search engines
We are currently debating upgrading from UBB 5.x to either UBB6 or vBulletin, but looking at other options and came across the Gossamer Threads product. Looks great! Really like the posting screen with the word-processor like interface.

We're also very keen on the possibility of being able to have our bulletin board threads indexed by Google, etc. Much valuable content exists on our bulletin board currently, and one of our big concerns with going to dynamic is the problem of external search engines not being able to index the content.

Upon posting an inquiry on our website hosting service forum (FutureQuest), I received the following comments:
Quote:
"There are at least 2 similar mods available for vBulletin to improve search engine spidering, but they are on the no-no list at FutureQuest. The problem is that you are essentially telling the search engine that the dynamic content is static, and while search engine spiders will slow down on dynamic content so as to not overload the server, on static content they will slurp your site as fast as possible which might result in those "too many connection" errors which can make it so everyone's forums are temporarily inaccessible. My guess would be that the Gossamer Threads dynamic->static templates would be in this same class since they use mod rewrite to do essentially the same thing - fine for a dedicated server but frowned upon for a shared server. "
The host's CTO also commented:

Quote:

"Though your normal traffic may be light, and your visitors are viewing a segment of your forums, the spiders will go in and view *all* of your forums contents in a much shorter amount of time... This in effect concentrates the activity into a sharp spike in activity...

Think of it this way, how long would it take you to click through every forum link...

Now if you can present the forums pages to the search engines as normal HTML files, without the assistance of any CGI methods - then I would be open to it... However if CGI is required for each page indexed - then yes, it can quickly become a problem... "

So my question is:

If we run Gossamer Threads Forum software with this 'Search Engine Friendly Templates' setup, will we get kicked off our website host? For reference, we have approx. 1,300 users, approx. 2,400 threads generating 65 MB of content in UBB 5.47.

Any information would be very useful in our deliberations Laugh

Thanks in advance

Susan Johnson

Share the Dream
Horizons Unlimited

Quote Reply
Re: [SusanHorizons] Static vs. dynamic bulletin boards for search engines In reply to
Hi Susan,

You can achieve the search engine friendly url's either using mod_rewrite or PATH_INFO, but both methods still display dynamic content. I can't predict whether you would get kicked off or not, but I'm not sure I agree with their reasoning. Most well behaved spiders (i.e. Google, Altavista, etc) will limit the number of requests to your domain to 1 or 2 hits per second top regardless of cgi or static html pages.

Even if you don't use the search engine templates, you can always have a misbehaved robot, or some user using an offsite downloader hammer a forum. I don't think it would be grounds for getting kicked off though.

Cheers,

Alex
--
Gossamer Threads Inc.