Home : Products : Gossamer Forum : Discussion :

Products: Gossamer Forum: Discussion: Googlebot and guest=xxxxxxxxx: Edit Log

Here is the list of edits for this post
Googlebot and guest=xxxxxxxxx
Hi.

gogglebot was hammering server like mad often triggering mod_dosevasive and it was on forum that it was happening most aggressively. Contacted 'G' and this is the reply:

Hi <Edited>,

Thank you for your reply. As you've observed, session IDs can cause
problems for our robots. While we can reduce the crawl rate for this page,
you can disable session IDs for Googlebot so that our robots may crawl
your site more efficiently and not affect your crawl rate.

Regards,
The Google Team


Is it possible to eliminate guest=xxxxxx from the url? Apparently each guest=xxxxxx for the same url (post/forum/etc etc) is being viewed as a different url by bot and hence the hammering. MSN hammering is also of a similar scale, the diference being that MSN doesn't care to bother. (Even their IE 6 causes hammering in certain cases, perhaps a ploy to jack up their "Browser Stats As Analyzed by Log Analyzers")

Thanks
HyTC

Last edited by:

HyperTherm: Mar 4, 2005, 3:21 PM

Edit Log: