Gossamer Forum
Home : Products : Gossamer Links : Development, Plugins and Globals :

Andy's spider plug in question..

Quote Reply
Andy's spider plug in question..
Hi

I have been using this great toolfor a while, BUT lately I am getting:

Restricted! on over 70% of the titles I am trying to spider,

My FollowRobotsTxt is set to 0..

Any ideas?
Regards
KaTaBd

Users plug In - Multi Search And Remote Search plug in - WebRing plug in - Muslims Directory
Quote Reply
Re: [katabd] Andy's spider plug in question.. In reply to
Which version are you using? This can be down to their being no <title></title> tags in the HTML page in question, or if you are using an older version of the plugin, the FollowRobotsTxt setting is not being read correctly :(

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Andy's spider plug in question.. In reply to
Hi

It shows 1.6

Thanks
Regards
KaTaBd

Users plug In - Multi Search And Remote Search plug in - WebRing plug in - Muslims Directory
Quote Reply
Re: [katabd] Andy's spider plug in question.. In reply to
Can you give me an example URL of where you are having this problem? (i.e a URL that its trying to spider) ?

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!