Gossamer Forum
Quote Reply
Re: SPIDER robots.txt In reply to
Hi Hemen,

The easiest way of dealing with this is to go into Plugins->Database-> Bulk Operations, choose Links and click on Go.

Look for any URL containing robots.txt.

When you have a collection of all such links, delete them from the database.

Hope this helps



Subject Author Views Date
Thread SPIDER robots.txt HEMEN 4056 Jul 30, 2001, 1:37 PM
Thread Re: SPIDER robots.txt
ryel01 3927 Jul 30, 2001, 1:58 PM
Thread Re: SPIDER robots.txt
Paul 3940 Jul 30, 2001, 2:02 PM
Thread Re: SPIDER robots.txt
HEMEN 3931 Jul 30, 2001, 2:13 PM
Post Re: SPIDER robots.txt
ryel01 3921 Jul 30, 2001, 2:37 PM
Thread Re: SPIDER robots.txt
kitsune 3917 Jul 30, 2001, 2:53 PM
Post Re: SPIDER robots.txt
HEMEN 3909 Aug 1, 2001, 12:25 AM