Gossamer Forum
Home : Products : Gossamer Links : Discussions :

SPIDER ACCESS

Quote Reply
SPIDER ACCESS
Hello

When i check my stats log i can't found the trace of spider access!
What the signature of spider access?

Thank you

Bachir
http://www.wilayates.com


Quote Reply
Re: SPIDER ACCESS In reply to
Hi,

Please take a look at this write up.

http://www.robotstxt.org/wc/faq.html#metoo

Here's a snippet from the site:

In Reply To:
How do I know if I've been visited by a robot?

You can check your server logs for sites that retrieve many documents, especially in a short time.

If your server supports User-agent logging you can check for retrievals with unusual User-agent heder values.

Finally, if you notice a site repeatedly checking for the file '/robots.txt' chances are that is a robot too.
Hope this helps :)
Aki

Quote Reply
Re: SPIDER ACCESS In reply to
Hello

my question is not about the robot?
When i check my access log i don't found any trace of links spider, as googlebot...

Thank you

Bachir

http://www.wilayates.com


Quote Reply
Re: SPIDER ACCESS In reply to
Is the spider spidering your site?






PUGDOG´┐Ż Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: SPIDER ACCESS In reply to
Yes