Gossamer Forum
Home : Products : Gossamer Links : Discussions :

google and bad link reports

Quote Reply
google and bad link reports
It seems google is spidering my pages (a good thing) but creating caches of pages from my link to report bad links.

A couple of people have found their links in my in google searches listed as a reported bad link. The pages listed as bad link reports are neither bad nor are they in my list or reported bad links. It also does the same thing for the recommed, rate and review links. Here's an example

http://www.google.com/...mp;start=20&sa=N

Has anyone else encountered this? Or can someone suggest a way around it?

CCUnet


my Christian web
Quote Reply
Re: [ccunet] google and bad link reports In reply to
Hi,

I think that the best way around this is to add a sentence at the top of the bad link report page to say 'If you arrived here from a search engine looking for <%Title%>, you can find it <link to detail page>here</link>.'

Alternatively, you could add an instruction to tell the robots not to index that page - but if you are getting good rankings with it I think that would be a waste.

Laura.
The UK High Street
Quote Reply
Re: [ccunet] google and bad link reports In reply to
This is a problem with the script, I've tried to figure out how to get around. The spider "clicks" on links, so it "clicks" on the "report bad link" link, and the link is flagged as bad.

I've tried various things to make it stop that, but short of requiring a user to log-on to access the bad-link report, nothing keeps the spiders out, other than editing your robots.txt and telling them to ignore the badlink.cgi

Might be able to modify it to do what some whois engines are doing, providing a code number on the page that you have to type into the form before it can report the link as bad -- maybe make the user type in the link ID? (confirm the ID?)

I'm almost ready to keep the spiders out of my sites except for the "home" pages. They do a lot of damage, and the competition is going to get worse, not better as the economy stalls.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [pugdog] google and bad link reports In reply to
Not a good idea to have just a link for bad link reporting.
Every curious people will click on it to see what's doing Wink

The solution is to use a form, before it is submitted as bad link.
So the link displays a confirm page with a form to submit bad link report.

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] google and bad link reports In reply to
I tried the form. The spiders figured it out. You need some interactive data with the form.

I tried to make it simple, click "report link as bad" get a page that says "report this link". The spiders keep pressing buttons, when a normal user would stop.

The only way is to make the user enter some "key" data, and I didn't want to go that far, since that is further than most users would go. Two clicks, yes. Two clicks, plus type in some numbers, no.

Users are lazy. Very.

Best way is to take badlink.cgi out via robots.txt


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [pugdog] google and bad link reports In reply to
Did you tried <input type=hidden name="form" value="1"> ?
This should work only if submit is pressed.

I do not think the spiders do the press the same say as a user. It may happen that the hidden value could help.

BTW: robots.txt can only exclude directories. badlink.cgi is not in a different directory.

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...

Last edited by:

webmaster33: Dec 31, 2002, 6:24 AM
Quote Reply
Re: [ccunet] google and bad link reports In reply to
Hi,

You could have badlink go to a page and have a dropdown select list with possible reasons, and a submit button. Robots don't submit forms, so you should be fine.

Cheers,

Alex
--
Gossamer Threads Inc.
Quote Reply
Re: [Alex] google and bad link reports In reply to
I didn't try a drop down list, *but* to say that spiders don't do submit forms is a bit simplistic.

There are many spiders set to specifically follow forms, bypass entrance blocks, and all sorts of other things. They are usually malicious, and there is no way to really block them.

The *only* so far successful way is to use an embedded graphic "key" in a mixed font face that the user has to type in to get the form to submit. If you are a "hot" site, there are ways to analyze the graphic, and even bypass that if you don't change your HTML often enough.

But, asking the user to make a selection, then fill in something, should block *most* spiders on most sites.

I think picking from a drop down list, and re-entering the Link ID should be sufficient. When I release the widgets kit, the badlink reporting will be a part of the updated scripts.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [pugdog] google and bad link reports In reply to
Well, try out the drop down list & form submit, what Alex suggested.
After a few weeks you will know, if you were able to stop the spiders or not. Wink

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] google and bad link reports In reply to
The bad link template is showing up as code in firefox and netscape can't quite figure out why

Anyone have any ideas?

http://www.mychristianweb.com/.../bad_link.cgi?ID=658

Thanks
CCUent
my Christian web
Quote Reply
Re: [ccunet] google and bad link reports In reply to
It seems the script outputs bad header: text instead html.

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] google and bad link reports In reply to
There were some fixes posted, the file in the download area is not the file that should be there, it seems.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [ccunet] google and bad link reports In reply to
In Reply To:
The bad link template is showing up as code in firefox and netscape can't quite figure out why

Anyone have any ideas?

http://www.mychristianweb.com/.../bad_link.cgi?ID=658

Thanks
CCUent


Anyone know how to fix this? I am getting the same error in FireFox.
Quote Reply
Re: [Jobu] google and bad link reports In reply to
No one? This is pretty weird... must be some sort of header/formatting error? Maybe the template is being passed as text?