Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Verisign Sitefinder Troubles

Quote Reply
Verisign Sitefinder Troubles
Hello,

I have been trying to clean our links database, but have run into some problems caused by this stupid sitefinder thing. It is seeing the sitefinder site when a domain isn't registered and letting it into the database!

How can I block my server from seeing the sitefinder domain so the link verifier works?

Thanks,
Kev

-------
BN23.com - LinksSQL powered search and directory.
5.2 Million Links, 685,000 Categories
Quote Reply
Re: [b0rdslide] Verisign Sitefinder Troubles In reply to
Not sure what you mean. What are you refering to when you say 'sitefinder' ?

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Verisign Sitefinder Troubles In reply to
I mean this new site verisign has done which comes up whenever you go to a domain that soes not resolve. There has been all sorts of problems that have been caused by it as suddenly previously unresolvable domains now come up with a site!

Edit:
Forgot to mention that this only occurs with .com's. Here is a litte more info on what verisgn have done:

Quote:
VeriSign's wildcard creates a registry-synthesized address record in response to lookups of domains that are not otherwise present in the zone (including restricted names, unregistered names, and registered but inactive names). The VeriSign wildcard redirects traffic that would otherwise have resulted in a "no domain" response to a VeriSign-operated website with search results and links to paid advertisements.

-------
BN23.com - LinksSQL powered search and directory.
5.2 Million Links, 685,000 Categories

Last edited by:

b0rdslide: Sep 21, 2003, 8:35 AM
Quote Reply
Re: [b0rdslide] Verisign Sitefinder Troubles In reply to
Ah...I see what you mean :( May have to talk to GT about this... as it will affect their products globally :( (i.e all their clients).

I'll have a quick look if there is a fix... but I'm not sure how it would quite work.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [b0rdslide] Verisign Sitefinder Troubles In reply to
BTW, what code is reported? 200? (may be a different code, which would be helpful, as this could be used to filter bad sites).

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Verisign Sitefinder Troubles In reply to
It returns a 302 code.

-------
BN23.com - LinksSQL powered search and directory.
5.2 Million Links, 685,000 Categories
Quote Reply
Re: [b0rdslide] Verisign Sitefinder Troubles In reply to
A quick fix would be to edit admin/Links/Tools.pm, and change the %STATUS_OK and %STATUS_BAD, so that the 302 status appears in the bad links part. Not a complete fix, as this would obviously put a few 'decent' links in the bad category. Upto you what you want to do.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Verisign Sitefinder Troubles In reply to
How about you scan for text that Verisign will always include?
Quote Reply
Re: [webslicer] Verisign Sitefinder Troubles In reply to
The way the LinksSQL verify tool works, is not to grab the whole site, but meerly to check the status of the URL. Imagine if you had to grab every page.... your bandwidth usage would be huge! (even on a small database of say a couple of thousand links).

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [b0rdslide] Verisign Sitefinder Troubles In reply to
Hi. Please give this mod a go. It will attept to grab the page in question, and if it finds that it is indeed a page from verifsign, it will assign a 400 (Bad request) status to it. Seems to work for me.

Be sure to change the paths on line 16 & 28

Backup the old copy first!

INSTRUCTIONS:

Open up admin/verify-child.pl... and after;

Code:
$status = int ( $status );

...add...

Code:
##### HACK BY ANDY ###############
use LWP::Simple;
if ($status == 302) {

my $page = get($url);
if ($page =~ /VeriSign/si) { $status = 400; }
}
##### END HACK BY ANDY ###########

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Verisign Sitefinder Troubles In reply to
The Network solutions mess effectively bypasses the bad url status...

It would be a little like the Cache Page plugin, which does not use crazy bandwidth. Some of course. But its front pages only, not whole sites, and it excludes images...
Quote Reply
Re: [webslicer] Verisign Sitefinder Troubles In reply to
Would it be possible just to block the domain using iptables/ipchains? This would alleviate the problem of having to load the page?

-------
BN23.com - LinksSQL powered search and directory.
5.2 Million Links, 685,000 Categories