Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Anti-scraping, URL re-direct scripts...

Quote Reply
Anti-scraping, URL re-direct scripts...
Is there such a script out there that offers additional protection of the URLs within my Links SQL directory from others scraping for their own purpose? If not, what additional protection could I employ from keeping others from snatching the URLs in my directory.

Thanks.

mgeyman
Quote Reply
Re: [mgeyman] Anti-scraping, URL re-direct scripts... In reply to
The best protection is using jump.cgi and not using the <%URL%> tag anywhere in your templates.

Beyond that, someone can follow your links out and get the urls. There is no way to prevent that. An automated robot can crawl your site, looking for all links a depth of 1 off your site, for example. All you can do is make it hard. There is no way to tell that sort of user from a person going to that site.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [mgeyman] Anti-scraping, URL re-direct scripts... In reply to
 
maybe you could use jump.cgi as pugdog said, and also use the frames plugin too after they've jumped so the URL is another level deep again?

regan.
Quote Reply
Re: [ryel01] Anti-scraping, URL re-direct scripts... In reply to
Thanks pugdog and ryel01 for your responses.

ryel01, where is this Frames Plug-in you are refering to? I can't seem to find it for some reason.

Thanks.

mgeyman
Quote Reply
Re: [mgeyman] Anti-scraping, URL re-direct scripts... In reply to
Here you go;

http://www.ultranerds.com/cgi-bin/details/24.html

Its a plugin I wrote a while back, which lets you have an 'Ask.com' style 'frame' when redirecting visitors off to a listings site.

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Anti-scraping, URL re-direct scripts... In reply to
Thanks a lot Andy!

mgeyman
Quote Reply
Re: [mgeyman] Anti-scraping, URL re-direct scripts... In reply to
Another benefit I have discovered is that google has all my jump.cgi links spidered, and since they use the frame that means if they turn up in a google search the url they click goes to the site in my frame. So make sure your frame has a link back to your main site!!!! Mine has a link to the site, and a link to review the site (which I also have labeled to report a site as a bad link). My reviews have increased since using this framing method, not a lot, but enough.