Gossamer Forum
Quote Reply
Bad Link automation
One of the most important things running a directory is maintaing the validity of the links. There are quite a few tools in links SQL but i think some automation would be good. This is what i suggest:

i) An extra field added to each link which is something like isError. This would be default to 0 but if is set to 1 then the link is not displayed at all.

Internal Link Checking:

--Every x days (cron job) Links SQL runs through and checks ALL the links.
--Any link that is dead/wrong has isError set to 1 and is thus not displayed
--Every x hours (cron job) Links SQL runs through and checks all the "error" links.
--If the link is ok, then isError is removed and the link is put back into normal status.
--If the link is not ok, the check is logged and nothing changes.
--If after x days (perhaps 2 weeks) and the link is still dead, the link is deleted.

So 3 sections:

i) checking of all links. Dead links set isError to 1
ii) checking of problem links. If ok, is error set back to 0.
iii) time checking, if link dead for x time then removed.


Using pugdog's deadlink.cgi

i) link is reported as DEAD by user.
ii) link does not have isError set to 1 (because user might be wrong!) but the link is added to the next "check problem links setting".

In other words, the use of deadlink.cgi increases the speed that a possible dead link is checked because otherwise it would have to wait until all the links are checked (which if you have a big database is going to be less often).


-Automatic fixing, the "trailing slash situation" is automatically fixed.
-Would there be a way to do the ok, but moved, error automatically?

What do people think? All of 1 would be a cron script i think with 1 extra field added to links. The harder bit will be that all the scripts that use links would need to add a AND isError="0" into the query, still i think this is definitely worth it as I have a lot of links that are currently 404 but don't want to delete them as it might be temporary. This is not however a good way to run a directory.

http://www.ASciFi.com/ - The Science Fiction Portal
Subject Author Views Date
Thread Bad Link automation padders 1853 Mar 31, 2001, 3:39 AM
Post Re: Bad Link automation
pugdog 1786 Mar 31, 2001, 7:39 AM
Post Re: [padders] Bad Link automation
dwh 1489 Jul 21, 2004, 11:44 AM