Gossamer Forum
Home : Products : Links 2.0 : Customization :

Re: [Kangaroo] Verifying big links databases: temporary solution.

Quote Reply
Re: [Kangaroo] Verifying big links databases: temporary solution. In reply to
Ok, i'm gonna put this in a zip for now at my site
http://lh.links247.net/downloads/verify

Ok, If you aren't on a cobalt, read at least the first 60 lines. There are a few comments in those lines of things to change.

By default lwp is taken off. It doesn't run well with the script. At least when i tested it, every link came up bad, but with IO it was correct. But if you feel the need, in the zip is a nph-converter.cgi script. Upload the script to the same dir as nph-verify.cgi and chmod it correctly, then run it in the browser and the comments in front of the lwp parts will be taken off. I don't recommend it.

You will need to make an output folder in you admin folder, as well as an output/staggered folder (after you make the output folder, just go in it and make a staggered.); Chmod them to 777. Do not put any folders or files in them. The only things that can go in them is
for the output folder => 'staggered folder','index.htm(l)','reports'
for the staggered folder => 'index.htm(l)','staggered reports'

The reason there is a staggered folder is because, in the output folder, standard reports will be made here when running from telnet or cron. But when in staggered mode, every new page the browser goes to will input the bad links into the html pages incremently Smile. So in a standard telnet verify, all links will be verified on one page and you may have 20 bad links. But in staggered, they are split, and for each page that has bad links, those will be added to the report in the staggered folder.

I set it up to only show reports for 6 days, after that they will be deleted. This is the case for both staggered and telnet mode.
Oh btw... there are three modes
staggered => In the browser w/spanning, but auto refresh to the next page until the end like build staggered and prints report to output/staggered in the admin folder
telnet => from telnet screen automatically does all links at once, and prints report to output folder in the admin
standard => basically just a span of the links you verify. No report building. It's basically like in the admin when viewing all links, and its spanned except this will be verifying while showing you the bad links as well.

That should be about it.
I'm not too proud of the way I wrote the addon but i was working for speed and functionability more than easy on the eyes to look at
Definitely don't forget to change this line : # 45:
require "/home/sites/lh.links247.net/web/look-bin/Look/admin/links.cfg"; # Change this to full path to links.cfg if you have problems.
To your full path.

This is why i said to read about the first 60 lines.
in cobalt, certain things don't need to be printed so you'll see this
Code:
# print "HTTP/1.0 200 OK\n";
print "Content-type: text/html\n\n"; # Delete this and the above line and uncomment the lines below if you aren't on a cobalt.
# if ($ENV{'REQUEST_METHOD'}) { # Replace with these
# print "HTTP/1.0 200 OK\n"; # Replace with these
# print "Content-type: text/html\n\n"; # Replace with these
# }
Just follow the comments and it should be fine.

Oh and so it can run in telnet, add this to db_utils.pl
or if you don't have telnet, just add it anyway:

Code:
sub parse_form1 {
# --------------------------------------------------------
# Parses the form input and returns a hash with all the name
# value pairs. Removes any field with "---" as a value
# (as this denotes an empty SELECT field.
#
my (@pairs, %in);
my ($buffer, $pair, $name, $value);

if ($ENV{'REQUEST_METHOD'} eq 'GET') {
@pairs = split(/&/, $ENV{'QUERY_STRING'});
}
elsif ($ENV{'REQUEST_METHOD'} eq 'POST') {
read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});
@pairs = split(/&/, $buffer);
}


PAIR: foreach $pair (@pairs) {
($name, $value) = split(/=/, $pair);

$name =~ tr/+/ /;
$name =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;

$value =~ tr/+/ /;
$value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;

($value eq "---") and next PAIR;
exists $in{$name} ? ($in{$name} .= "~~$value") : ($in{$name} = $value);
}
return %in;
}


And this to links.cfg
Code:
$db_verify_url = $db_dir_url . "/nph-verify.cgi"; # Verify script.
Lavon Russell
LookHard Mods
lavon@lh.links247.net
Subject Author Views Date
Thread Verifying big links databases: temporary solution. Kangaroo 14756 Nov 5, 2001, 6:24 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13946 Nov 5, 2001, 6:30 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 13968 Nov 5, 2001, 7:28 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13922 Nov 5, 2001, 10:04 AM
Post Re: [Bmxer] Verifying big links databases: temporary solution.
Vs_Greg 13875 Nov 5, 2001, 11:52 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 13923 Nov 5, 2001, 12:22 PM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13994 Nov 6, 2001, 5:38 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 13934 Nov 6, 2001, 6:42 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Paul 13923 Nov 6, 2001, 7:01 AM
Thread Re: [RedRum] Verifying big links databases: temporary solution.
Kangaroo 13926 Nov 6, 2001, 8:14 AM
Post Re: [Kangaroo] Verifying big links databases: temporary solution.
Paul 13933 Nov 6, 2001, 8:17 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13869 Nov 6, 2001, 11:23 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 14715 Nov 6, 2001, 12:16 PM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13853 Nov 7, 2001, 11:26 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 13914 Nov 7, 2001, 11:34 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13893 Nov 7, 2001, 11:58 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
marinedesign 13828 Nov 7, 2001, 1:05 PM
Post Re: [marinedesign] Verifying big links databases: temporary solution.
Vs_Greg 13834 Nov 7, 2001, 2:25 PM
Thread Re: [marinedesign] Verifying big links databases: temporary solution.
Bmxer 13869 Nov 7, 2001, 2:31 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 13931 Nov 8, 2001, 5:17 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Paul 13870 Nov 8, 2001, 5:21 AM
Thread Re: [PaulW] Verifying big links databases: temporary solution.
madtech 12502 Nov 16, 2001, 3:31 PM
Post Re: [madtech] Verifying big links databases: temporary solution.
Paul 12559 Nov 16, 2001, 3:38 PM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 13850 Nov 8, 2001, 1:36 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Vs_Greg 14002 Nov 8, 2001, 2:30 PM
Thread Re: [sc2utp] Verifying big links databases: temporary solution.
Bmxer 13793 Nov 8, 2001, 2:50 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Vs_Greg 13804 Nov 8, 2001, 2:55 PM
Post Re: [sc2utp] Verifying big links databases: temporary solution.
Vs_Greg 12637 Nov 8, 2001, 3:10 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 12659 Nov 8, 2001, 4:21 PM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 12686 Nov 8, 2001, 7:15 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 12684 Nov 9, 2001, 2:05 AM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
Bmxer 12696 Nov 9, 2001, 2:08 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Bmxer 12636 Nov 10, 2001, 10:47 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
dvd871 12643 Nov 10, 2001, 5:37 PM
Thread Re: [dvd871] Verifying big links databases: temporary solution.
dvd871 12611 Nov 10, 2001, 6:53 PM
Thread Re: [dvd871] Verifying big links databases: temporary solution.
Bmxer 12880 Nov 10, 2001, 7:14 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
marinedesign 12616 Nov 10, 2001, 8:38 PM
Post Re: [marinedesign] Verifying big links databases: temporary solution.
marinedesign 12582 Nov 10, 2001, 8:57 PM
Post Re: [Bmxer] Verifying big links databases: temporary solution.
dvd871 12595 Nov 10, 2001, 9:13 PM
Post Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 12549 Nov 11, 2001, 4:33 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Vs_Greg 12614 Nov 11, 2001, 2:28 AM
Thread Re: [sc2utp] Verifying big links databases: temporary solution.
Vs_Greg 12584 Nov 11, 2001, 2:51 AM
Thread Re: [sc2utp] Verifying big links databases: temporary solution.
Bmxer 12522 Nov 11, 2001, 6:04 AM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
madtech 12584 Nov 13, 2001, 7:29 PM
Thread Re: [madtech] Verifying big links databases: temporary solution.
madtech 12611 Nov 13, 2001, 8:04 PM
Thread Re: [madtech] Verifying big links databases: temporary solution.
Bmxer 12597 Nov 13, 2001, 9:31 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
madtech 12570 Nov 13, 2001, 10:56 PM
Post Re: [madtech] Verifying big links databases: temporary solution.
madtech 12456 Nov 14, 2001, 1:37 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
Kangaroo 12480 Nov 16, 2001, 2:58 PM
Thread Re: [Kangaroo] Verifying big links databases: temporary solution.
gossy 3372 Jan 25, 2003, 1:44 AM
Thread Re: [gossy] Verifying big links databases: temporary solution.
wysardry 3471 Jan 25, 2003, 2:29 PM
Post Re: [wysardry] Verifying big links databases: temporary solution.
gossy 3437 Jan 25, 2003, 11:02 PM
Thread Re: [Bmxer] Verifying big links databases: temporary solution.
timvdh 12419 Mar 26, 2002, 3:25 AM
Post Re: [timvdh] Verifying big links databases: temporary solution.
sensovision 3410 Mar 23, 2003, 5:04 AM