Gossamer Forum
Home : Products : Links 2.0 : Customization :

altavista search - new mod

Quote Reply
altavista search - new mod
This mod allows you to search theweb, categories, or both... Keep in mind I have posted the code here as if you are using templates. It shouldn't be too hard to figure out how to change it...

You can see this in action here:
www.magicdirectory.com/cgi-bin/search.cgi?query=david+blaine&search=both

or just the web search:
www.magicdirectory.com/cgi-bin/search.cgi?query=david+blaine&search=theweb

or just the category search:

www.magicdirectory.com/cgi-bin/search.cgi?query=david+blaine

In order for the option to search both the web and categories follow these instructions...

In search.cgi under:

Code:
# Display the form if called with no input.
(keys %in <= 0) and &site_html_search_form() and return;

put this:

Code:
($in{'search'} eq "theweb") and &altavista;
($in{'search'} eq "both") and &altavista;

also at the bottom of search.cgi add these routines...

Code:
sub altavista {
# --------------------------------------------------------
# Search Altavista
my %in = &parse_form;
use LWP::Simple;
my $query = $in{'query'};
my $query_q = &urlencode($query);
my $page = ($in{'nh'} * 10) | | 0;
my $pageXX = $page - 10;
my $url = "http://www.altavista.com/cgi-bin/query?q=$query_q&text=yes&stq=$pageXX";
my $results = get ($url);
$results or &site_html_search_failure ({ error => "No Results Available.", %in }) and return;
# If we see this, then we couldn't get a match.
my $none = quotemeta ("AltaVista found no document matching your query.");
if ($results =~ m/$none/) {
&site_html_search_failure ({ error => "No Matching Links", %in }) and return;
}
else {
if ($results =~ /([^>\s]+) pages found./) { $link_hits = $1; }
else { &site_html_search_failure ({ error => "Can't Parse Results", %in }) and return;
}
my $FONT = 'font face="verdana, arial" size="2"';
my $FONT_SMALL = 'font face="verdana, arial" size="1"';
if ($link_hits) {
while ($results =~ m#<dl><dt>\n<b>(\d+)\.\s*</b>\n<a href="([^"]+)">\n<b>([^<]+)</b></a></dt>\n<dd>([^<]*)</dd>\n<br><b>URL:</b> <font color="\#808080">([^<]+)</font>\n<br><font color="\#808080">\n([^<]+\n[^<]+)</font>#sog) {
($count, $url, $title, $description, $url2, $last) = ($1, $2, $3, $4, $5, $6);$url2 =~ s,/$,,;
$last =~ s,\n,,;$last =~ s,\s\s+, ,;
$link_results .= qq~<p>\n<$FONT_SMALL><b>$count.</b></font> <$FONT><a href="$url">$title</a><br>\n$description</font><br>\n<$FONT_SMALL><b>URL:</b> <font color="#808080">http://$url2</font></font>\n~;
}
}
}
$next = &next_hits($page, $link_hits, 10)

if ($in{'search'} eq "both") {
$link_results2 = $link_results;
$link_hits2 = $link_hits;
$next2 = $next;
return;
}
&site_html_search_results;
exit;
}
The routine above this has changed...
Code:
sub next_hits {
# ----------------------------------------------
# Creates a next hits toolbar.
#
my ($page, $numhits, $maxhits) = @_;
my ($left, $right, $upper, $lower, $next_hit, $prev_hit, $i);
my $nh = int ($page / $maxhits) | | 1;
my $output = '';
$numhits =~ s/,//g;
$numhits = int ($numhits);

my $next_url = $ENV{'QUERY_STRING'};
$next_url =~ s/\&nh=\d+//;
$next_hit = $nh + 1; $prev_hit = $nh - 1;

# First, set how many pages we have on the left and the right.
$left = $nh; $right = int($numhits/$maxhits) - $nh;
# Then work out what page number we can go above and below.
($left > 7) ? ($lower = $left - 7) : ($lower = 1);
($right > 7) ? ($upper = $nh + 7) : ($upper = int($numhits/$maxhits) + 1);
# Finally, adjust those page numbers if we are near an endpoint.
(7 - $nh >= 0) and ($upper = $upper + (8 - $nh));
($nh > ($numhits/$maxhits - 7)) and ($lower = $lower - ($nh - int($numhits/$maxhits - 7) - 1));

# Then let's go through the pages and build the HTML.
($nh > 1) and ($output .= qq~<a href="$ENV{'SCRIPT_NAME'}?$next_url&nh=$prev_hit">[<<]</a> ~);
for ($i = 1; $i <= int($numhits/$maxhits) + 1; $i++) {
last if ($i > 19);
if ($i < $lower) { $output .= " ... "; $i = ($lower-1); next; }
if ($i > $upper) { $output .= " ... "; last; }
($i == $nh) ?
($output .= qq~$i ~) :
($output .= qq~<a href="$ENV{'SCRIPT_NAME'}?$next_url&nh=$i">$i</a> ~);
(($i * $maxhits) >= $numhits) and print "Exiting3" and last; # Special case if we hit exact.
}
$output .= qq~<a href="$ENV{'SCRIPT_NAME'}?$next_url&nh=$next_hit">[>>]</a> ~ unless (($nh == $i) or ($nh == 19));
return $output;
}

In site_html_templates.pl add these under sub site_html_search_results:

Code:
link_hits2 => $link_hits2,
next2 => $next2,
link_results2 => $link_results2,

Now in your actual template add:

Code:
<%if link_hits2%>
<%link_hits2%>
<%endif%>

<%if next2%>
<%next2%>
<%endif%>

<%if link_results2%>
<%link_results2%>
<%endif%>

Thats it! Keep in mind that you can edit the failure page to display one result if the other isn't working etc...!

Don't forget about deleting the space between | |.

------------------
- Jonathan Gamble

www.magicdirectory.com

[This message has been edited by jdgamble (edited April 09, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Oh yeah, you can add a drop down list that will select what search you want on your home page and on your search page:
http://www.magicdirectory.com/cgi-bin/search.cgi

Code:
<select size="1" name="search">
<option value="theweb">The Web</option>
<option value="categories">Categories</option>
<option selected value="both">Both</option>
</select>

------------------
- Jonathan Gamble

http://www.magicdirectory.com/

[This message has been edited by jdgamble (edited March 15, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Nice Mod. Keep up the good work. HOWEVER, you should probably delete the extraneous codes like:

Code:
($in{'query'} eq "david blaine") and &david_blaine;

so that novice users won't freak when they try to use your Mod and experience error messages.

Also...nice use of the Category Number code hack...

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums


[This message has been edited by AnthroRules (edited March 13, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Thanks. I had made this mod before experimenting, but I couldn't find any use for it.

I removed the extra code. That was for a different mod I made.

------------------
- Jonathan Gamble

http://www.magicdirectory.com/




Quote Reply
Re: altavista search - new mod In reply to
For anyone interested, I've updated this mod again to make it easier to install and less complicated. Also it is one less sub routine to worry about.

------------------
- Jonathan Gamble

http://www.magicdirectory.com/




Quote Reply
Re: altavista search - new mod In reply to
Nice job...I was going to suggest that...Step ahead of the game!

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
I am looking for help for this mod,
I have copied and pasted it from here and did exactly what was mentioned in this thread.

I have the LWP module so that is not the problem.

You can view my search.cgi as a text file at
http://www.on-linecanada.com/search.txt

I did notice that when I cut and paste using IE 5.0 it paste it as one long line,
In Netscape it paste it like is seen in the txt file I do not know if this is why I am getting an Internal Server Error.

Any help would be appreciated, also if someone could email me the code as a text file if the way it was pasted is the problem.

Eliot if you see this post could you maybe recommend some good perl books Title or Author I would like to pick some up.

Thanks People

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
Exactly what problems are you having? What errors are you getting?

------------------
- Jonathan Gamble

http://www.magicdirectory.com/




Quote Reply
Re: altavista search - new mod In reply to
* "Perl for Dummies"
* "Perl Cookbook"
* "Learning Perl, 2nd Edition"
* "Programming Perl, 2nd Edition"
* "Advanced Perl Programming"

All can be found at Amazon.com or ora.com.

Please provide the exact error messages you receive if any.

Regards,


------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
I search for "whatever" in all area's
web, categories and both and I get an Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request.

I am positive that it was done exactly like you mention above. In fact I leave the other pages alone ie. site_html_templates.pl (leave the changes to that page there)and re-install my back up search.cgi and my search works fine for my catagories then try it again and same problem.

I think it has to do with the cut and paste, for some reason it distorts I used both browsers IE and netscape yet when I have done other mods it doesn't and it works fine.

Hmmmmm...

Thanks Redneck

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
hi,

this mod look so great i must have it.. but i can get it to work (interal server error)

there was some talk about a new and more simple code !! could I see it ?

Allan
Quote Reply
Re: altavista search - new mod In reply to
crudetower,

Uh...the edited codes are in the FIRST Post. See the edited message at the bottom of the Post...jdgamble has edited the codes in his first Post. Copy the codes as written and paste them into your search.cgi file.

RedNeck,

What Editor program are you using to edit your search.cgi file? I would recommend editing your cgi scripts in WordPad, EditPlus (www.editplus.com). If you are editing "on-line" via pico or other on-line editors in your telnet shell account, that is not very wise idea.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
I've had several emails about how people can't get this to work.

If you have the origional altavista mod installed and working, here is all you have to do...

under this in search.cgi:
Code:
($in{'search'} eq "theweb") and &altavista;

add this:

Code:
($in{'search'} eq "both") and &altavista;

In sub altavista, under this:

Code:
$next = &next_hits($page, $link_hits, 10)

add this:

Code:
if ($in{'search'} eq "both") {
$link_results2 = $link_results;
$link_hits2 = $link_hits;
$next2 = $next;
return;
}

In site_html_templates.pl add these under sub site_html_search_results:

Code:
link_hits2 => $link_hits2,
next2 => $next2,
link_results2 => $link_results2,

Now in your actual template add:
Code:
<%if link_hits2%>
<%link_hits2%>
<%endif%>
<%if next2%>
<%next2%>
<%endif%>
<%if link_results2%>
<%link_results2%>
<%endif%>

To add a drop down list etc, add this in your search.html template:

Code:
<select size="1" name="search">
<option value="theweb">The Web</option>
<option value="categories">Categories</option>
<option selected value="both">Both</option>
</select>


------------------
- Jonathan Gamble

http://www.magicdirectory.com/

[This message has been edited by jdgamble (edited March 19, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
I use notetab but I went and got edit plus 2.01a and try it again same problem,
Internal Server Error.

I'm going to find the other altavista mod and
try jdgamble's update.

I don't know that much about writting perl
but I am able to play with it and get things
to go right, But like I said before even in
edit plus 2.01a when you paste the code it paste about 3 inches in from the left.

when I leave it like that internal server
error, so I backspace it to where it should
be but still internal server error.

Eliot when you write perl does it matter
about spaces and line breaks etc. or does this effect the code.

well of to the drawing board
Regards,
Redneck

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
Okay here is what I have found out so far, when I telnet into
the server and run the script from there I get two errors

(1) syntax error near line 420 which is $link_hits2 = $link_hits;

(2) syntax error near line 426 which is } the last line of the first sub routine.

It executes the search.cgi great up until it hits this point. This is what's causing the Internal server error.

Any suggestions?

Regards,
Redneck

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
 
Quote:
Eliot when you write perl does it matter
about spaces and line breaks etc. or does this effect the code.

Yes...line breaks and spaces do matter.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
Well I don't know what you did jdgamble but the new codes worked like
a charm, and beleive it or not I didn't even have the other altavista
search mod working I got the same errors as before, but I figured
I had nothing else to loose so I tried it and
it Worked.

Thanks for your time and help and also for the great mod.

Thanks to you to Eliot for wisdom.

Regards,
Redneck

Now if I can only figure out how my case of beer got on the roof??? Hmmmm...

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
You're welcome.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
May I suggest when you have the time, to format the Mod into one section and post-it into the Resources section?

I would think many people look there first for modifications rather than search this forum.!

Best regards,

The Marshal
--------------


[This message has been edited by The Marshal (edited March 19, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
jdgamble:

I ran into a snag with your mod again, when
you search for different
sites it does what ever it wants. It is really hard to explain,
but I'll try.

If you search for say "free" using both it
comes up that it found
55 sites on the sites directory, but it doesn't show anything only
web matches, but if you searched for "canada" using both it does.

Further if you searched for "free" using the web selection it again
comes up it found 55 sites on the sites directory and doesn't
show anything for web matches.

If you want to see for yourself go to http://www.on-linecanada.com
This has really got me thinking, Hmmmm...

Regards,
Redneck

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
One problem I see with jdgamble codes is the following:

Code:
<option value="categories">Categories</option>

This should be:

Code:
<option value="">Categories</option>

With regards to your search results problem, RedNeck, I think that you may be confusing the placement of your tags. Could you please upload your search_results.html file in a publicly accessibly directory in your server? So, that we can see what codes and tags you are using.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
Eliot:

The search_results.html page is up, you can find it at:

http://www.on-linecanada.com/test/site_html.html

Thanks for looking into this.

Regards,
Redneck

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
One thing that I see missing from your codes is a conditonal tag for the main link_results and link_hits. These tags should look like the following:

Code:
<%if link_results%>
<%link_results%>
<%endif%>

Code:
<%if link_hits%>
<%link_hits%>
<%endif%>


And again, that option field I believe is wrong. Refer to my previous Reply.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
Eliot:

Thanks for pointing that out I do have those
tags but did not have
the <%endif%> part. (must have gotten deleted some how)

Also I think I found the problem, in search.cgi under
sub main {

I had: ($in{'search'} eq "altavista") and &altavista;
($in{'search'} eq "both") and &altavista;

which I copied from the altavista search mod
in another thread

I believe it should be ($in{'search'} eq "the web") and &altavista;

as jdgamble mentions above to be able to use his mod, I don't have
time to verify this but will look more into it later tonight.

Regards,
Redneck


------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
I forgot about the extra tags in site_html_templates, thanks. Frontpage added the "category" think automaticaly and I ignored it since extra tags are ignored.

I could be wrong, but I think Redneck's thing about canada and free is because your database doesn't have any results on it and it will try and go to site_html_failure page. You might concider adding the web code to site_html_failure.


------------------
- Jonathan Gamble

http://www.magicdirectory.com/




Quote Reply
Re: altavista search - new mod In reply to
I have been trying for days to get this to work - 500 error each time.

As others have done it successfully, i can only concur that the formatting is at fault here, although I’ve tried cut & paste with 4 different browsers all to no avail!

You can see my search.cgi here:
http://www.comtensolutions.com/test/search.txt

any ideas?

The Marshal
--------------
Quote Reply
Re: altavista search - new mod In reply to
Replace your sub altavista with this:

Code:
sub altavista {
# --------------------------------------------------------
# Search Altavista
my %in = &parse_form;
use LWP::Simple;
my $query = $in{'query'};
my $query_q = &urlencode($query);
my $page = ($in{'nh'} * 10) | | 0;
my $pageXX = $page - 10;
my $url = "http://www.altavista.com/cgi-bin/query?q=$query_q&text=yes&stq=$pageXX";
my $results = get ($url);
$results or &site_html_search_failure ({ error => "No Results Available.", %in }) and exit;
# If we see this, then we couldn't get a match.
my $none = quotemeta ("AltaVista found no document matching your query.");
if ($results =~ m/$none/) {
&site_html_search_failure ({ error => "No Matching Links", %in }) and exit;
}
else {
if ($results =~ /([^>\s]+) pages found./) { $link_hits = $1; }
else { &site_html_search_failure ({ error => "Can't Parse Results", %in }) and exit;
}
my $FONT = 'font face="verdana, arial" size="2"';
my $FONT_SMALL = 'font face="verdana, arial" size="1"';
if ($link_hits) {
while ($results =~ m#<dl><dt>\n<b>(\d+)\.\s*</b>\n<a href="([^"]+)">\n<b>([^<]+)</b></a></dt>\n<dd>([^<]*)</dd>\n<br><b>URL:</b> <font color="\#808080">([^<]+)</font>\n<br><font color="\#808080">\n([^<]+\n[^<]+)</font>#sog) {
($count, $url, $title, $description, $url2, $last) = ($1, $2, $3, $4, $5, $6);$url2 =~ s,/$,,;
$last =~ s,\n,,;$last =~ s,\s\s+, ,;
$link_results .= qq~<p>\n<$FONT_SMALL><b>$count.</b></font> <$FONT><a href="$url">$title</a><br>\n$description</font><br>\n<$FONT_SMALL><b>URL:</b><font color="#808080">http://$url2</font></font>\n~;
}
}
}
$next = &next_hits($page, $link_hits, 10);
if ($in{'search'} eq "both") {
$link_results2 = $link_results;
$link_hits2 = $link_hits;
$next2 = $next;
return;
}
&site_html_search_results;
exit;
}

And remember to remove the space between | |.

------------------
- Jonathan Gamble

www.magicdirectory.com/






[This message has been edited by jdgamble (edited April 12, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Hi,

Sounds like a great mod. If I understand it right, the mod will give you altavista results in your site ? besides links ?

Could this complete and revised mod be posted "together" in one post ? Please Smile

I read everything and couldn't help to be confused, thank you in advance,

Pat//
Quote Reply
Re: altavista search - new mod In reply to
Well the changes I made worked, to a point.
As you have already stated jdgamble that is what is exactly happening now.

If the search returns no matches on our site using both then it
returns a no matches found.

I understand your comment about adding the tags to the search_error.html
but I don't see how this will work properly as when some one finds a match
using just "the web" they would be taken to the search error page
which is set-up to say something like No sites matched your search.
but then it would list all the matches for just the web, if you
follow that.

It wouldn't show matches if some one didn't get a match by using
"categories" because of the tag <%if link_hits2%> or the tag
<%if link_results2%> is there a way to set it up using "if else"

This way if it went to the error page it would first look to
see if "both" was used and display the matches, or "else" display
the statement no matches found. Am I right about this and if so
how can this be done. If I am not please correct me. Any ideas??
Maybe Eliot can help with this.

jdgamble, don't get me wrong this is by far one of the best mods I
have seen as it adds lots of content to a site and makes any site
with your mod that much more attractive to users, there has to be
a way for this to work the way you planned it would, Also there
seems to be a problem in the way you list your code as I have
stated before and now the Marshal, I use IE and netscape and I can
cut and paste other peoples codes just fine
yet when I cut and paste
yours I get errors, is there a way you can copy your codes to a
plain text file and post it on a public page at your site for others.

Regards,
Redneck

------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
sorry the above should be using the "both" selection and not
"the web" selection

ie.
I understand your comment about adding the tags to the search_error.html
but I don't see how this will work properly as when some one does
not find a match on the sites end using "both" they would
be taken to the search error page which is set-up to say something
like No sites matched your search but then it would list all the
matches for just the web, if you follow that.


------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
Patricio,

Look at the examples jgamble provided.

RedNeck,

I still don't understand why it is not working for you.

BTW: I have my search templates structured very differently. But it still works for me.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
There still is a few bugs in it that I have found. I think this is what you are talking about. For example, if you search for something that can't be found in your database, but altavista can find it, then it goes biserk and has problems. I will get this straightened out, and when I do, I will post it here.

------------------
- Jonathan Gamble

http://www.magicdirectory.com/




Quote Reply
Re: altavista search - new mod In reply to
That problem is solved if you go with your original Modification, jdgamble. If you have two subs..."both" and "altavista"...the second will be referenced when there are no search terms found. This is what I have and it works just fine.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
I was talking about that if you search 'both' and you don't have any results in your links database, rather there is altavista results or not, you don't get any results and you get site_html_failure.

But your right and thats really not a big deal.

------------------
- Jonathan Gamble

http://www.magicdirectory.com/

[This message has been edited by jdgamble (edited March 27, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
I have switched to the other mod till you get this sorted out
jdgamble, I can't wait till it works.

Eliot:
Yep that is exactly what is happening, like jdgamble said if you
search using say "the directory" (local) works fine as well
if you search using "altavista" it works fine.

But if you search using "both" and if you don't have any local
matches it won't return altavista matches it throws up the search
error page.

Any Ideas Eliot.

Regards,
Redneck



------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
Yea...Redneck....

Change the following codes:

Code:
&site_html_search_failure ("no matching records") and return;

with the following:

Code:
&altavista () and return;

Regards,


------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
Quote Reply
Re: altavista search - new mod In reply to
The only problem I can see with that is that when you run the altavista sub routine... again, it will keep returning because of the "search=both".

------------------
- Jonathan Gamble

http://www.magicdirectory.com/




Quote Reply
Re: altavista search - new mod In reply to
Not for me. Works just fine. And to prove my point...go here:

http://vlib.anthrotech.com/...l=or&format=full

You will see that if there is absolutely NO matching words between my directory or altavista...the Search Failed page comes up.

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums


[This message has been edited by AnthroRules (edited March 30, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Eliot:

I just installed the mod again with the info you gave, same problem.

I've put my search.cgi script on the site if you want to have a look
http://www.on-linecanada.com/search.html

try searching for "marcus" you'll see that my site returns no matches
but neither does altavista, if you go to altavista it does return
matches for "marcus"

Regards,
Redneck


------------------
On-line Canada.com
http://www.on-linecanada.com
Internet Directory & Website Resource Centre
Quote Reply
Re: altavista search - new mod In reply to
 
Code:
# Return unless we have results.
((keys %link_results > 0) or ($#category_results >= 0)) or
&site_html_search_failure ("no matching records") and return;

See the line that I told you change...

Code:
&site_html_search_failure ("no matching records") and return;

This needs to be changed to:

Code:
&altavista () and return;

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
--------------------------------------------------------------------------------
Quote:
Not for me. Works just fine. And to prove my point...go here:
http://vlib.anthrotech.com/bin/search.cgi?query=asdfasrwer&mode=both&bool=or&format=full

You will see that if there is absolutely NO matching words between my directory or altavista...the Search Failed page comes up.

Maybe I made some confusion. Sorry. I was trying to agree with you.


------------------
- Jonathan Gamble

www.magicdirectory.com/
Quote Reply
Re: altavista search - new mod In reply to
 
Quote:
The only problem I can see with that is that when you run the altavista sub routine... again, it will keep returning because of the "search=both".

This statement does not seem to be agreeing, but contesting (disagreeing) with my statements.

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
Sorry, I didn't mean returning, I meant the site_html_search_failure page.

------------------
- Jonathan Gamble

www.magicdirectory.com/
Quote Reply
Re: altavista search - new mod In reply to
What about the this page? If you put the codes as I suggested for RedNeck the Mod will work fine....but I think, again, our channels crossed.

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
I was talking about something different anyway. Oh well. It doesn't matter.

Smile



------------------
- Jonathan Gamble

www.magicdirectory.com/
Quote Reply
Re: altavista search - new mod In reply to
What a great mod!

I keep getting two syntax errors at the bottom of the altavista routine though. The problems pop up on the 2nd and 7th lines of the code below:

Code:
$next = &next_hits($page, $link_hits, 10)
if ($in{'search'} eq "both") {
$link_results2 = $link_results;
$link_hits2 = $link_hits;
$next2 = $next;
return;
}

In both cases, my checker indicates the errors are near the curly brackets at the end of the lines. If I leave the code in, I can't get through to the search page. When I take it out, I am able to search, but only through my own database OR Altavista -- not both. (Setting my search form to "Both" gives me Altavista-only results.)

Any suggestions for troubleshooting? I've poured over the code several times and can't seem to see the problem.

Thanks!
Quote Reply
Re: altavista search - new mod In reply to
In regards to my previous post, changing...

Code:
$next = &next_hits($page, $link_hits, 10)

...to (note the semi-colon at the end)...

Code:
$next = &next_hits($page, $link_hits, 10);

...made all the difference in the world!

Thanks, Jonathan, for the great mod.
Quote Reply
Re: altavista search - new mod In reply to
is there a finished version of this mod in the resource section, I havent seen it..

Quote Reply
Re: altavista search - new mod In reply to
I don't believe so...jgamble, I believe, is still working on fixing some of the bugs.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
It is finished. Look at the first two posts at the top of this message. The only bug in it is that if you select search=both, and you don't have any links in your database, then you will get the site_html_search_failure page and you won't get any altavista results. But this can be fixed easily by editing your sub routine for the search failure page. Example:

Code:
There are no links in my database so here are altavista's results:

<%link_results2%>

and by adding the

Code:
link_results2 = $link_results2;

in site_html_templates.


------------------
- Jonathan Gamble

www.magicdirectory.com


Quote Reply
Re: altavista search - new mod In reply to
 
Quote:
It is finished. Look at the first two posts at the top of this message...

Then...Have you submitted it to the Resource Center?

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
I knew I was forgeting something...

I just submitted it unless someone beat me...

------------------
- Jonathan Gamble

www.magicdirectory.com


Quote Reply
Re: altavista search - new mod In reply to
Actually, I think there's another bug in the code as written above. I'll try and explain it the best I can.

In my search_results template, I have three separate areas defined by conditional statements:

Code:
<%if category_results%>
...
<%endif%>

<%if link_results%>
...
<%endif%>

<%if link_results2%>
...
<%endif%>

Within each of those statements, I have my appropriate link results, link hits, and next hits toolbars. I ALSO have a text string that identifies the area (i.e. Category Matches, Site Matches, Web Page Matches -- like Yahoo).

Everything works great when I run a search in my own categories, or both the categories and the Web.

BUT... when I run a Web-only search, I get my AltaVista results back within the <%if link_results%> statement and NOT the <%if link_results2%> statement. Which means the text string that pops up says "Site Matches" rather than "Web Page Matches."

I noticed at that Magic Directory Jonathan linked to in his first post the problem doesn't occur. Anyone have an idea where the best place to look would be? The solution to all this is probably staring me right in the face, but code-crunching has me a little weary.

Thanks!

[This message has been edited by troyw (edited April 10, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Well, its not my code that has the bug, its your code. Simple. In your case make the following changes.

Change:
Code:
sub altavista {
# --------------------------------------------------------
# Search Altavista
my %in = &parse_form;
use LWP::Simple;
my $query = $in{'query'};
my $query_q = &urlencode($query);
my $page = ($in{'nh'} * 10) | | 0;
my $pageXX = $page - 10;
my $url = "http://www.altavista.com/cgi-bin/query?q=$query_q&text=yes&stq=$pageXX";
my $results = get ($url);
$results or &site_html_search_failure ({ error => "No Results Available.", %in }) and return;
# If we see this, then we couldn't get a match.
my $none = quotemeta ("AltaVista found no document matching your query.");
if ($results =~ m/$none/) {
&site_html_search_failure ({ error => "No Matching Links", %in }) and return;
}
else {
if ($results =~ /([^>\s]+) pages found./) { $link_hits = $1; }
else { &site_html_search_failure ({ error => "Can't Parse Results", %in }) and return;
}
my $FONT = 'font face="verdana, arial" size="2"';
my $FONT_SMALL = 'font face="verdana, arial" size="1"';
if ($link_hits) {
while ($results =~ m#<dl><dt>\n<b>(\d+)\.\s*</b>\n<a href="([^"]+)">\n<b>([^<]+)</b></a></dt>\n<dd>([^<]*)</dd>\n<br><b>URL:</b> <font color="\#808080">([^<]+)</font>\n<br><font color="\#808080">\n([^<]+\n[^<]+)</font>#sog) {
($count, $url, $title, $description, $url2, $last) = ($1, $2, $3, $4, $5, $6);$url2 =~ s,/$,,;
$last =~ s,\n,,;$last =~ s,\s\s+, ,;
$link_results .= qq~<p>\n<$FONT_SMALL><b>$count.</b></font> <$FONT><a href="$url">$title</a><br>\n$description</font><br>\n<$FONT_SMALL><b>URL:</b> <font color="#808080">http://$url2</font></font>\n~;
}
}
}
$next = &next_hits($page, $link_hits, 10);
if ($in{'search'} eq "both") {
$link_results2 = $link_results;
$link_hits2 = $link_hits;
$next2 = $next;
return;
}
&site_html_search_results;
exit;
}

to...

Code:
sub altavista {
# --------------------------------------------------------
# Search Altavista
my %in = &parse_form;
use LWP::Simple;
my $query = $in{'query'};
my $query_q = &urlencode($query);
my $page = ($in{'nh'} * 10) | | 0;
my $pageXX = $page - 10;
my $url = "http://www.altavista.com/cgi-bin/query?q=$query_q&text=yes&stq=$pageXX";
my $results = get ($url);
$results or &site_html_search_failure ({ error => "No Results Available.", %in }) and return;
# If we see this, then we couldn't get a match.
my $none = quotemeta ("AltaVista found no document matching your query.");
if ($results =~ m/$none/) {
&site_html_search_failure ({ error => "No Matching Links", %in }) and return;
}
else {
if ($results =~ /([^>\s]+) pages found./) { $link_hits2 = $1; }
else { &site_html_search_failure ({ error => "Can't Parse Results", %in }) and return;
}
my $FONT = 'font face="verdana, arial" size="2"';
my $FONT_SMALL = 'font face="verdana, arial" size="1"';
if ($link_hits2) {
while ($results =~ m#<dl><dt>\n<b>(\d+)\.\s*</b>\n<a href="([^"]+)">\n<b>([^<]+)</b></a></dt>\n<dd>([^<]*)</dd>\n<br><b>URL:</b> <font color="\#808080">([^<]+)</font>\n<br><font color="\#808080">\n([^<]+\n[^<]+)</font>#sog) {
($count, $url, $title, $description, $url2, $last) = ($1, $2, $3, $4, $5, $6);$url2 =~ s,/$,,;
$last =~ s,\n,,;$last =~ s,\s\s+, ,;
$link_results2 .= qq~<p>\n<$FONT_SMALL><b>$count.</b></font> <$FONT><a href="$url">$title</a><br>\n$description</font><br>\n<$FONT_SMALL><b>URL:</b> <font color="#808080">http://$url2</font></font>\n~;
}
}
}
$next2 = &next_hits($page, $link_hits2, 10);
if ($in{'search'} eq "both") {
return;
}
&site_html_search_results;
exit;
}

or something of the sort...



------------------
- Jonathan Gamble

www.magicdirectory.com

[This message has been edited by jdgamble (edited April 10, 2000).]

[This message has been edited by jdgamble (edited April 12, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Bingo!

Thanks, Jonathan, for the great mod and the suggested code change.
Quote Reply
Re: altavista search - new mod In reply to
I am having some problems with this mod.. if anyone wants to have a look at it its at
internetevim.virtualave.net/links2/pages/ try searching for anything all you get is 500 server error.

you can see my site_html_templates.cgi at internetevim.virtualave.net/links/site_html_templates.txt and the search.cgi at internetevim.virtualave.net/links/search.txt

really need some help, thanks.
Ugur
Quote Reply
Re: altavista search - new mod In reply to
Hey, Jonathan--

Excellent mod! I'm really excited about the prospects of using this. I just have two quick questions for you (or anyone else who's working with it):

1.) Why does the next hits toolbar return only the first 19 pages of AltaVista results (even if you have something like 7 million matches)?

2.) I applied the change you suggested to troyw up above, and now I'm getting some weirdness if I try to run a search on the Web or Both and I don't have anything typed into the Search field. My search error layout gets repeated twice on the same page, the top one saying "There was a problem with your search: HASH(0x102039b0)" and the bottom one saying what it should ("There was a problem with your search: Please enter one or more keywords.") This doesn't occur when I run a search in only my categories and leave the field blank (in which case I get just the "Please enter one or more keywords" statement). Any thoughts on where I might be triggering the dual error?

At any rate, great job! This is really cool!!

Thanks,

blane
Quote Reply
Re: altavista search - new mod In reply to
Hey blane,

I do not really understand your first problem. Did you trie to click on for example link 18 ? Then you automaticaly will see the next 20 on next hits toolbar.
If you have 2.000.000 links and want to see the toolbar display all (10 links per page) you will get 200.000 links on your page Smile little too much i think Wink

When your second error occurs - are you online ? I always get the same when im not online - how can the script search the web if you are not online Smile
Try it when you are in the internet.
cya,

nikolai
Quote Reply
Re: altavista search - new mod In reply to
DigitalFusion,

The problem I'm having with the next hits toolbar is the very same one that's apparent on those Magic Directory pages linked way up at the top of this topic. Click on page 19 of the AltaVista area and that's the end of the line...

As for the second issue involving the stacked error messages, yes I'm online. I have a direct connection. The problem just started occuring when I applied those code changes suggested by Jonathan a few posts up.

Thanks for the input! Always appreciated.

blane
Quote Reply
Re: altavista search - new mod In reply to
I think I fixed the 19 page think. In sub next_hits put # sign on this line:

Code:
# last if ($i > 19);

http://www.magicdirectory.com/...mp;bool=or&nh=19

Quote:
2.) I applied the change you suggested to troyw up above, and now I'm getting some weirdness if I try to run a search on the Web or Both and I don't have anything typed into the Search field. My search error layout gets repeated twice on the same page, the top one saying "There was a problem with your search: HASH(0x102039b0)" and the bottom one saying what it should ("There was a problem with your search: Please enter one or more keywords.") This doesn't occur when I run a search in only my categories and leave the field blank (in which case I get just the "Please enter one or more keywords" statement). Any thoughts on where I might be triggering the dual error?

The change was only meant for troyw's case. I suggest you copy the origional altavista from the top of this message and change the first three return; to exit;.


------------------
- Jonathan Gamble

www.magicdirectory.com


Quote Reply
Re: altavista search - new mod In reply to
Jonathan:

I'm actually experiencing the same thing as blane in regards to dual error messages. (I never thought to run a search for nothing until he mentioned it...) It's something about that new code. I'll post if I can run the bug down.

Also, I noticed when I took a look at
http://www.magicdirectory.com/...mp;bool=or&nh=19
you have a cosmetic issue that I had when I first got everything up and running. At the very top there's an orphaned next hits toolbar for the Sites section just floating up there all by itself. I fixed this by making sure the if statement for the hits toolbar was INSIDE the link_results if statement.

Take care,

Troy
Quote Reply
Re: altavista search - new mod In reply to
Looks like Jonathan is editing his script since I am getting an Internal Server Error.

I tried looking at the examples provided to get a feel for what people are talking about, and I got the Internal Server Error.

Wink

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
Hey all,

I tracked down why I was getting those dual error messages. It had to do with some code I changed to fix an earlier problem.

I'd suggest to Blane that he fly through and double-check everything, especially the first couple lines of the "main" subroutine in search.cgi. That's where I had made a change that I forgot about. If all else fails, reapply Jonathan's code from scratch.

Take care,

Troy
Quote Reply
Re: altavista search - new mod In reply to
Yeah Eliot, I was working on a different mod for a different post. I usually make a new search file but I didn't have the time.

Quote:
troyw
I fixed this by making sure the if statement for the hits toolbar was INSIDE the link_results if statement.

Can you post the code here or the exact changes to make... I'm trying to do 10 different things right now and it would really help me out.

Also, as for the 19 thing, on the 19th page it replaces '>>' with '...' for some reason, any ideas?

Thanks,

------------------
- Jonathan Gamble

www.magicdirectory.com


Quote Reply
Re: altavista search - new mod In reply to
Jonathan,

Here's the code for the link_results section of my search_results.html file. As I mentioned before, I just made sure the second <if next> statement was within the overall <if link_results> statement. You'll have to customize, of course, if you're within a table...

Code:
<%if link_results%>
Site Matches <b>(<%link_hits%> found)</b></p>
<%if next%>
<p>Pages: <%next%></p>
<%endif%>
<ul>
<%link_results%>
</ul>
<%if next%>
<p>Pages: <%next%></p><%endif%>
<%endif%>

I'll take a look at that "..." issue tonight.

[This message has been edited by troyw (edited April 11, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Thanks, I thought you were talking about if statements in the actual cgi-file. I had my first next hits in the if statement, but my second next hits were not. It is fixed now.

Right now I'm looking into the 19th thing, it may not be a problem, but I need to see.

------------------
- Jonathan Gamble

www.magicdirectory.com


Quote Reply
Re: altavista search - new mod In reply to
if you guys have sometime I would appreciate if you could also have a look at my problem..

thanks
ugur
Quote Reply
Re: altavista search - new mod In reply to
UGUR--

In the altavista subroutine in search.cgi, try changing...

Code:
$next = &next_hits($page, $link_hits, 10)

...to this:

Code:
$next = &next_hits($page, $link_hits, 10);

Note the semi-colon at the end of the line...

JONATHAN--

"19" is mentioned twice down there in the code. You commented out the first occurence, but I think it's that second one that's giving you the "..." Haven't had time to play with some fixes, but will tomorrow.

BTW, how did you handle returning a "Please enter one or more characters" error when searching "the web" or "both" with a blank search field? I'm getting a pesky Hash error. I get the correct error ONLY when I have it set to "categories."

Also, you might want to change the code up at the top of this page to address my above comment to Ugur.

Take care,

Troy

[This message has been edited by troyw (edited April 12, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Troy,

thanks for the reply, I have changed what you advised, but I beleive there is still one or more bracket or semi-colon problem at my search.cgi. cause when I try to search anything it just gives me 500 server stuff, doesnt bring any page at all.

here is my updated pages at.

thanks for your time

my main page
internetevim.virtualave.net/links2/pages/
template.pl
internetevim.virtualave.net/links/site_html_templates.txt
search.cgi
internetevim.virtualave.net/links/search.txt
Quote Reply
Re: altavista search - new mod In reply to
Ok.... here we go...

To fix the 19 thing again... in search.cgi:

Change the sub next hits thing to:
Code:
$output .= qq~<a href="$ENV{'SCRIPT_NAME'}?$next_url&nh=$next_hit">[>>]</a> ~ unless ($nh == $i);

For the error for no search terms, add the following line to sub altavista:

Code:
if (!$in{'query'}) { return ("Please enter one or more keywords."); }

Thanks for pointing that out to me...


------------------
- Jonathan Gamble

www.magicdirectory.com


Quote Reply
Re: altavista search - new mod In reply to
Might I suggest that people create another Topic and link to this Topic...This Topic is getting hard to follow and it is 125KB large.

Just a suggestion.

Regards,

------------------
Eliot Lee....
Former Handle: Eliot
* Check Resource Center
* Search Forums
* Thinking out of the box (codes) is not only fun, but effective.
Quote Reply
Re: altavista search - new mod In reply to
Ugur

Sorry I didn't reply earlier, I missed your message.

I took a look at your search file, and let me say it is VERY unorganized and hard to find information. I copyied it and ran it on my server still having problems.

You entire next hits sub routine is in the middle of the sub search. And also some of the next hits search from the top is gone. I suggest you reinstall every mod that you did for search.cgi. If you tell me what mods you have on there (obviously the non-engish mod), I will do it for you.

PAGE2!!!
http://www.gossamer-threads.com/...um3/HTML/005757.html

------------------
- Jonathan Gamble

www.magicdirectory.com

[This message has been edited by jdgamble (edited April 12, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
Hi
i was wondering if this mod work with other search engine such as allmp3.com and 2look4.com or maybe yahoo.com

Thanx
Quote Reply
Re: altavista search - new mod In reply to
I am having problems running, this Links 2.0 mod. When I run it I get a 500 error.
I have tryed to install this mod a few times.

You can see my search.cgi file by gooing to http://hiphopbot.com/search.txt. You
can see my site_html_templates.pl file by gooing to http://hiphopbot.com/site_html_templates.txt

Can anyone tell me what I am dooing wrong?

------------------
Thank You,
James Hardy
HipHopBot Co.
support@hiphopbot.com
http://hiphopbot.com

[This message has been edited by wesmix (edited May 07, 2000).]

[This message has been edited by wesmix (edited May 07, 2000).]
Quote Reply
Re: altavista search - new mod In reply to
First of all... DON'T POST ANY MORE POSTS ON THIS MESSAGE...

use this message for page 2!

http://www.gossamer-threads.com/scripts/forum/resources/Forum3/HTML/005757.html


and add a ';' to the end of this line
Code:
$next = &next_hits($page, $link_hits, 10)



------------------
- Jonathan Gamble

www.magicdirectory.com