I'm getting returns on words like "the". What's happening?! Yikes, yikes, yikes.

I have this in search.cgi:

# Word is too common, don't try and sort it, can cause problems.
if (($numhits > 50) and (($grand_total * 0.75) < $numhits)) {
return "Search term is too common.";
}

But I just got 4727 returns (25 pages) when I searched on "the". Why am I not getting "Search term is too common."?

I've searched the forum, and everyone else seems to have the opposite problem--they get "Search term is too common." when they don't want to.

I was looking at search.cgi, and it seems that the above "Search term is too common." bit conflicts with this bit:

# Set maximum hits -- default to 25.
local $maxhits = 25;
if ($in{'mh'} && (($in{'mh'} == 10) || ($in{'mh'} == 25) || ($in{'mh'} == 50) || ($in{'mh'} = 100))) {
$maxhits = $in{'mh'};
}

Why would you need 100 pages of search results if you're limiting search words to 50 returns? What am I missing here?

Anyway, that's what I'm getting. On the form on my search.cgi page, if a user selects maxhits = 10, I get 10 pages of returns when I search on "the." AND (gulp), I get 100 pages of returns if maxhits = 100.

Someone, please, help.

Kathy

Once upon a time there was a mod called "Human Language" that would filter out certain words. The links to it are dead, and I do not have it saved on disc, but lo and behold, I have a hard copy in my files!


This mod is by Aaron C. Mast, who used to be at gamefinder.psybercounsellor dot com. I take no credit for this mod, but I hate to let it stay lost when it's needed. If this works OK, let me know, as I have not tried it myself.

Here's what to do...

In search.cgi, just below:

# Set maximum hits -- default to 25.
local $maxhits = 25;
if ($in{'mh'} && (($in{'mh'} == 10) || ($in{'mh'} == 25) || ($in{'mh'} == 50) || ($in{'mh'} = 100))) {
$maxhits = $in{'mh'};
}

Add this:

# Check to see if Human Language search was set.
$orgquery = "$in{'query'}";
if ($in{'humanlang'} eq "1") {$orgquery = "$in{'query'}";
# Perform the Human Language to search conversion
@normlang = split(//,$refnormlang);
foreach $excnormlang (@normlang)
{$in{'query'}=~s/\b$excnormlan\b/\/gi;}
}


Next, find:

# Print out the HTML results.
&site_html_search_results;
}

And add this line, like so:

# Print out the HTML results.
$in{'query'}="$orgquery";
&site_html_search_results;
}



Now, in links.cfg, add this somewhere in the file:

# Search Options
# -------------------
# Normal Human Language to exclude from search
$refnormlang = "you your with will the so and or";

Add as many words as you want to the list. To make this the default operation, add this to your search forms:

<input type="hidden" name="humanlang" value="1">

Or to make it an option:

Ignore common words: <input type="radio" name="humanlang" value="1" checked="checked"> Include common words:<input type="radio" name="humanlang" value="2">


Leonard
aka PerlFlunkie

This is just what I need! Now if I can only get it to work. I keep getting error messages. So, I tried fiddling with things. It didn't look quite right to me--though I have no idea what I'm doing, really--so I edited the code a bit, just based on what seems to be normal usage:

# Check to see if Human Language search was set.
$orgquery = $in{'query'};
if ($in{'humanlang'} eq "1") {$orgquery = $in{'query'};
# Perform the Human Language to search conversion
@normlang = split (//, $refnormlang);
foreach $excnormlang (@normlang)
{$in{'query'} =~ s/\b$excnormlan\b/\/gi;}
}

and:

$in{'query'} = $orgquery;

Unfortunately, I keep getting error messages. Do you see anything wrong that I don't? Mostly I just added spaces and took out commas. Did I mess things up even worse than before?

Kathy

What kind of error messages? I looked over the post, and I typed it in just like I have it on paper, so that's not the problem...

I'm an amatuer at this too, but you might try...

# Check to see if Human Language search was set.
if ($in{'humanlang'} eq "1") {
# Perform the Human Language to search conversion
@normlang = split(//,$refnormlang);
foreach $excnormlang (@normlang)
{$in{'query'}=~s/\b$excnormlan\b/\/gi;}
}

Then remove this:
$in{'query'}="$orgquery";


Leonard
aka PerlFlunkie

Nope. I tried that (and about a thousand variations) and it still doesn't work. Does anyone have this mod working? Want to share the secret?

Kathy

Here's another shot at it... Put the red under where you find the black in search.cgi, and remove/comment-out all the previous code for this mod.

# Set search type -- either phrase or keyword. Also build keyword list to search on.
my @search_terms = ();
($in{'type'} eq 'phrase') ?
(@search_terms = ($in{'query'})) :
(@search_terms = split (/\s/, $in{'query'}));
($in{'humanlang'} eq '1') ?
(($in{'query'}) = $orgquery) :
(($in{'query'}) = ($in{'query'}));

# Check to see if Human Language search was set.
if ($orgquery) {
$orgquery = "$in{'query'}";
# Perform the Human Language to search conversion
@normlang = split(//,$refnormlang);
foreach $excnormlang (@normlang) {
$in{'query'}=~s/\b$excnormlan\b/\/gi;
}
}


Leonard
aka PerlFlunkie

Nope. That didn't work either. I tried it a hundred different ways, editing it character by character and saving and uploading. Nothing worked. (BTW, I think it should be $in{'query'} =~ s/\b$excnormlang\b/\/gi; or maybe gie). I am really bummed.

How is it going with the exact word stuff? Did you get it to return exact words?

Kathy

Bummer... I know how frustrating it is when you can't get a piece of code to work.

The Exact Word thing still gives me results like 'testing' when I search for 'test'. That's not exact...


Leonard
aka PerlFlunkie