Gossamer Forum
Home : Products : Gossamer Links : Development, Plugins and Globals :

[Development] Spell Correction on Query

Quote Reply
[Development] Spell Correction on Query
Here is a development idea (sorry this is the ole "Sooke" coming out in me again).
  1. You have a custom list of commonly mis-spelled words: badspelling.txt
  2. You trap the query and if no results or return on a search you try the search again if a like match is found in the badspelling.txt list.
  3. Optionally you trap the query and before the search occurs you automatically correct the word, like our friends do at google.


It actually sounds like a pretty easy plug-in doesnt it? Hmmmmm

Comments?Wink


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [Ian] [Development] Spell Correction on Query In reply to
Google don't actually correct the word automatically, they just give an example of what they think you may have meant to say....good idea for a plugin though Tongue

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] [Development] Spell Correction on Query In reply to
If you tell me how google does it, it will make a plugin for this.

Ivan
-----
Iyengar Yoga Resources / GT Plugins
Quote Reply
Re: [yogi] [Development] Spell Correction on Query In reply to
Hi Guys.

I am glad you both liked my suggestion.

Ivan... I am SOOOOO glad you volunteered (or are at least considering) writing this one. I feel "PLUG-IN SNOWED" right now!!

I am still burried deep in



Thanks for the correction on Google Andy. I think the type of thing I am suggestion is BETTER than what google does anyways... at least for my purposes.Smile

EDIT: I think point 2 above is the one to go with, unless you really want point 3. This way it all happens behind the scenes to the user. The first time they see anything after searching for a mis spelt world is a page full of results, with a a little note saying "casr" was corrected to "cars".


http://www.iuni.com/...tware/web/index.html
Links Plugins

Last edited by:

Ian: May 31, 2002, 9:08 AM
Quote Reply
Re: [Ian] [Development] Spell Correction on Query In reply to
I thought I formulated my post in such a way that I will certainly not have to write this plugin. Wink

Because: what google does seems to be quite advanced, and I don't think it's generally known how they do it. Correct me if I'm wrong...

So tell me what google does, and I'll volunteer...

Ivan
-----
Iyengar Yoga Resources / GT Plugins
Quote Reply
Re: [yogi] [Development] Spell Correction on Query In reply to
Thats ok IvanWink I will try if you don't to... but you would just be helping me anyways!!! (just kidding).

Google example:

You type in jackte (mis spelling of jacket)... give it a try.

It comes up with:

Did you mean: jacket

The correctly spelt word is hyperlinked so you could click on it and search again.


http://www.iuni.com/...tware/web/index.html
Links Plugins

Last edited by:

Ian: May 31, 2002, 1:29 PM
Quote Reply
Re: [Ian] [Development] Spell Correction on Query In reply to
I know what they do on google, I just don't know how they do it..... and it's not so easy I think. If you have any suggestions, tell me.

Maybe you could use the spelling checker (GForum) for this....

Ivan
-----
Iyengar Yoga Resources / GT Plugins
Quote Reply
Re: [yogi] [Development] Spell Correction on Query In reply to
Ahhh, sorry, I misunderstood you.Blush

I don't think it has to be that complicated.
  1. Hook (POST) on search query, check if zero results returned, and if so then run the correction sub.
  2. Correction sub... load in a file containing a list of commony mis spelled words, and compare the $query with each item in the list.... if it finds a word which is only one letter switched then produce the suggestion


I guess the magic comes in "if it finds a word which is only one letter switched ". Being a complete begineer at perl, this would be hard for me to suggest how.

But the logic would be:
  • Load in one word at a time from the misspelledword.txt.
  • Loop through each pair of letters in the word (1st and 2nd, then 2nd and 3rd etc) and swap these two letters to form a temporary word $match. If $match == $query, then we have a match.




That is a pretty basic mispell checker, but most of the time it is just a swapped letter isnt it?

I guess to take it one step further would be to look for a missing letter (on either end) or a superfulous letter.

Doesnt mysql or perl of a decent %like% function inbuilt? sorry for my ignorance here.

Or, lastely, find out if there is a GT module we could use, like the spell check in GT forum.

EDIT: It seems a lot of people opt for spell.vbs (MS Word's spell checker). But I think a simple function like above could at least get a lot of the common mistakes.


http://www.iuni.com/...tware/web/index.html
Links Plugins

Last edited by:

Ian: May 31, 2002, 2:07 PM
Quote Reply
Re: [Ian] [Development] Spell Correction on Query In reply to
Here's a word list (in Access) if it helps... You could convert it to SQL and check the query against it. Just a thought.

Sean
Quote Reply
Re: [SeanP] [Development] Spell Correction on Query In reply to
Yes, that is a great idea..... sql has a "like" function.

Thanks for the list!


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [yogi] [Development] Spell Correction on Query In reply to
I have thought of another, possibly easier and faster way to solve this!

  1. Create another SQL table called: spelling




table ('spelling')

cols -> Word - this is the correct spelling for a word
BadWord - this contains a list of similar misspellings for the word

2. Do as I suggested above..... when a search returns no results, match $query with and of the BadWord items and search on the corresponding Word item!! If no match in spelling table, then return the normal nothing found code!

Easy??

I think this would work.... I think I could write this myself (with some help)... or you can write... let me know which.

Edit: I've started working on this (sorry, I have put my other plugins on hold out of frustration, so I thought this may be a little easier for now).... and i'll post back with what I did.

Smile


http://www.iuni.com/...tware/web/index.html
Links Plugins

Last edited by:

Ian: May 31, 2002, 4:33 PM
Quote Reply
Re: [yogi] [Development] Spell Correction on Query In reply to
Ok, I have simplified the concept of the table two just (word, badword) for now... and will change this to (word, badwords - with a list of misspelled words) later.

Any ideas on how I return the matching word into <%suggested_spelling%> so that if there is a match, then the search template can say:

"Suggested Spelling for $original_term is <i>$new_term</i>"

It would also be better if it found a match it just went ahead and did a search again... but how do you get Links to search again from within a hook sub??



sub search_results {
# -------------------------------------------------------------------
# This subroutine will get called whenever the hook 'search_results'
# is run. You should call GT::Plugins->action ( STOP ) if you don't
# want the regular code to run, otherwise the code will continue as
# normal.
#
my (@args) = @_;

my $opts = shift;
my $db = $DB->table ('Spelling');
my $query = $IN->param('query');
my $newquery = '';

$query =~ s/^\s*|\s*$//g;
$query = lc $query;
$query = substr ($query, 0, 24);

#Find matching correctly spelt word
$newquery = select ('Word',{Badword => $query} );


return @args;
}


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [Ian] [Development] Spell Correction on Query In reply to
Hi,

I know this is an old thread but does anyone know if it came to fruition as a paid insert or not?

Thank you

Cheers
KevM
Quote Reply
Re: [KevM] [Development] Spell Correction on Query In reply to
You can just register on google and use their suggestions. You only get a limited number of searches a day though.

This is the global I use:

sub {
my $query = shift;
use Net::Google;
use constant GOOGLE_LICENSE_KEY => ''; #You need to get a key and enter it here.
my $google = Net::Google->new( key => GOOGLE_LICENSE_KEY );
my $spelling = $google->spelling( phrase => $query );
my $result = $spelling->suggest;
return {Suggest=>$result};
}

Then you can call the global and use <%if Suggest%>Did you mean <%Suggest%>?<%endif%>
Quote Reply
Re: [afinlr] [Development] Spell Correction on Query In reply to
Thanks Afinlr,

once server indexing is sorted out I'll give this one a try.

kind rgds

Cheers
KevM
Quote Reply
Re: [yogi] [Development] Spell Correction on Query In reply to
I like the Google API idea. Just wanted to bring up this thread again and suggest this link I just stumbled over:

http://norvig.com/spell-correct.html

Regards

Niko