Gossamer Forum
Home : Products : Gossamer Links : Version 1.x :

Query + Field Searching - Please support bug fix

Quote Reply
Query + Field Searching - Please support bug fix
Hi All,

The Query Searching + other field searching is discussed in many places around this site (primarily in the context of category searching). There's mention of logic bugs in the search.pm query module. I have spent a considerable amount of time in trying to figure out where this particular bug is in hopes that I could fix it (it may or may not be related to other bugs in search.pm, that I don't know) but alas, it's too complex for me (see below for what I have discovered). Many threads say to wait for the new version of Links SQL, but for me (anybody else?), upgrading to that system is not be an option because of mods installed, and time constraints.

In any case, the query search is supposed to allow us to search for something like query=something&field=something. This type of search brings back only 1 hit instead of all the appropriate hits.

While I can't speak for others, my primary reason for upgrading to the SQL version was because of the search features, and I specifically asked about the above mentioned feature before purchasing, so I feel this bug fix should be supported.

If this bug can't be fixed, at least I would appreciate an explanation of why it can't be fixed. The error as you'll see below, is I hope, a relatively minor one:


In search.pm sub query there is this set of routines.
In Reply To:
my $sth = $self->{dbh}->prepare ( $query );
$sth->execute ();
my $row;
my %needed = map { $$_[0] => 1 } @query_results;
foreach $row ( $sth->fetchrow_hashref () ) {
delete $needed { $$row { $id_col } };
}
foreach $row ( keys %needed ) {
delete $$id_results { $row }; #This seems be deleting valid hits
}
#AS SOON AS THE FOLLOWING LINE IS COMPLETED THE DATA GETS LOST.
@query_results = sort { $$b[1] <=> $$a[1] } map ( [$_, $$id_results {$_} ], keys %$id_results );
This is the content of $query BEFORE that set of routines gets run.
SELECT count(*) FROM Links WHERE ID in (125,118,87) and (( CategoryID = "1"))

This is the content of $query AFTER that set of routines gets run.
SELECT ID FROM Links WHERE ID in (118) and (( CategoryID = "1"))

So, The query results before this point are fine, but once this set of routines gets run,
the query results get dropped from all the appropriate hits to just one
(something to do with delete $$id_results { $row }; - I think).
I temporarily commented out that line and my results were as expected but then I got this error:


Can't use an undefined value as an ARRAY reference at admin/Links/Search.pm line 935.

And now I'm way in over my head.

Can anyone take this from here? Or at least explain why this can't be fixed? I'll be happy to post whatever code I need to to get this up and working.

Thanks and peace.

Kyle




Quote Reply
Re: Query + Field Searching - Please support bug fix In reply to
This is exactly why I went with Links SQl, a modular program, so I could be shielded from all the low-level back-end logic.

Anything with two or more -> 's in an expression, or $$ inside {}, or trying to explain a Schwartzian Transform (sort a[1]<=>b[1]) is beyond any new trick this old dog can learn.

You have to enter the mindset of what is going on, and visualize those data structures. That's why I'm waiting for the next release, which I hope will have shielded me even further from this sort of ugly code.

This is a low-level logic/datastructures type of coding. This version didn't work, and it's quite possible, that while those lines you are pointing to are a problem, the problem is really higher up in the logic flow. That's just where they manifest.

Good luck :)


http://www.postcards.com
FAQ: http://www.postcards.com/FAQ/LinkSQL/

Quote Reply
Re: Query + Field Searching - Please support bug fix In reply to
Thanks for insight Pugdog, although it's not what I was hoping to hear, I share you sentiments.

I know you're really busy trying to answer these questions for everyone, and you do so in a very timely manner - thank you - but is there any else out there who does know this stuff and can fix it?

I am wondering if this problem will be looked at some later point, or if the next generation search functions will be able to integrate into this version of Links SQL.

My primary concern is that I won't be able to upgrade to the next generation, and this problem exists on every copy of Links SQL that I own (5 so far), and I'm not looking forward to the time it's going to take to upgrade them if that's even an option for me.


Also, one final note about this problem is this:
In trying to suss out this problem I cleared out my word_index and scores_index tables and let the Add and Modify functions put in the words and scores one at a time as opposed to re-indexing the entire table. What I discovered was that if I had links in one category only, this search feature would work fine, but once I added a link into a different category, this problem arises. - I found that very wierd Shocked. Maybe that will also be of use to someone who can fix this.

Peace.

Kyle

Quote Reply
Re: Query + Field Searching - Please support bug fix In reply to
This goes back to what I was saying. As I've probed the internals of the code, there are obsurce type of logic bugs that create problems when the routines are used as they are _defined_ rather than how they were used in the default configuration.

Once the next version of links comes out, and is much more stable, and more 'together' as a product, people will be able to find and point out fixes to these problems, and changes to the back-end engines can be implemented without affecting the higher level code.

This is one reason open source code will always blow away the commercial programs (or compiled distribution code). People _will_ hit a problem, try to figure it out, localize an area, and someone will get interested and try to make a fix. That may point out a cascade of other problems that result in whole modules being rewritten -- or just an 'and' being changed to an 'or' <G>

Code changes are dynamic, broad based, and often get hacked to perfection before they are finally made.

If it works for one category, but not two, then it sounds like a loop, counter, hash, reference, or other container or counter is being overwritten or reset as the search progresses. It would be the outer iterator, most likely, not the inner one.

http://www.postcards.com
FAQ: http://www.postcards.com/FAQ/LinkSQL/

Quote Reply
Re: Query + Field Searching - Please support bug fix In reply to
Hi Kyle,

Yes, it is a bug. Ugh, an ugly one too. Quick answer:

Replace in Search.pm around line 721:

Code:
foreach $row ( $sth->fetchrow_hashref () ) {
with:

Code:
while ( $row = $sth->fetchrow_hashref () ) {
and it works.

Longer answer:

In Reply To:
my %needed = map { $$_[0] => 1 } @query_results;
foreach $row ( $sth->fetchrow_hashref () ) {
delete $needed { $$row { $id_col } };
}
foreach $row ( keys %needed ) {
delete $$id_results { $row }; #This seems be deleting valid hits
}
What's going on here is id_results holds your current results. If we are doing a filter, and we found a match, we need to remove from id_results anything that doesn't match the filter.

So first we setup a hash of all the current results in %needed. Then we loop through our filter results, and remove anything we found from needed. Anything that's left in needed, didn't match our filter and needs to be removed from id_results.

The bug was that the

foreach $row ( $sth->fetchrow_hashref () ) {

only ever ran once obviously (well it is once you see what's going on). =)

Let me know if you have any further problems.

Cheers,

Alex

--
Gossamer Threads Inc.