Gossamer Forum: Products: DBMan: Customization: Still about merging fields

Jul 31, 2002, 4:02 AM

Fabianope

Novice (7 posts)

Jul 31, 2002, 4:02 AM

Post #1 of 11

Shortcut

Still about merging fields

hello to Everybody
I'm setting DbMAN for the e-journals of a University Library.
you can find the "beta" to:
http://ecolex.bib.uniud.it/...man/db.cgi?db=rieltp
now the problem.
From a "librarian standpoint" a journal like "the economist" really begins with "e" and not with "t".
in other words, "the" should be a stopword.
This is the problem, and I've found a solution through the forum for the correct display of the records:first I've created 2 separate fields for the stopwords and the title (in italian, TITOLO), then I've added the lines:

$total = 0;
$total = $rec{'Stopword'}." ".$rec{'Titolo'};

in the sub html_record of the html.pl file and, third, $total is displayed in the record as "Titolocomp" (=complete title).
So if you click the "E" link (="journals that begins with E in the field TITOLO") you 'll find also "The economist", and that's fine.

now I'd like to add the $total as a "permanent field" for searching in the sub html_record_form, but the solutions found in the unofficial faq and in the following threads:

"Merge, Concatenate, Combine or whatever - it doesn't work "
"Adding Fields Together"
"Ignoring stopwords in search results"

doesn't work for me, so now my DbMAN has the bug of finding ONLY "economist" but not "the economist"...sigh!!! Pirate

please can you help?
Thnanks in advance,

Fabiano

Jul 31, 2002, 7:45 AM

alon_g

User (51 posts)

Jul 31, 2002, 7:45 AM

Post #2 of 11

Shortcut

Re: [Fabianope] Still about merging fields In reply to

Hey Fabiano, I'm pretty sure I can help you but I'm not sure I understood the problem due to my not-so-great English (and my very bad Italian :-)..

For each record in the database you have 2 fields for a name?
For a example, for a record called "The Economist",
- You Have the name alone which is "Economist",
- And the words before that which is "The".

With the search-by-letter, you use the short title field, whice has no 'the's in it.
Now - In the normal search you want to search "The Economist"?

I just had solved a problem similar to this. You need to create another empty field in your database (and your CFG file) for all the records (just add another | at the end of every record), let's call this 'TotalTitolo' - sounds cool too. In your search form, name the field that you want for the title search TotalTitolo.
Now, in your db.cgi, below this code:

Code:
# Normal searches.		 
		$key_match = 0; 
		foreach $field (@search_fields) {

add this code (with the right numbers as they appear in the CFG file)

Code:
$values[TotalTitoloNUMBER] = $values[StopWordNUMBER] . " " . $values[TitoloNUMBER];

This should work - although I'm not sure this is the most efficient solution for this. There is still a problem with the bolding of the title in the search results - I'm trying to solve it too..

Hope it helps, Alon Gal.

Jul 31, 2002, 9:11 AM

LoisC

Veteran / Moderator (3034 posts)

Jul 31, 2002, 9:11 AM

Post #3 of 11

Shortcut

Re: [Fabianope] Still about merging fields In reply to

Have you by any chance viewed this record in the FAQ?

HTML list with the content of a field (sub build_select_field_from_words)

original thread:

Subject: list of field
Stephane - 10-Sep-99
Thread: http://gossamer-threads.com/p/000954

The url may be outdated but you could search by the subject or title to find original thread. The FAQ has a condensed version of that thread.

Unoffical DBMan FAQ

http://creativecomputingweb.com/dbman/index.shtml/

Jul 31, 2002, 11:37 AM

sigrid

User (107 posts)

Jul 31, 2002, 11:37 AM

Post #4 of 11

Shortcut

Re: [Fabianope] Still about merging fields In reply to

This has been bothering me for a while. I am also a librarian and use dbman for the exact same purpose as you. Making two fields, a long title and a short title would be way too much work and make the database much larger than needed.

Here is a script you can install independently which will solve the problem. I am working on a way to integrate it into the html.pl file, but until then, this seems to work okay. Go to

http://www.lib.lsu.edu/...rid/journalfind2.cgi?

and try The Economist and Economist in the top input box to try it out. The sorting isn't right- it still falls under The, but at least it shows up.

Here's the script. I am going to keep working on it and will post more if I can come up with an improvement, but maybe this will get you started, or give you an idea for something more useful to share.

Code:


#! /usr/sbin/perl    

############ Advanced Options   
print "Content-type: text/html\n\n";    

#############################    

use LWP::Simple;   
&Parse_Form;   
$search = $formdata{'search'};    

$search =~ s/the //g;   
$search =~ s/the//g;   
$search =~ s/ /+/g;    

print qq|    

<html>   
<head>   
<title>$title</title>|;    

print qq|   
</head>   
<body bgcolor="#ffffff">   
      <form method=GET name="searchform">   
      <input name="search" value="$search" type=TEXT size=15 maxlength=80>   
      &nbsp;    
      <input type="submit" value="submit" name="submit">   
      </form>   

|;    


$url =   
"/cgi-bin/dbman/db.cgi?db=default&uid=default&JournalName=$search&view_records=1";   

$html = get $url;    

print qq| $html |;    

print qq|</body></html>|;    

sub Parse_Form {   
 if ($ENV{'REQUEST_METHOD'} eq 'GET') {   
  @pairs = split(/&/, $ENV{'QUERY_STRING'});   
 } elsif ($ENV{'REQUEST_METHOD'} eq 'POST') {   
  read (STDIN, $buffer, $ENV{'CONTENT_LENGTH'});   
  @pairs = split(/&/, $buffer);   

  if ($ENV{'QUERY_STRING'}) {   
   @getpairs =split(/&/, $ENV{'QUERY_STRING'});   
   push(@pairs,@getpairs);   
   }   
 } else {   
  print "Content-type: text/html\n\n";   
  print "<P>Use Post or Get";   
 }    

 foreach $pair (@pairs) {   
  ($key, $value) = split (/=/, $pair);   
  $key =~ tr/+/ /;   
  $key =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;   
  $value =~ tr/+/ /;   
  $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;   

  $value =~s/<!--(.|\n)*-->//g;   

  if ($formdata{$key}) {   
   $formdata{$key} .= ", $value";   
  } else {   
   $formdata{$key} = $value;   
  }   
 }   
}    
1;

Last edited by:

sigrid: Jul 31, 2002, 11:50 AM

Jul 31, 2002, 2:24 PM

sigrid

User (107 posts)

Jul 31, 2002, 2:24 PM

Post #5 of 11

Shortcut

Re: [Fabianope] Still about merging fields In reply to

I think I actually figured something out. I'm not must of a programmer, so if anyone reads this and sees a problem, please let me know.

In db.cgi, in sub parse_form, after

$value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;

add:

$value =~ s/^the //g;
$value =~ s/^the//g;

I think this should take care of titles beginning with "The" but not affect titles with the in the middle.

In my database, I can search all these successfully now:

The Economist

Economist

Journal of the London Mathematical Society

If you want to try it out here is our database http://www.lib.lsu.edu/...ses/journalfind.html

Jul 31, 2002, 3:13 PM

Paul

Veteran (19537 posts)

Jul 31, 2002, 3:13 PM

Post #6 of 11

Shortcut

Re: [sigrid] Still about merging fields In reply to

>>
$value =~ s/^the //g;
$value =~ s/^the//g;
<<

I've not read the whole thread so sorry if I'm just leading you astray but:

$value =~ s/^the //g;
$value =~ s/^the//g;

...could be changed to...

$value =~ s/^the\s*//ig;

Last edited by:

Paul: Jul 31, 2002, 3:13 PM

Aug 1, 2002, 8:35 AM

Fabianope

Novice (7 posts)

Aug 1, 2002, 8:35 AM

Post #7 of 11

Shortcut

Re: [Paul] Still about merging fields In reply to

Hello,

before all, great great thanks to Everybody for the quick & great replies.

well, the

$value =~ s/^the\s*//ig;

works, but I MUST -at last in my customized DBman- put a " " (space) immediatley after the "the":

$value =~ s/^the \s*//ig;

If I don't make it, DBman searches for " economist" and NOT for "economist" and obviously the search is unsuccessful.

why this?

thanks again,

Fabiano

Aug 1, 2002, 8:40 AM

Paul

Veteran (19537 posts)

Aug 1, 2002, 8:40 AM

Post #8 of 11

Shortcut

Re: [Fabianope] Still about merging fields In reply to

Oops could you try changing \s* to \s+ and remove the sapce you added in?

Last edited by:

Paul: Aug 1, 2002, 8:41 AM

Aug 5, 2002, 5:58 AM

sigrid

User (107 posts)

Aug 5, 2002, 5:58 AM

Post #9 of 11

Shortcut

Re: [Paul] Still about merging fields In reply to

Paul, Out of curiousity, what is the \s+ for? I'm still learning...

The reason I had two lines (I probably only need one) is because one does have a space after the "the."

Aug 5, 2002, 7:07 AM

Paul

Veteran (19537 posts)

Aug 5, 2002, 7:07 AM

Post #10 of 11

Shortcut

Re: [sigrid] Still about merging fields In reply to

\s+ tells the regex engine to match one or more spaces. \s* will match 0 or more spaces.

Aug 5, 2002, 2:08 PM

sigrid

User (107 posts)

Aug 5, 2002, 2:08 PM

Post #11 of 11

Shortcut

Re: [Paul] Still about merging fields In reply to

OH! And I just realized with mine, if you search "theater" for example, it only searches "ater." oops. You definitely want to search the space as well, so either take out the line of mine without the space, or use Paul's suggestion. Thanks!