Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Do text files get parsed into Links_Score_List?

Quote Reply
Do text files get parsed into Links_Score_List?
Hi:

When I import a text file of info into my Links Database (via MySQLMan) does it update Links_Score_List & Links_Word_List... or do I need to rebuild Search?

thanks!

dave

Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
No...you must re-index your Links and Category tables.

Regards,

Eliot Lee
Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
Eliot:

Oh, I was afraid of that. That takes a good 3-4 hours....

dave

Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
Run it via telnet...and it will take a lot less time...also if you run it via Crontab, it can run in the background.

Regards,

Eliot Lee
Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
Eliot:

I am (via telnet), and it is taking even longer than I thought it would. I have a background telnet process doing it (perl nph-index.cgi &). I started at 8 AM, it is now 1:30...

Of course, I have 20 or so fields in each record to parse, and it looks like 18.5 megs of data/23,000 records, so it will take a while. See, that is why I was hopoing not to do this!

Maybe I will make it a once a week cron job Sunday at 2 AM....

dave

Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
I would recommend changing your weight index values for your fields. It is really UNNECESSARY to index all 20 fields in your LINKS table. You should set the most prevalent fields to 3 and others to 1 or 2. Setting all fields to 3 will eat up CPU and Memory in servers and really a waste if you ask me!

Regards,

Eliot Lee
Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
Eliot:

I have tried my best to "hone" down my list of weighted fields, and have been using this:

1 ID
2 Title 9
3 URL
4 LinkOwner
5 Add_Date
6 Mod_Date
7 Description
8 Hits
9 isNew
10 isChanged
11 isPopular
12 isValidated
13 Rating
14 Votes
15 Status
16 Date_Checked
17 Timestmp
18 Characters 6
19 Awards
20 Image
21 Prod_Num
22 Animator 4
23 Production_Notes 2
24 Voice 5
25 Director 5
26 Writer 3
27 Producer 3
28 Music 4
29 Editor 2
30 Studio 2
31 Distributor 1
32 Network 1
33 Format
34 Running_Time
35 Theatrical
36 CategoryNum
37 Country 1


Title is at 9, and the rest are in decending order, 6 through 1. Given that this is a bit different than a typical links installation... But I cannot see droping any further fields!

The reindex ended up taking 9 1/2 hours... I will just cron it at 1 AM on Sunday and forget about it!

dave

Quote Reply
Re: Do text files get parsed into Links_Score_List? In reply to
Uh...9???

That is way too high...I've found that going up to 3 on fields is sufficient. Again, like I mentioned REDUCE the weight values and your indexing process will be faster and also you will reduce the amount of extraneous index data within your MySQL app.

Regards,

Eliot Lee