Gossamer Forum
Home : Products : Gossamer Links : Discussions :

Indexing Database will take a year +!

Quote Reply
Indexing Database will take a year +!

I'm using links SQL 2.10 with almost a complete import of the dmoz database. Everything works fine except that it seems endless to index the database for use with the internal or the mysql search driver.

This wasn't a problem with the previous version 2.01 even if it took some time. With links SQL 2.1 the first 100 000 links works fine, it takes about an hour then it starts to take more and more time to get each link indexed. From my calculation it will take almost a year to get the complete database indexed, a week have been acceptable.

I'm not a programmer but it seems like the script may run in to an endless loop or something. Alex how can this be solved?

PIII/Dual CPU 733
RAM 512
OS: Redhat 7.2 + Apache + Mod perl

All the best, Tomas
Quote Reply
Re: [soobe] Indexing Database will take a year +! In reply to
I'm not sure it would take a year ;)

Indexing 500,000 links took several hours for a client of mine.

How long have you left it running?.....it is normal for it to take quite a few hours with that many links.

Last edited by:

RedRum: Mar 12, 2002, 2:17 AM
Quote Reply
Re: [RedRum] Indexing Database will take a year +! In reply to
Yes, I know it's time consuming, but here something for sure is wrong with links SQL 2.01 it doesn't take this amount of time. At this moment it has been running for 5,35 days and 570 000 links has get indexed, for each links it gets slower and slower. The machine were i index the database is just dedicated to this job.

Have your client been using version 2.1 or version 2.0? With version 2.0 I had no problems.....