Gossamer Forum
Home : Products : Gossamer Links : Version 1.x :

how long should this take?

Quote Reply
how long should this take?
Hi,
We are now on our way to parsing the dmoz contents thanks to the people at GT, that gave us a fix for the parse rdf script.

But my question is this, it's now been running for over 12 hours, and it's only up to topic 110483 about one third of the way.

We are running this on a PIII 600mhz, 256mb ram SuSE 6.4 Linux

Im curious, perhaps we havn't set things up correctly, since it's taken over 12 hours to get just 1/3 of the way through.

ANy thoughts on the subject.

Harrison

"I've got if's pretty good, but that's about it"
Quote Reply
Re: how long should this take? In reply to
When I was looking for categories, one specific, and it was running through the database mostly looking, not actually parsing, the whole 600 meg (at that time) file took about 3.5-6 hours to parse on a lightly loaded Sun Sparc with 128 meg.

I haven't tried it in 6+ months, but I would bet your stats are pretty close to what a brute force type script would do.

http://www.postcards.com
FAQ: http://www.postcards.com/FAQ/LinkSQL/