I am working on the dmoz .rdf import. The file size of the .rdf is in the 700 MB range (uncompressed). The import has been running for 6 days now and no where near completion. Last check, Catlinks are at 667,426 and Categories are at 54,205.
My question is a matter of performance. Does this at all sound odd that the import has been running for 6 days and only 1/4 of the way complete? At this rate, were talking 2-3 weeks for total import (fingers are crossed that it doesn't unexpectedly terminate along the way).
CPU resources are averaging 90-95% and memory only 10-12%. This leads me to believe it's not a performance issue on my end as the memory isn't strapped and now using disk cache.
The machine: 400 MHZ PII, 196 MB Ram, newest Perl, newest MySQL and mod_perl.
Any ideas on how to increase performance and lower import time?
The import will soon be running for an entire week. Does this sound odd?
My question is a matter of performance. Does this at all sound odd that the import has been running for 6 days and only 1/4 of the way complete? At this rate, were talking 2-3 weeks for total import (fingers are crossed that it doesn't unexpectedly terminate along the way).
CPU resources are averaging 90-95% and memory only 10-12%. This leads me to believe it's not a performance issue on my end as the memory isn't strapped and now using disk cache.
The machine: 400 MHZ PII, 196 MB Ram, newest Perl, newest MySQL and mod_perl.
Any ideas on how to increase performance and lower import time?
The import will soon be running for an entire week. Does this sound odd?
DMOZ import help