Gossamer Forum
Home : Products : Gossamer Links : Discussions :

HELP: Want to import ALL dmoz data

Quote Reply
HELP: Want to import ALL dmoz data
I've seen several postings here concerning the import of the dmoz.org .rdf dump and I have a few questions.

I would like to import ALL of the dmoz data, 600-700 MB of .rdf data and need help with the command line.

I've got the content.rdf.gz file (Linux machine) but it doesn't appear to be gzipped at all. The file size is 139 MB and had a .gz extension. If I CAT out the file it displays text, not compressed data. The FILE command also tells me that it's ASCII text. This is odd to me. If I get the file to my home PC, the .gz is the same file size as my Linux machine but uncompressed to 702 MB. No big deal, it could be something on the Unix machine.

The question I have is:
How do I import ALL of the dmoz data? I will more than likely import twice a month to keep the links fresh. After I populate the database once (with all data), do I need to re-import all of the data or can I provide a flag to only import the data that is NEW or has CHANGED?

Please provide command line parameters for both. I'm new to this and would appreciate if it could be spelled out exactly as needed. Once for FULL import and once for INCREMENTAL import.

Thank you !!!

Subject Author Views Date
Thread HELP: Want to import ALL dmoz data lisco 4176 Feb 2, 2001, 8:58 AM
Thread Re: HELP: Want to import ALL dmoz data
Paul 4085 Feb 2, 2001, 12:15 PM
Thread Re: HELP: Want to import ALL dmoz data
pugdog 4089 Feb 2, 2001, 12:49 PM
Post Re: HELP: Want to import ALL dmoz data
Paul 4076 Feb 2, 2001, 12:56 PM
Thread Re: HELP: Want to import ALL dmoz data
lisco 4081 Feb 2, 2001, 5:30 PM
Thread Re: HELP: Want to import ALL dmoz data
Paul 4084 Feb 2, 2001, 5:34 PM
Thread Re: HELP: Want to import ALL dmoz data
lisco 3978 Feb 5, 2001, 2:22 PM
Thread Re: HELP: Want to import ALL dmoz data
Robert_B 3969 Feb 5, 2001, 2:34 PM
Post Re: HELP: Want to import ALL dmoz data
Paul 3978 Feb 5, 2001, 2:44 PM