Paul,
On a fast machine, with enough RAM, the DMOZ import takes only about 10 hours. ON a multi-cpu machine it might take less.
The speed issues on import are directly related to hardware, and what else the cpu is doing during the import.
If I got a little more revenue support from the forums here, either in module registrations or consulting work, I could have a separate machine here doing the DMOZ import on a weekly basis. Exporting the data from links, into a links-importable database is not as much of a problem -- would take a bit of work, but it could be done. It's a matter of not putting ID"s on the "links" but merging the categories in to an existing database, or starting the category count at the next ID for the categories, and updating the "CatLinks" table accordingly.
I need about $2k of direct revenue from this forum (ie: Links SQL consulting, registrations, support, install, etc) in order to offer this service. I'm only about halfway there at this point.
From our (PUGDOG Enterprises, Inc) point of view, I need to set up this sort of machine, and export capability, because I want to set up for display and sale about 300+ Links/Directory sites over the next 6 months.
In order to do that, there are still some bugs/quirks in Links 2.x that need to be worked out, and I have to have the file upload/postcards, and hopefully the Image Gallery mods all in operation.
But the offshoot is that for very reasonable fees, (on the order of $15 to $100) I could offer portions of DMOZ in Links-Digestable/importable, even UPDATABLE formats!!
This is the last project on my list of "must do's" becuase it's such a demand thing, if I wait long enough maybe other people will start off code that can help, and because for the short term I can' do it all via interactive SQL commands on a fast machine in the middle of the night.
But, automating the process for people to do key-word or category select downloads/imports, etc is something I don't have time for, _UNLESS_ there is a direct revenue stream.
Unfortunately, because of my restrictions on time owed to my primary job, I can't do this unless the revenue is already in place -- or it is part of my primary job.
I figure in about 6 months I'll have the "time" to do this as part of my primary job. The more revenue generated here, the higher up I can push this, and the sooner it will get done.
It's an economic reality. None of us -- except GT -- is getting directly paid for anything done here. It's all back-end, side deals or profits off our running sites -- and we run FREE sites! So, people who need programming or services are going to have to pay their own way -- there are no advertisers doing it any more!
Make sense?
It's just reality.
I'll have the 01.02.xx version of the graphics/logo mod out today or tomorrow, and will freeze the features. I will start with the 02.xx multiple attachment version right after that, and the postcards plug-in should be released this week as well.
Hopefully that will help my situation, but right now I'm time-crunched, and revenue starved from all of this "new" programming.
AND!!! Don't forget, I've agreed (offered, foolishly raised my hand) to coordinate and start off the Ratings/Review mod, as a group project/learning experience -- and this may be the grandest project I've ever undertaken. The possibilities are endless!! (even making it a Links specific chunk of code!).
PUGDOGŪ Enterprises, Inc.
FAQ:
http://LinkSQL.com/FAQ Forum:
http://LinkSQL.com/forum
