Gossamer Forum
Home : Products : Gossamer Links : Discussions :

DMOZ Extractor -> LSQL 2.1.0

Quote Reply
DMOZ Extractor -> LSQL 2.1.0
Here is some information on how I was able to get DMOZ Extractor / Links III files to import to LSQL 2.1.0, without using Links 2, for anyone who is interested.

The example is using DMOZ Extractor (DE):

1. With DE, browse the top level of the area you want to extract i.e. Arts->Music->Bands and Artists

2. Double click Auto Extact Set Category, and set the top level category to Bands and Artists

3. Once DE has finished extracting, OUTPUT-> Gossamer Links 2


This produces categories.db, links.db and url.db, plus two text files.


4. Move these files to a directory on your server such as /import/data.... you may need to move the .def files (which come with LSQL and define the tables) to the /import directory also.

5. In LSQL Admin goto Database -> Import/Export and set as follows (not sure why but this worked for me)


Import from: Links 2.x

Location of def files: pathtowebroot/import

Checked: Show Errors, Create Missing Categories, Extra Data Integrity, rest unchecked


6. Click IMPORT, and enjoy!

This worked for me, and I would love to hear if anyone has worked out a better way or has improvements on what I have done here... especially a way of solving the related categories.... (i could not import this information).

The reason why I use DMOZ extractor, is I dont have shell/telnet access with my host, and I do not know how to import the RDF file otherwise. I would also love to know how to do this, especially if it kept the category relations in tact!


http://www.iuni.com/...tware/web/index.html
Links Plugins

Last edited by:

sooke: Apr 4, 2002, 11:56 AM
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
I should mention, that after you have done the import, you will need to run a few SQL functions to tidy things up. (otherwise you will get errors when you try to move links).

Example:

UPDATE lsql_Links SET isPopular = 'No' WHERE isPopular != 'Yes';

There are a couple of others, I just don't remember them off hand.

Also, run 'Repair Table' before doing any of the SQL. Once everything is looking good you will need to do a 'Rebuild Search' for you search tables to pick up the newly imported links and categories.

Wink


http://www.iuni.com/...tware/web/index.html
Links Plugins

Last edited by:

Ian: Jul 1, 2002, 2:41 PM
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
>>
UPDATE lsql_Links SET isPopular = 'No' WHERE isPopular != 'Yes';
<<

Is that query right?...it doesn't seem to make sense to me Unsure
Quote Reply
Re: [Paul] DMOZ Extractor -> LSQL 2.1.0 In reply to
Hmmm, I wrote it from my not-so-good memory.

It is supposed to set all the ispopulars to No, if they are not already equalling yes... in other words... don't mess with the ones which say yes already....

Its probably wrong, but it worked way back when I did this import ie it made the error go away without messing with my existing links...Crazy


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
UPDATE lsql_Links SET isPopular = 'No' WHERE isPopular = 'Yes';

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
Ok I see what you mean now, it just looked strange at first.

I just checked my links table and all links are either Yes or No by default though as it is an ENUM Yes/No column.
Quote Reply
Re: [Paul] DMOZ Extractor -> LSQL 2.1.0 In reply to
Yeah, and after this type of import, you have to do this to the required columns, so they have a value set before links looks at them and produces an error.

I think my logic was correct... it should be !='Yes';


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
Perhaps = '' would be quicker so it doesn't try to update columns already set as no.
Quote Reply
Re: [Paul] DMOZ Extractor -> LSQL 2.1.0 In reply to
What about blank columns?


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
A am also using the DMOZ Extractor (Links III), but I am setting up a regional directory and do not need the links. Is there a way using this method of just importing the categories. I would like to use the category structure.

(I tested your method of importing demoz, Cats & Likks, and it also works for me.)
Quote Reply
Re: [jgkiefer] DMOZ Extractor -> LSQL 2.1.0 In reply to
Hi,

It has been quite a while since I used this software. I would guess that you could import both (links and categories) then just delete all the links from these categories (this could be done with SQL).


http://www.iuni.com/...tware/web/index.html
Links Plugins
Quote Reply
Re: [Ian] DMOZ Extractor -> LSQL 2.1.0 In reply to
Seems like an awful lot of links to manipulate for deletion. I wonder if using the method in the above posts that I would be able to import the links2 category.db with an empty links.db. Would that just setup the category structure? Or better yet is there a way to use the DMOZ rdf and just import the categories? (Not the links) All I am trying to do is import the directory category structure.
Quote Reply
Re: [jgkiefer] DMOZ Extractor -> LSQL 2.1.0 In reply to
I recently had to do a c omplete import of the DMOZ categories for someone....he didn't want the links, so I just had to delete them all after it was done..

It was a lot more work than I would have liked, but it did the job Smile

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] DMOZ Extractor -> LSQL 2.1.0 In reply to
Is there an MySQL command that would delete the un-needed information. Seems like it would be easier to do this directly to the db.
Quote Reply
Re: [jgkiefer] DMOZ Extractor -> LSQL 2.1.0 In reply to
This seemed to do the job for me;

DELETE FROM lsql_Links;
DELETE FROM lsql_CatLinks;

Hope that helps.

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!