Gossamer Forum
Home : Products : Gossamer Links : Development, Plugins and Globals :

Re: [webslicer] DMOZ Wizard

Quote Reply
Re: [webslicer] DMOZ Wizard In reply to
What you wants defeats the point of this plugin. This is *supposed* to be straight forward, simple, and AUTOMATED.

I can see the potential for another plugin called "DMOZ Updater" for instance, which would in essence be a spider, that simply took it's data from the DMOZ dump.

What you are describing, is the behaviour of the original GT spider. The second version was sort of different, and current versions are too difficult to set up (they need sample import filters, for a variety of things).

Actually, I wish I could locate the original GT Spider tar, as it had some really neat code in it that was sort of abandoned in later versions.

The *PROBLEM* with what you want to do, is that moving a link from one category to another, can either screw things up -- ie: duplicate insertions, or create *massive* problems in "suggesting" where the link should go.

Some sites will want to have unique URL's and SINGLE categories, some Unique URLS and multiple categories, and some will allow multiple URL's in different categories.

It depends on if the site is CATEGORY or LINK driven.

ie: if searches are #1, unique URL's are more valid, if the site is mostly browsed, proper categorization is more important.

DMOZ provides only 3 pieces of data in their dump:

Category
Title
Description

Creating such a list for matched/itemized import is not trivial, but not hard.

the problem is it's *NOT* scalable. This is why the original plugin was automated. On larger sites, the amount of comparasons would be daunting, and people would beg for automation. The automatiion is built in, and you are begging for manual system!

What size sites are you talking about ?? Over 300 links, starts to be a pain, over 1000 links starts to be a real pain, and 10,000 links is flat out unmanagable.

You are asking to page through several thousand links, when 90% of the posts on this sort of item complain about having to page through a few dozen, or a few hundred.

I'm thinking about this, but I still don't see the complete functionality, since especially for larger sites, unless existing+unchanged links are completely ignored, there is a data logging/pigging problem that starts to get insurmountable. It's sort of triplicating data -- one in DMOZ, one in your directory, now one in the "to be imported" area. This so violates database normalization, it's giving me hives <G>


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Subject Author Views Date
Thread DMOZ Wizard ccjong 9735 Mar 10, 2004, 4:57 PM
Thread Re: [ccjong] DMOZ Wizard
Andy 9650 Mar 11, 2004, 12:28 AM
Thread Re: [Andy] DMOZ Wizard
ccjong 9597 Mar 11, 2004, 1:57 AM
Thread Re: [ccjong] DMOZ Wizard
Andy 9591 Mar 11, 2004, 2:54 AM
Thread Re: [Andy] DMOZ Wizard
pugdog 9579 Mar 14, 2004, 6:11 PM
Thread Re: [pugdog] DMOZ Wizard
Andy 9543 Mar 14, 2004, 11:15 PM
Thread Re: [Andy] DMOZ Wizard
rh 9518 Mar 21, 2004, 4:33 PM
Post Re: [rh] DMOZ Wizard
Andy 9515 Mar 22, 2004, 12:07 AM
Thread Re: [pugdog] DMOZ Wizard
pugdog 9513 Mar 25, 2004, 8:18 PM
Thread Re: [pugdog] DMOZ Wizard
KevM 9449 Mar 26, 2004, 3:47 PM
Thread Re: [KevM] DMOZ Wizard
pugdog 9454 Mar 26, 2004, 4:54 PM
Thread Re: [pugdog] DMOZ Wizard
Andy 9427 Mar 27, 2004, 1:57 AM
Thread Re: [Andy] DMOZ Wizard
pugdog 9440 Mar 27, 2004, 11:18 AM
Thread Re: [pugdog] DMOZ Wizard
webslicer 9405 Mar 28, 2004, 9:33 AM
Post Re: [webslicer] DMOZ Wizard
Andy 9394 Mar 29, 2004, 1:08 AM
Thread Re: [webslicer] DMOZ Wizard
pugdog 9388 Mar 29, 2004, 6:18 AM
Thread Re: [pugdog] DMOZ Wizard
webslicer 9404 Mar 29, 2004, 7:53 AM
Post Re: [webslicer] DMOZ Wizard
Andy 9369 Mar 29, 2004, 9:04 AM
Thread Re: [webslicer] DMOZ Wizard
pugdog 9376 Mar 29, 2004, 9:07 AM
Thread Re: [pugdog] DMOZ Wizard
webslicer 9377 Mar 29, 2004, 11:53 AM
Post Re: [webslicer] DMOZ Wizard
pugdog 9354 Mar 29, 2004, 7:19 PM