Gossamer Forum
Home : Products : Gossamer Links : Development, Plugins and Globals :

Spider Plugin - Import only

Quote Reply
Spider Plugin - Import only
Spider Plugin,

Ive looked on both the gossamer spider and Andy's plugin spider. But I can really confirm if it can do what I need it for.

Im wondering if any of the two or any other plugin can do the following.

Spider, index and add links to the database.. by Admin / Myself giving the spider a URL, for example http://dir.yahoo.com/...d_Internet/Contests/ which contains the links im interested in. The spider visit the URL and pull all the URL from that page, go to those pages (no further) and pull their titles, description etc and add them into my database for me to validate them and also select a category for them.

What Im trying to do is to simply be able to add links with their description from other sites to my own database. Much like google or yahoo does..

The existing spiders does not seem to able to do this, am I wrong? It seems as a fairly simple thing to do but there isnt any automatic way to do it, the closest thing I can see is a DMOZ import, but im not willing to download a 2gig file to my server etc..
Quote Reply
Re: [demon] Spider Plugin - Import only In reply to
Hi. My one should do this. Give it a go here;

http://demo.ace-installer.com/...amp;func=spider_site

You provide a starting URL, and it then grabs the list of URL's on that page, and gives you the option to spider them for title/author/email/description etc Smile

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] Spider Plugin - Import only In reply to
Wow, great. Just what I need than. Thanks!

I will most defenately try it out then.
Quote Reply
Re: [Andy] Spider Plugin - Import only In reply to
Andy;

Have you changed your plugin?

Last I looked I could not use it exactly because it doesn't do as he asked.

He's asking for a straight import into the database, without setting values for each URL, then later to assign category and approve through the normal Links method..

Please update me on this, ok?
Quote Reply
Re: [webslicer] Spider Plugin - Import only In reply to
Nope, it still uses the same way.

I'm not sure demon is asking for what you said, as he took a look at the demo, and said that it looks ok :)

http://demo.ace-installer.com/...amp;func=spider_site

It also has the ability to show "minor" warnings for duplicates, and turn on/off the Whois checks Smile (not on that demo URL yet, as I havn't managed to find the time to update the plugin).

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!