Does anyone here use a spidering program to gather links.. Like "Links Suite III"? Is there one that works well with Links 2.0? How about one that works well for specific search engines instead of random links (say for example.. one that spiders by keyword)?

Any advise before I purchase Links Suite III would be appriciated...

Thanx guys..

As discussed as recent as the past week in this forum....

1) Use Links Spider Pro...linked in the Resources section.

-OR-

2) Use goFetch, which can be found at:

http://lookhard.hypermart.net/links-mods/

-OR-

3) Use the DMOZ-RDF DUMP MOD linked in this forum.

-OR-

4) Upgrade to Links SQL, which includes a robust SPIDER PLUGIN that will do exactly what you want.

Regards,

Eliot Lee

Yeah I put together one a while back for a site using LWP. It's not that hard to create and was useful as it added relevent links directly into links.db.

Glenn

Links 2 Mods Site:
http://cgi-resource.co.uk/pages/links2mods.shtml

How did you do it if you don't mind me asking?

Did it perform a search at a search engine first or did it work some other way?

Installs:http://wiredon.net/gt
FAQ:http://www.perlmad.com

Prob not the best way but worked fine for me...:

Yeah it performed a search first and took the results, parsed them and added them directly to the links.db

I set it up so as first the user enters a keyword and selects a category to add the links to. Then submits. A results page appears with the titles of those spidered. There's a checkbox where they could delete any of these if they don't like them. There's also a button they can click on to spider the next page of results.

I have been meaning to post it but haven't got round to it yet.

Glenn

Links 2 Mods Site:
http://cgi-resource.co.uk/pages/links2mods.shtml

That sounds great. Only thing I would be concerned with, (unless I mistook "User") if the fact that a user could add spidered content directly to the links.db. I would set it up so a Links Admin (from the admin page) could add directly to the links.db or I would set up a second validate.db and call it spidered.db or such.

Does you mod search from within a certain engine? Could it be setup to manually spider different engines or perhaps other webpages?

Sounds like a great mod. If it's even partitially working I'd like to see it posted.. :)

[I ended up ordering Link Suite III for $25.00, it works ok, but could be better. It does have a "Output to GT Links 2.0 Addon at least.]

It was set up to work via links admin, I figured a seperate db is not needed as the user can simply delete the links there and then via the select box. It searches eitehr Dmoz or Excite but could be altered to do others. Will prob post it at some point when I get the time.

Glenn

Links 2 Mods Site:
http://cgi-resource.co.uk/pages/links2mods.shtml

I use Innerprise URL Spider Pro. It is a costly application, but it serves the purpose. You can spider and use the program to create your links.db for an initial installation, or you can customize it to save as a validate.db in which you decide the validation. This will enable you to increase your hits to your site by providing constant updated links to your site. Users can add their links as well as you. I love it! You can see it in action at http://search.rootssearch.net I'm spidering the net right now, and will add as much as I can each day/week.

Ivana

Like I already mentioned in my reply...

Regards,

Eliot Lee