Gossamer Forum
Home : Products : Gossamer Links : Discussions :

A spider to index all my links and return results based on that...

Quote Reply
A spider to index all my links and return results based on that...
quick question. I want to give people the option to search my web directory and also for them to do a web search based on the links in my web directory.

I have something like 98,000 links. I want people to be able through my site to search all the pages that are contained within those pages. So for example, if someone enters "fish" they can see all the sites indexed whever there is the term fish. So for the web directory there may be say 50 results, but for the web search results it may be much more because it is based on what content is contained within the links.

I guess I am kind of asking for something akin to a google or a gigablast. Is there a script that can do this, has anyone tried something similar.

thanks

Leslie
Quote Reply
Re: [bunder] A spider to index all my links and return results based on that... In reply to
GT had a spider that did that, or something similar.

They don't seem to be supporting or promoting it any more. If you contact them, they may have an answer.

I *really* would like this sort of plugin, and never was able to get the spider to do it, but I had a lot going on at the time.

In short, having a spider go out, get the page, "cache" it, and then serve it up if an error occurs would be great. Then, having it parse all the links, etc and "google-ize" it, would be great too.

It's on my list, since LWP and CPAN have many of the modules necessary to do this, at least in a kludgy way. Making it "elegant" is another story :)


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.